On 2012.12.16 11:57 AM, Leon Timmermans wrote:
I can agree with all of that. Actually, starting a discussion about
this was on my todo-list for the last QA hackathon but I didn't get
around to it. Ideally, it should replace not only packlists but also
I was thinking about what you said about packlists, and I wonder how much
information one could scrape out of them. Would it be enough to reconstruct
at least that a group of files belongs to a release? That would be enough to
be able to fully uninstall a requested module. For example, if the user asks
to uninstall ExtUtils::MakeMaker the database could have seen that
ExtUtils/MakeMaker.pm was in a packlist together with ExtUtils/MM_Unix.pm and
so on and uninstall them. Probably given their original purpose was to
provide an uninstaller.

Also what's with this .meta directory I see popping up? I missed a memo.

This is all totally doable, and efficient enough, with a small pile of DBM
files and Storable. Where to put the database is a bit more complicated, see
the list of open problems below.
Given that Storable's format isn't forward-compatible, something more
stable such as JSON would be more appropriate.
That's a good point about Storable. JSON requires a dependency.
ExtUtils::Install could bundle JSON::PP, but it would be simpler to use
Data::Dumper. It makes de/serialization faster and simpler. The main
disadvantage is its only readable by Perl, but that's ok since this pile of
DBM files will be opaque to everything but the Perl API. Too much of a mess
to contemplate otherwise.

* Auto-upgrade to SQLite if ExtUtils::Install::DB::SQLite is installed.

If a special module is installed we can offer SQLite support (or whatever) for
a more advanced database. At install time it would copy the existing DBM
system into its own database.

In general, more functionality can be added as more optional (or bundled)
dependencies are available to the system. Through it all the basic DBM
database would continue to be redundantly maintained to provide a fallback
should those optional modules break or go away.
Having a proper database would be really nice, but I'm not sure if
it's going to be worth the hassle if we have a robust system already.
That's the great thing about Open Source and possibilities. You just have to
make the possibilities available and let someone surprise you!

* Where to put the database? What about non-standard install locations?

$Config{archlib} would seem the obvious location, but it presents a
permissions problem. If a non-root user installs into their home
directory, you don't want them needing root to write to the installation
database. There's several ways to deal with this.

One is to simply not record non-standard install locations, but this loses
data and punishes all those local::lib users out there.

Another is to have a separate install database for non-standard install
locations. This makes sense to me, but it brings in the sticky problem
of having to merge install databases. Sticky, but still a SMOP. Once you
have to implement merging anyway, it now makes sense to have an install
database for each install location. One for core. One for vendor. One for
perl. And one for each custom location. This has a lot of advantages to
better fit how Perl layers module installs.

* allows separation of permissions
* allows queries of what's installed based on what's in @INC

That second one is important. When a normal user queries the database, they
want to get what's installed in the standard library location. When a
local::lib user queries the database, they want to get what's installed in the
standard library locations AND their own local lib.
The combination of these is problematic. You might upgrade EU::Install
in your local module path, but not have write permissions on the
system paths. In practice, we might have to support all our older
versions :-|
Erg, good point. That very likely scenario is definitely going to require
some thought.

Not perfect, but gets us off the ground. Its not a great database, but it
does the important job of recording the critical install-time data for later
use. Its implementable within the current system. It doesn't require a bunch
of dependencies, just one upgrade. It works with most existing module
releases. It solves a major design problem with the Perl module system.

I think it's a Simple(?!) Matter Of Programming in ExtUtils::Install to get it
off the ground. IMO the most important bit of coordination is putting some
thought into what the basic database should look like so we don't have to
worry about complicated upgrades later.
I'm not sure it's as simple as you make it sound, but it is a good
idea nonetheless.
Many devils in the details. Glad you like it. Thanks for looking it over.

60. "The Giant Space Ants" are not at the top of my chain of command.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army

Search Discussions

Discussion Posts


Follow ups

Related Discussions



site design / logo © 2017 Grokbase