[PHP-INTERNALS] Linking and autoloading of dependent extensions

Hi,
This is a problem hurting me for a while and I'm happy you're looking
into it! I'm not sure about the solution here, though. This solution
assumes

       * the extension is inside extension_dir and
       * a specific naming scheme on the extension's shared library file.

Both assumptions might be wrong and might lead to loading the wrong file
when different ones are around. This will also lead to a misleading
error message ("PHP Warning: Module 'foo' already loaded in Unknown on
line 0") where the users just tries to load it once in wrong order.

I'm not sure this is better than completely offloading it to the user,
as we do now.

johannes

Hey

I once worked on extension improvements with Joe.
We stopped at extension dependency management, as this needs deep rethink.
I plan to write an RFC and work on the subject for PHP-Next, perhaps we
could merge our ideas ?
Code is here :
https://github.com/jpauli/php-src/compare/extensions_improvements

Cheers,


Julien

Sure, I’d be glad to combine efforts. Yet, I don’t see you linking dependent extensions, so I’m not sure what your approach is to even load the top extension of the dependency chain?

Cheers,
Mike

Yeah, that was the idea we discussed but never implemented.
Beeing able to declare a weigthed heap of extensions as dependencies,
and having the system try to load them using the weigth as load
order, and recover cleanly in case on error. All should have taken
care of recursion.

Julien

Thanks Johannes, as always you’ve got a bigger picture than I have!
The warning should be easy to mitigate except for multiple extension INI entries; not sure if this is that important, though.

The second problem is a bit harder, except for source distribution installations where EXTENSION_DIR should be fine and PECL installs where I could use INI’s extension_dir.
For relocated extension_dirs like Linux dist packages, this might indeed be a problem.

Thank you,
Mike

Wait, what? I guess I read too fast. Did you see extensions not named that way?

I myself do this when keeping different versions around for testing etc.
and I've even different (commercial) extensions doing that for
distributing differently named files for different PHP versions.This
also happens when handling mixed cased names vs. lowercase (i.e. if you
build phar shared you get phar.so but it is called Phar, I assume there
are others)

johannes

file.
that way?
Well, ok... so that's absolutely not the problem I'm trying to solve. You
can still load that stubbornly named extensions manually and nothing will
complain, even if something has a dependency on it.

I'm about e.g. pecl install & going ahead without reordering extension
entries...

Cheers,
Mike

The issue is this can pull in the wrong extension giving the user a hard
time to understand how and why that happens.

And I don't think this will solve the "pecl install & going ahead" wish
- what should "pecl install" do? - Add new extensions to the end of the
file? So it will be loaded two times?

I think the solution is more in the loading and startup logic. I think
we should only load explicitly mentioned extensions (or alternatively
*all* extension in extension dir, while that has issues, too, think i.e.
Windows where we distribute a bunch of dlls which require some 3rd party
libs in proper places), then analyze the dependencies, order the startup
call accordingly and start them up.

(this all won't solve issues where lazy linking doesn't work and we need
system level symbols from other extensions, though ... but I think
systems without working RTLD_LAZY are rare these days)

johannes

Wa? pecl will download, build and install the dependencies listed in
package.xml.
It is smart enough to not add the same extension line twice. Frankly, I'm
not sure what you are talking about.
My approach is quite unintrusive, should work in 90% of cases as before,
and in 10% improved as intended. (Given I solve the install_root dilemma)

Cheers,
Mike

Whenever there are multiple versions of an extension around. This
happens for different (bad) reasons.
It is enough to add it once in the wrong place (too late) to produce a
"already loaded warning" - first the extension will be pulled in by the
new magic, then by the line from php.ini.

pecl install also won't know which php.ini file(s) to edit - some setups
use different files for cli and web server etc.
nope. php_load_extension() opensthe file, fetches the module structure
and starts it up in one go.
The could solve it by symlinking "activated" ones in a directory which
is configured as extension dir. Mentioned it mostly for brainstorming
purpose.
... and gives harder to debug behavior in case a system isn't properly
setup, which likely means there is a less experienced user, which will
have an even harder time.

johannes

solve.
extension
hard
So you are not able to provide an example either?
If you manage multiple versions of extensions, you're probably also
capable of managing your extension section to be loaded in proper manner.
wish
the
package.xml.
You obviously do not use the pecl command. It edits whichever file you tell
it to maintain.
Which is not possible for extensions that use symbols exportet from
dependent extensions, it won't dlopen.
i.e.
party
startup
need
I wouldn't call such a system not properly set up, but rather severely
fucked up :)

Re-reading the manpage it's now obvious why my first tests failed, which
caused me to abandon RTLD_LAZY. So back to the start.

Anyway, whatever I propose, I cannot fix your extension naming quirks.

Cheers, Mike

I'm assuming the idea here is to fix the chicken and egg problem of
loading pdo_mysql before pdo? or mysqlnd before mysqli? and whatever
extensions that depend on ext/json...

Big +1 from me.

But I think the extension system needs more overhaul then just this.
We need to think about the next 10 years. pecl is clearly not a tool
that can tackle the next decade of PHP.

And why oh why do we have almost 80 extensions bundled with php?
There are plenty of pecl extensions that are used by gazillion more
people then half of them.

http://bjori.blogspot.com/2014/05/i-have-dream.html

Should we try to work on new system for PHPNG?
Something that could also be more friendly to things like composer? :]

-Hannes