FAQ
Hi,

I've prepared a tiny POC which enables linking and autoloading of
dependent extensions:
https://github.com/m6w6/php-src/compare/module_deps

Anybody else other than me interested in this feature?

--
Regards,
Mike

Search Discussions

  • Johannes Schlüter at May 27, 2014 at 12:52 pm
    Hi,
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    This is a problem hurting me for a while and I'm happy you're looking
    into it! I'm not sure about the solution here, though. This solution
    assumes

           * the extension is inside extension_dir and
           * a specific naming scheme on the extension's shared library file.

    Both assumptions might be wrong and might lead to loading the wrong file
    when different ones are around. This will also lead to a misleading
    error message ("PHP Warning: Module 'foo' already loaded in Unknown on
    line 0") where the users just tries to load it once in wrong order.

    I'm not sure this is better than completely offloading it to the user,
    as we do now.

    johannes
  • Julien Pauli at May 27, 2014 at 1:13 pm

    On Tue, May 27, 2014 at 2:32 PM, Johannes Schlüter wrote:
    Hi,
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    Hey

    I once worked on extension improvements with Joe.
    We stopped at extension dependency management, as this needs deep rethink.
    I plan to write an RFC and work on the subject for PHP-Next, perhaps we
    could merge our ideas ?
    Code is here :
    https://github.com/jpauli/php-src/compare/extensions_improvements

    Cheers,


    Julien
  • Michael Wallner at May 27, 2014 at 1:30 pm

    On 27 05 2014, at 15:12, Julien Pauli wrote:
    On Tue, May 27, 2014 at 2:32 PM, Johannes Schlüter wrote:
    Hi,
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    Hey

    I once worked on extension improvements with Joe.
    We stopped at extension dependency management, as this needs deep rethink.
    I plan to write an RFC and work on the subject for PHP-Next, perhaps we could merge our ideas ?
    Code is here : https://github.com/jpauli/php-src/compare/extensions_improvements
    Sure, I’d be glad to combine efforts. Yet, I don’t see you linking dependent extensions, so I’m not sure what your approach is to even load the top extension of the dependency chain?

    Cheers,
    Mike
  • Julien Pauli at May 27, 2014 at 1:59 pm

    On Tue, May 27, 2014 at 3:30 PM, Michael Wallner wrote:
    On 27 05 2014, at 15:12, Julien Pauli wrote:
    On Tue, May 27, 2014 at 2:32 PM, Johannes Schlüter wrote:
    Hi,
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    Hey

    I once worked on extension improvements with Joe.
    We stopped at extension dependency management, as this needs deep rethink.
    I plan to write an RFC and work on the subject for PHP-Next, perhaps we
    could merge our ideas ?
    Code is here :
    https://github.com/jpauli/php-src/compare/extensions_improvements


    Sure, I’d be glad to combine efforts. Yet, I don’t see you linking dependent
    extensions, so I’m not sure what your approach is to even load the top
    extension of the dependency chain?
    Yeah, that was the idea we discussed but never implemented.
    Beeing able to declare a weigthed heap of extensions as dependencies,
    and having the system try to load them using the weigth as load
    order, and recover cleanly in case on error. All should have taken
    care of recursion.

    Julien
  • Michael Wallner at May 27, 2014 at 1:29 pm

    On 27 05 2014, at 14:32, Johannes Schlüter wrote:
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    This is a problem hurting me for a while and I'm happy you're looking
    into it! I'm not sure about the solution here, though. This solution
    assumes

    * the extension is inside extension_dir and
    * a specific naming scheme on the extension's shared library file.

    Both assumptions might be wrong and might lead to loading the wrong file
    when different ones are around. This will also lead to a misleading
    error message ("PHP Warning: Module 'foo' already loaded in Unknown on
    line 0") where the users just tries to load it once in wrong order.
    Thanks Johannes, as always you’ve got a bigger picture than I have!
    The warning should be easy to mitigate except for multiple extension INI entries; not sure if this is that important, though.

    The second problem is a bit harder, except for source distribution installations where EXTENSION_DIR should be fine and PECL installs where I could use INI’s extension_dir.
    For relocated extension_dirs like Linux dist packages, this might indeed be a problem.

    Thank you,
    Mike
  • Michael Wallner at May 27, 2014 at 2:11 pm

    On 27 May 2014 15:29, Michael Wallner wrote:
    On 27 05 2014, at 14:32, Johannes Schlüter wrote:
    On Tue, 2014-05-27 at 14:13 +0200, Michael Wallner wrote:
    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    This is a problem hurting me for a while and I'm happy you're looking
    into it! I'm not sure about the solution here, though. This solution
    assumes

    * the extension is inside extension_dir and
    * a specific naming scheme on the extension's shared library file.
    Wait, what? I guess I read too fast. Did you see extensions not named that way?


    --
    Regards,
    Mike
  • Johannes Schlüter at May 27, 2014 at 2:25 pm

    On Tue, 2014-05-27 at 16:11 +0200, Michael Wallner wrote:
    * the extension is inside extension_dir and
    * a specific naming scheme on the extension's shared library file.
    Wait, what? I guess I read too fast. Did you see extensions not named that way?
    I myself do this when keeping different versions around for testing etc.
    and I've even different (commercial) extensions doing that for
    distributing differently named files for different PHP versions.This
    also happens when handling mixed cased names vs. lowercase (i.e. if you
    build phar shared you get phar.so but it is called Phar, I assume there
    are others)

    johannes
  • Michael Wallner at May 27, 2014 at 3:30 pm

    On 27 May 2014 16:25, "Johannes Schlüter" wrote:
    On Tue, 2014-05-27 at 16:11 +0200, Michael Wallner wrote:
    * the extension is inside extension_dir and
    * a specific naming scheme on the extension's shared library
    file.
    Wait, what? I guess I read too fast. Did you see extensions not named
    that way?
    I myself do this when keeping different versions around for testing etc.
    and I've even different (commercial) extensions doing that for
    distributing differently named files for different PHP versions.This
    also happens when handling mixed cased names vs. lowercase (i.e. if you
    build phar shared you get phar.so but it is called Phar, I assume there
    are others)
    Well, ok... so that's absolutely not the problem I'm trying to solve. You
    can still load that stubbornly named extensions manually and nothing will
    complain, even if something has a dependency on it.

    I'm about e.g. pecl install & going ahead without reordering extension
    entries...

    Cheers,
    Mike
  • Johannes Schlüter at May 27, 2014 at 3:47 pm

    On Tue, 2014-05-27 at 17:30 +0200, Michael Wallner wrote:

    Well, ok... so that's absolutely not the problem I'm trying to solve.
    You can still load that stubbornly named extensions manually and
    nothing will complain, even if something has a dependency on it.


    I'm about e.g. pecl install & going ahead without reordering extension
    entries...
    The issue is this can pull in the wrong extension giving the user a hard
    time to understand how and why that happens.

    And I don't think this will solve the "pecl install & going ahead" wish
    - what should "pecl install" do? - Add new extensions to the end of the
    file? So it will be loaded two times?

    I think the solution is more in the loading and startup logic. I think
    we should only load explicitly mentioned extensions (or alternatively
    *all* extension in extension dir, while that has issues, too, think i.e.
    Windows where we distribute a bunch of dlls which require some 3rd party
    libs in proper places), then analyze the dependencies, order the startup
    call accordingly and start them up.

    (this all won't solve issues where lazy linking doesn't work and we need
    system level symbols from other extensions, though ... but I think
    systems without working RTLD_LAZY are rare these days)

    johannes
  • Michael Wallner at May 27, 2014 at 4:10 pm

    On 27 May 2014 17:47, "Johannes Schlüter" wrote:
    On Tue, 2014-05-27 at 17:30 +0200, Michael Wallner wrote:

    Well, ok... so that's absolutely not the problem I'm trying to solve.
    You can still load that stubbornly named extensions manually and
    nothing will complain, even if something has a dependency on it.


    I'm about e.g. pecl install & going ahead without reordering extension
    entries...
    The issue is this can pull in the wrong extension giving the user a hard
    time to understand how and why that happens.
    Could you give an example? I can hardly imagine a likely scenario...
    And I don't think this will solve the "pecl install & going ahead" wish
    - what should "pecl install" do? - Add new extensions to the end of the
    file? So it will be loaded two times?
    Wa? pecl will download, build and install the dependencies listed in
    package.xml.
    It is smart enough to not add the same extension line twice. Frankly, I'm
    not sure what you are talking about.
    I think the solution is more in the loading and startup logic. I think
    we should only load explicitly mentioned extensions (or alternatively
    Which is the way it currently is, isn't it?
    *all* extension in extension dir, while that has issues, too, think i.e.
    Sounds crazy. Distributions also ship lots of not enabled extensions.
    Windows where we distribute a bunch of dlls which require some 3rd party
    libs in proper places), then analyze the dependencies, order the startup
    call accordingly and start them up.

    (this all won't solve issues where lazy linking doesn't work and we need
    system level symbols from other extensions, though ... but I think
    systems without working RTLD_LAZY are rare these days)
    My approach is quite unintrusive, should work in 90% of cases as before,
    and in 10% improved as intended. (Given I solve the install_root dilemma)

    Cheers,
    Mike
  • Johannes Schlüter at May 27, 2014 at 4:26 pm

    On Tue, 2014-05-27 at 18:09 +0200, Michael Wallner wrote:
    On 27 May 2014 17:47, "Johannes Schlüter" wrote:
    On Tue, 2014-05-27 at 17:30 +0200, Michael Wallner wrote:

    Well, ok... so that's absolutely not the problem I'm trying to solve.
    You can still load that stubbornly named extensions manually and
    nothing will complain, even if something has a dependency on it.


    I'm about e.g. pecl install & going ahead without reordering extension
    entries...
    The issue is this can pull in the wrong extension giving the user a hard
    time to understand how and why that happens.
    Could you give an example? I can hardly imagine a likely scenario...
    Whenever there are multiple versions of an extension around. This
    happens for different (bad) reasons.
    And I don't think this will solve the "pecl install & going ahead" wish
    - what should "pecl install" do? - Add new extensions to the end of the
    file? So it will be loaded two times?
    Wa? pecl will download, build and install the dependencies listed in package.xml.
    It is smart enough to not add the same extension line twice. Frankly,
    I'm not sure what you are talking about.
    It is enough to add it once in the wrong place (too late) to produce a
    "already loaded warning" - first the extension will be pulled in by the
    new magic, then by the line from php.ini.

    pecl install also won't know which php.ini file(s) to edit - some setups
    use different files for cli and web server etc.
    I think the solution is more in the loading and startup logic. I think
    we should only load explicitly mentioned extensions (or alternatively
    Which is the way it currently is, isn't it?
    nope. php_load_extension() opensthe file, fetches the module structure
    and starts it up in one go.
    *all* extension in extension dir, while that has issues, too, think i.e.
    Sounds crazy. Distributions also ship lots of not enabled extensions.
    The could solve it by symlinking "activated" ones in a directory which
    is configured as extension dir. Mentioned it mostly for brainstorming
    purpose.
    Windows where we distribute a bunch of dlls which require some 3rd party
    libs in proper places), then analyze the dependencies, order the startup
    call accordingly and start them up.

    (this all won't solve issues where lazy linking doesn't work and we need
    system level symbols from other extensions, though ... but I think
    systems without working RTLD_LAZY are rare these days)
    My approach is quite unintrusive, should work in 90% of cases as
    before, and in 10% improved as intended. (Given I solve the
    install_root dilemma)
    ... and gives harder to debug behavior in case a system isn't properly
    setup, which likely means there is a less experienced user, which will
    have an even harder time.

    johannes
  • Michael Wallner at May 27, 2014 at 4:58 pm

    On 27 May 2014 18:26, "Johannes Schlüter" wrote:
    On Tue, 2014-05-27 at 18:09 +0200, Michael Wallner wrote:
    On 27 May 2014 17:47, "Johannes Schlüter" wrote:
    On Tue, 2014-05-27 at 17:30 +0200, Michael Wallner wrote:

    Well, ok... so that's absolutely not the problem I'm trying to
    solve.
    You can still load that stubbornly named extensions manually and
    nothing will complain, even if something has a dependency on it.


    I'm about e.g. pecl install & going ahead without reordering
    extension
    entries...
    The issue is this can pull in the wrong extension giving the user a
    hard
    time to understand how and why that happens.
    Could you give an example? I can hardly imagine a likely scenario...
    Whenever there are multiple versions of an extension around. This
    happens for different (bad) reasons.
    So you are not able to provide an example either?
    If you manage multiple versions of extensions, you're probably also
    capable of managing your extension section to be loaded in proper manner.
    And I don't think this will solve the "pecl install & going ahead"
    wish
    - what should "pecl install" do? - Add new extensions to the end of
    the
    file? So it will be loaded two times?
    Wa? pecl will download, build and install the dependencies listed in
    package.xml.
    It is smart enough to not add the same extension line twice. Frankly,
    I'm not sure what you are talking about.
    It is enough to add it once in the wrong place (too late) to produce a
    "already loaded warning" - first the extension will be pulled in by the
    new magic, then by the line from php.ini.
    Which I already mentioned is probably possible to mitigate.
    pecl install also won't know which php.ini file(s) to edit - some setups
    use different files for cli and web server etc.
    You obviously do not use the pecl command. It edits whichever file you tell
    it to maintain.
    I think the solution is more in the loading and startup logic. I think
    we should only load explicitly mentioned extensions (or alternatively
    Which is the way it currently is, isn't it?
    nope. php_load_extension() opensthe file, fetches the module structure
    and starts it up in one go.
    Which is not possible for extensions that use symbols exportet from
    dependent extensions, it won't dlopen.
    *all* extension in extension dir, while that has issues, too, think
    i.e.
    Sounds crazy. Distributions also ship lots of not enabled extensions.
    The could solve it by symlinking "activated" ones in a directory which
    is configured as extension dir. Mentioned it mostly for brainstorming
    purpose.
    Windows where we distribute a bunch of dlls which require some 3rd
    party
    libs in proper places), then analyze the dependencies, order the
    startup
    call accordingly and start them up.

    (this all won't solve issues where lazy linking doesn't work and we
    need
    system level symbols from other extensions, though ... but I think
    systems without working RTLD_LAZY are rare these days)
    My approach is quite unintrusive, should work in 90% of cases as
    before, and in 10% improved as intended. (Given I solve the
    install_root dilemma)
    ... and gives harder to debug behavior in case a system isn't properly
    setup, which likely means there is a less experienced user, which will
    have an even harder time.
    I wouldn't call such a system not properly set up, but rather severely
    fucked up :)

    Re-reading the manpage it's now obvious why my first tests failed, which
    caused me to abandon RTLD_LAZY. So back to the start.

    Anyway, whatever I propose, I cannot fix your extension naming quirks.

    Cheers, Mike
  • Hannes Magnusson at May 27, 2014 at 5:36 pm

    On Tue, May 27, 2014 at 5:13 AM, Michael Wallner wrote:
    Hi,

    I've prepared a tiny POC which enables linking and autoloading of
    dependent extensions:
    https://github.com/m6w6/php-src/compare/module_deps

    Anybody else other than me interested in this feature?
    I'm assuming the idea here is to fix the chicken and egg problem of
    loading pdo_mysql before pdo? or mysqlnd before mysqli? and whatever
    extensions that depend on ext/json...

    Big +1 from me.

    But I think the extension system needs more overhaul then just this.
    We need to think about the next 10 years. pecl is clearly not a tool
    that can tackle the next decade of PHP.

    And why oh why do we have almost 80 extensions bundled with php?
    There are plenty of pecl extensions that are used by gazillion more
    people then half of them.

    http://bjori.blogspot.com/2014/05/i-have-dream.html

    Should we try to work on new system for PHPNG?
    Something that could also be more friendly to things like composer? :]

    -Hannes

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupphp-internals @
categoriesphp
postedMay 27, '14 at 12:13p
activeMay 27, '14 at 5:36p
posts14
users5
websitephp.net

People

Translate

site design / logo © 2022 Grokbase