FAQ
I'm looking at fixing pkgutil.extend_path in order to support namespace
packages where some portions use PEP-420 and some use extend_path.

The first thing I notice is that there are no tests for
pkgutil.extend_path :(

extend_path currently just examines the filesystem directly, which means
it doesn't support portions in zip files or other finder/loaders.

But if I understand PJE and others correctly, the idea is to modify
extend_path so it calls the path_hook finders instead of looking at the
filesystem (in order to find the other __path__ entries). This looks
like a change in functionality: previously only real filesystem packages
would be found. With this change it would include other finders (like
zip files). While it's a change, it would align well with importlib.
Personally I'm okay with this change.

There's the added issue of how to deal with .pkg files. Is only
supporting them from the filesystem okay? Or is it worth the hassle of
creating some finder API to access them?

Any thoughts?

Eric.

Search Discussions

  • PJ Eby at May 15, 2012 at 10:04 pm

    On Tue, May 15, 2012 at 1:26 PM, Eric V. Smith wrote:

    I'm looking at fixing pkgutil.extend_path in order to support namespace
    packages where some portions use PEP-420 and some use extend_path.

    The first thing I notice is that there are no tests for
    pkgutil.extend_path :(

    extend_path currently just examines the filesystem directly, which means
    it doesn't support portions in zip files or other finder/loaders.

    But if I understand PJE and others correctly, the idea is to modify
    extend_path so it calls the path_hook finders instead of looking at the
    filesystem (in order to find the other __path__ entries).

    Well, I actually wasn't trying to support that at all; I've never used
    extend_path() myself, so I was only talking about implementing transitional
    support for the PEP in declare_namespace().

    That being said, it sounds like adding support in extend_path() might also
    be worthwhile. Presumably, it would *not* auto-update, since extend_path()
    doesn't currently do that. (Just as declare_namespace() *should*
    auto-update, because it currently does.)


    This looks
    like a change in functionality: previously only real filesystem packages
    would be found. With this change it would include other finders (like
    zip files). While it's a change, it would align well with importlib.
    Personally I'm okay with this change.
    Makes sense to me.

    There's the added issue of how to deal with .pkg files. Is only
    supporting them from the filesystem okay? Or is it worth the hassle of
    creating some finder API to access them?
    I don't know who actually uses them, but then I don't know who uses
    extend_path(), period. The only examples I was able to find with Google
    and Nullege are of the form "try: declare_namespace() except:
    extend_path()" -- that is, code that first tries to use declare_namespace()
    instead.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/import-sig/attachments/20120515/0d3b0c2d/attachment.html>
  • Eric V. Smith at May 16, 2012 at 1:40 am

    On 05/15/2012 06:04 PM, PJ Eby wrote:
    On Tue, May 15, 2012 at 1:26 PM, Eric V. Smith <eric at trueblade.com
    wrote:

    I'm looking at fixing pkgutil.extend_path in order to support namespace
    packages where some portions use PEP-420 and some use extend_path.

    The first thing I notice is that there are no tests for
    pkgutil.extend_path :(

    extend_path currently just examines the filesystem directly, which means
    it doesn't support portions in zip files or other finder/loaders.

    But if I understand PJE and others correctly, the idea is to modify
    extend_path so it calls the path_hook finders instead of looking at the
    filesystem (in order to find the other __path__ entries).


    Well, I actually wasn't trying to support that at all; I've never used
    extend_path() myself, so I was only talking about implementing
    transitional support for the PEP in declare_namespace().
    I'm mostly interested in extend_path() because:
    a) it's in the standard library
    b) if it can be made to work, I assume declare_namespace() can, too
    That being said, it sounds like adding support in extend_path() might
    also be worthwhile. Presumably, it would *not* auto-update, since
    extend_path() doesn't currently do that. (Just as declare_namespace()
    *should* auto-update, because it currently does.)
    Right. I have extend_path() working with mixed namespace packages: those
    that have an __init__.py with extend_path, and those with no
    __init__.py. In this case, it does not support auto-updating (although
    it easily could, if we expose that from _bootstrap). I'll update the PEP
    to mention returning the portions iterable even if a loader is returned,
    clean up the code, and check it in.

    I'll leave declare_namespace() for someone else. I might have time to
    look at it if/when the PEP is accepted, but no promises.
    There's the added issue of how to deal with .pkg files. Is only
    supporting them from the filesystem okay? Or is it worth the hassle of
    creating some finder API to access them?


    I don't know who actually uses them, but then I don't know who uses
    extend_path(), period. The only examples I was able to find with Google
    and Nullege are of the form "try: declare_namespace() except:
    extend_path()" -- that is, code that first tries to use
    declare_namespace() instead.
    I haven't touched the .pkg code. It doesn't work terribly well if a
    zipimporter is used, but then it never did. At least I'm not making it
    worse.

    Once this is checked in, I think the work on the PEP and sample
    implementation is done.

    Eric.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimport-sig @
categoriespython
postedMay 15, '12 at 5:26p
activeMay 16, '12 at 1:40a
posts3
users2
websitepython.org

2 users in discussion

Eric V. Smith: 2 posts PJ Eby: 1 post

People

Translate

site design / logo © 2018 Grokbase