FAQ
The builtin reload function is very useful for iterative development, but it is also limited.? Because references to types and functions in the old version of the module may persist after reloading, the builtin reload function is typically only useful in simple use cases.

This is a proposal (pre-PEP?) for an enhanced reloading mechanism especially designed for iterative development:

https://docs.google.com/document/pub?id=1GeVVC0pXTz1O6cK5mo-EaOJFqrL3PErO4okmHBlTeuw

The basic plan is to use the existing cycle-detecting GC to remap references from objects in the old module to equivalent objects in the new module.

I have a patch against the current 3.2 branch that adds the gc.remap function (and unit tests, etc...) but not any of the additional reloading functionality. I have a separate prototype of the reloading module as well, but it only implements a portion of the proposal (one module at a time, and dicts/sets are not fixed up).

A few questions:

1) Does this approach seem reasonable?? Has anyone tried something similar and run into unsolvable problems?

2) Would there be interest in a PEP for enhanced reloading? I would be happy to rewrite the proposal in PEP form if people think it would be worthwhile.

3) Should I submit my gc.remap patch to the issue tracker?? Because the change to visitproc modifies the ABI I would like to get that portion of the proposal in before 3.2 goes final. Since the bulk of the change is adding one method to the gc module I was hoping it might be accepted without requiring a PEP.

Search Discussions

  • Chris Kaynor at Oct 19, 2010 at 7:53 pm

    On Tue, Oct 19, 2010 at 12:17 PM, Peter Ingebretson wrote:

    The builtin reload function is very useful for iterative development, but
    it is also limited. Because references to types and functions in the old
    version of the module may persist after reloading, the builtin reload
    function is typically only useful in simple use cases.

    This is a proposal (pre-PEP?) for an enhanced reloading mechanism
    especially designed for iterative development:


    https://docs.google.com/document/pub?id=1GeVVC0pXTz1O6cK5mo-EaOJFqrL3PErO4okmHBlTeuw

    The basic plan is to use the existing cycle-detecting GC to remap
    references from objects in the old module to equivalent objects in the new
    module.

    I have a patch against the current 3.2 branch that adds the gc.remap
    function (and unit tests, etc...) but not any of the additional reloading
    functionality. I have a separate prototype of the reloading module as well,
    but it only implements a portion of the proposal (one module at a time, and
    dicts/sets are not fixed up).

    A few questions:

    1) Does this approach seem reasonable? Has anyone tried something similar
    and run into unsolvable problems?

    2) Would there be interest in a PEP for enhanced reloading? I would be
    happy to rewrite the proposal in PEP form if people think it would be
    worthwhile.

    3) Should I submit my gc.remap patch to the issue tracker? Because the
    change to visitproc modifies the ABI I would like to get that portion of the
    proposal in before 3.2 goes final. Since the bulk of the change is adding
    one method to the gc module I was hoping it might be accepted without
    requiring a PEP.

    What happens if you change the __init__ or __new__ methods of an object or
    if you change a class's metaclass? It seems like those types of changes
    would be impossible to propagate to existing objects, and without
    propagating them any changes to existing objects may (are likely?) to break
    the object.


    _______________________________________________
    Python-ideas mailing list
    Python-ideas at python.org
    http://mail.python.org/mailman/listinfo/python-ideas
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101019/2764451f/attachment.html>
  • Peter Ingebretson at Oct 19, 2010 at 8:25 pm
    --- On Tue, 10/19/10, Chris Kaynor wrote:
    This is a proposal (pre-PEP?) for an enhanced reloading mechanism especially designed for iterative development:



    https://docs.google.com/document/pub?id=1GeVVC0pXTz1O6cK5mo-EaOJFqrL3PErO4okmHBlTeuw



    The basic plan is to use the existing cycle-detecting GC to remap references from objects in the old module to equivalent objects in the new module.

    What happens if you change the __init__ or __new__ methods of an object or if you change a class's metaclass? It seems like those types of changes would be impossible to propagate to existing objects, and without propagating them any changes to existing objects may (are likely?) to break the object.Yes, this is a limitation of the approach. ?More generally, any logic that has already runand would execute differently with the reloaded module has the potential to break things.
    Even with this limitation I think the approach is still valuable. ?I spend far less time modifying__new__ methods and metaclasses than I spend changing the implementation and API ofother class- and module-level methods.
    The issue of old instances not having members that are added in a new __init__ isproblematic, but there are several workarounds such as temporarily?wrapping the newmember in a property, or potentially the @reloadable decorator alluded to in the doc.



    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-ideas/attachments/20101019/9947798e/attachment.html>
  • Peter Ingebretson at Oct 19, 2010 at 8:46 pm
    (Sorry, I sent an html-formatted email by accident)

    --- On Tue, 10/19/10, Chris Kaynor wrote:
    This is a proposal (pre-PEP?) for an enhanced reloading mechanism
    especially designed for iterative development:

    https://docs.google.com/document/pub?id=1GeVVC0pXTz1O6cK5mo-EaOJFqrL3PErO4okmHBlTeuw

    The basic plan is to use the existing cycle-detecting GC to remap
    references from objects in the old module to equivalent objects in
    the new module.
    What happens if you change the __init__ or __new__ methods of an object
    or if you change a class's metaclass? It seems like those types of
    changes would be impossible to propagate to existing objects, and without
    propagating them any changes to existing objects may (are likely?) to
    break the object.
    Yes, this is a limitation of the approach. ?More generally, any logic that
    has already run and would execute differently with the reloaded module has
    the potential to break things.

    Even with this limitation I think the approach is still valuable. ?I spend
    far less time modifying __new__ methods and metaclasses than I spend
    changing the implementation and API ofother class- and module-level methods.

    The issue of old instances not having members that are added in a new
    __init__ is problematic, but there are several workarounds such as
    temporarily?wrapping the new member in a property, or potentially the
    @reloadable decorator alluded to in the doc.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-ideas @
categoriespython
postedOct 19, '10 at 7:17p
activeOct 19, '10 at 8:46p
posts4
users2
websitepython.org

2 users in discussion

Peter Ingebretson: 3 posts Chris Kaynor: 1 post

People

Translate

site design / logo © 2018 Grokbase