FAQ
I prepared a PEP and was wondering what your thoughts are about it:


   PEP: <pep number>
   Title: ``isimmutable(Obj)`` and/or ``ImmutableNester``
   Version: <version string>
   Last-Modified: <date string>
   Author: Frank-Rene Schaefer, fschaef at users.sourceforge.net
* BDFL-Delegate: <PEP czar's real name>
* Discussions-To: fschaef at users.sourceforge.net
   Status: Draft
   Type: <Standards Track | Informational | Process>
* Content-Type: text/x-rst
* Requires: <pep numbers>
   Created: 11-nov-2013
* Python-Version: 2.7.1
   Post-History: <dates of postings to python-list and python-dev>
* Replaces: <pep number>
* Superseded-By: <pep number>
* Resolution: <url>


General Idea
============


A built-in function 'isimmutable()' shall tell efficiently whether the object
of concern is mutable or not. That is it must reflect on the whole object tree
whether it contains mutable elements or not. For example, in the code fragment


::
     verdict_0 = isimmutable(3.14)
     verdict_1 = isimmutable((1,2,3))
     verdict_2 = isimmutable(((1,2),(2,3),(3,4)))


all verdicts are 'True' because the tested objects consist of purely immutable
components. However, the ``x`` in


::
     x = (1,(2,"abc", [1,2,3]))
     verdict = isimmutable(x)


triggers the verdict to be 'False' because ``x[1][2]`` is a list and therefore
mutable.


It may be conceivable to have a special class-type called ``ImmutableNester``
which has no write-access member functions and does not allow its derived
classes to have write-access member functions. Instead, any derived class
aggregates members at the time of construction. At this point in time, when
members are nested in the class, it is checked if the members are of subclasses
of ``ImmutableNester``.


The advantage of the ``isimmutable()`` function is that no extra class
functionality needs to be implemented. The disadvantage is that the
immutability must be checked manually and at each time the object is used. The
``ImmutableNester`` class type checks for immutability once, at construction
time and no further manual checks are necessary.


Rationale
=========


If an object is immutable then copying of it can be safely be replaced by a
setting of a reference. The principal scenario is when an instance A gives an
instance B access to some data D under the provision that B does not change it.
Therefore, B must either clone the data or it must be safe to assume that the
data cannot change, i.e. is immutable.


If the objects are large and/or many there a significant performance impact may
raise from a deepcopy or manual cloning of objects. Therefore, the
``isimmutable()`` built-in function is key for a safe implementation of
reference-instead-of-copying.


Ensuring immutability is also key for the so called 'Flyweight Design Pattern'.

Search Discussions

  • Ned Batchelder at Nov 11, 2013 at 8:55 pm

    On Monday, November 11, 2013 3:47:45 PM UTC-5, Frank-Rene Sch?fer wrote:
    I prepared a PEP and was wondering what your thoughts are about it:

    The best place to discuss proposals for changes to the Python language and library is the Python-Ideas mailing list: https://mail.python.org/mailman/listinfo/python-ideas


    There you will get in-depth discussion about the details of your proposal. Fair warning: it's very unlikely that your proposal will be adopted (most are not), but you will learn a lot about how Python works in the process. :)


    --Ned.
  • Steven D'Aprano at Nov 12, 2013 at 12:17 am

    On Mon, 11 Nov 2013 12:55:56 -0800, Ned Batchelder wrote:

    On Monday, November 11, 2013 3:47:45 PM UTC-5, Frank-Rene Sch?fer wrote:
    I prepared a PEP and was wondering what your thoughts are about it:
    The best place to discuss proposals for changes to the Python language
    and library is the Python-Ideas mailing list:
    https://mail.python.org/mailman/listinfo/python-ideas

    Actually it is recommended to get at least initial feedback here first,
    to weed out proposals like:


    "Python ought to allow function currying, like in Haskell."


    "You mean like functools.partial?"


    "Oh, never mind then."




    --
    Steven
  • Mark Lawrence at Nov 12, 2013 at 12:30 am

    On 12/11/2013 00:17, Steven D'Aprano wrote:
    On Mon, 11 Nov 2013 12:55:56 -0800, Ned Batchelder wrote:
    On Monday, November 11, 2013 3:47:45 PM UTC-5, Frank-Rene Sch?fer wrote:
    I prepared a PEP and was wondering what your thoughts are about it:
    The best place to discuss proposals for changes to the Python language
    and library is the Python-Ideas mailing list:
    https://mail.python.org/mailman/listinfo/python-ideas
    Actually it is recommended to get at least initial feedback here first,
    to weed out proposals like:

    "Python ought to allow function currying, like in Haskell."

    "You mean like functools.partial?"

    "But I don't want to do it like that, I want..."


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • Chris Angelico at Nov 12, 2013 at 1:35 am

    On Tue, Nov 12, 2013 at 11:17 AM, Steven D'Aprano wrote:
    On Mon, 11 Nov 2013 12:55:56 -0800, Ned Batchelder wrote:
    On Monday, November 11, 2013 3:47:45 PM UTC-5, Frank-Rene Sch?fer wrote:
    I prepared a PEP and was wondering what your thoughts are about it:
    The best place to discuss proposals for changes to the Python language
    and library is the Python-Ideas mailing list:
    https://mail.python.org/mailman/listinfo/python-ideas
    Actually it is recommended to get at least initial feedback here first,
    to weed out proposals like:

    "Python ought to allow function currying, like in Haskell."

    "You mean like functools.partial?"

    "Oh, never mind then."

    aka "Guido's Time Machine" situations. I think this might be one of
    them - I read the proposal and thought "Hashability should answer
    that", and random832 also posted the same. Love that time machine!


    ChrisA
  • Random832 at Nov 11, 2013 at 9:40 pm

    A built-in function 'isimmutable()' shall tell efficiently whether the
    object
    of concern is mutable or not.

    What's the benefit over attempting to hash() the object?


    copy.deepcopy already has special case for int, string, and tuples
    (including tuples that do and do not have mutable members) - could what
    you need be accomplished by overriding __copy__ and __deepcopy__ in your
    custom class to return itself if it is immutable?
  • Frank-Rene Schäfer at Nov 12, 2013 at 7:01 am
    A tuple is immutable but it may contain mutable objects. In larger
    hierarchies of objects it may become less obvious whether down
    the lines, there is some mutable object somewhere in the data tree.


    One can define a recursive function to check for immutability
    manually. However first, it may not be as efficient as if it was
    built-in. Second, the existence of a built-in function 'isimmutable'
    puts the concept of immutability some more into the spotlight.


    You might indeed implement some personal 'policy for copy/deepcopy'.
    But, how can you prevent the insertion of an object into the data
    tree which does not follow your copy/deepcopy convention? As soon
    as you allow members of type 'tuple' you must either check recursively
    or only allow ints and strings as tuple members.
  • Chris Angelico at Nov 12, 2013 at 7:12 am

    On Tue, Nov 12, 2013 at 6:01 PM, Frank-Rene Sch?fer wrote:
    A tuple is immutable but it may contain mutable objects. In larger
    hierarchies of objects it may become less obvious whether down
    the lines, there is some mutable object somewhere in the data tree.

    One can define a recursive function to check for immutability
    manually. However first, it may not be as efficient as if it was
    built-in. Second, the existence of a built-in function 'isimmutable'
    puts the concept of immutability some more into the spotlight.

    You might indeed implement some personal 'policy for copy/deepcopy'.
    But, how can you prevent the insertion of an object into the data
    tree which does not follow your copy/deepcopy convention? As soon
    as you allow members of type 'tuple' you must either check recursively
    or only allow ints and strings as tuple members.
    x=1,2,3
    hash(x)
    -378539185
    x=1,2,[3]
    hash(x)
    Traceback (most recent call last):
       File "<pyshell#424>", line 1, in <module>
         hash(x)
    TypeError: unhashable type: 'list'


    There's your recursive function!


    def isimmutable(x):
         try:
             hash(x)
             return True
         except TypeError:
             return False
  • Steven D'Aprano at Nov 12, 2013 at 11:14 am

    On Tue, 12 Nov 2013 18:12:43 +1100, Chris Angelico wrote:


    def isimmutable(x):
    try:
    hash(x)
    return True
    except TypeError:
    return False

    I'm afraid that doesn't test for immutability. It tests for hashability,
    which is different.


    No well-behaved mutable object can be hashable, but that's not to say
    that badly-behaved mutable objects won't be hashable. And every immutable
    object should be hashable, but that's not to say that some immutable
    objects might choose, for their own reasons, not to be hashable.


    So your function is subject to both false negatives and false positives.




    --
    Steven
  • Robert Kern at Nov 12, 2013 at 2:00 pm

    On 2013-11-12 11:14, Steven D'Aprano wrote:
    On Tue, 12 Nov 2013 18:12:43 +1100, Chris Angelico wrote:

    def isimmutable(x):
    try:
    hash(x)
    return True
    except TypeError:
    return False
    I'm afraid that doesn't test for immutability. It tests for hashability,
    which is different.

    I am going to nitpick below for nitpicking's sake, but I agree with this.

    No well-behaved mutable object can be hashable, but that's not to say
    that badly-behaved mutable objects won't be hashable.

    That's not quite true. A well-behaved mutable may be (well-behaved) hashable as
    long as the allowed mutations do not affect the equality comparison. For
    example, in Python 2, all new classes are mutable by default, but they are also
    well-behaved hashable by default because their equality comparison is identity
    comparison. None of the mutations affect object identity, so the hash based on
    identity remains well-behaved.

    And every immutable
    object should be hashable, but that's not to say that some immutable
    objects might choose, for their own reasons, not to be hashable.

    I would also dispute this. A tuple itself is immutable, but it may not be
    hashable because one of its contained objects is unhashable (whether due to
    mutability or something else).

    So your function is subject to both false negatives and false positives.

    Agreed.


    --
    Robert Kern


    "I have come to believe that the whole world is an enigma, a harmless enigma
       that is made terrible by our own mad attempt to interpret it as though it had
       an underlying truth."
        -- Umberto Eco
  • Frank-Rene Schäfer at Nov 12, 2013 at 9:12 am
    (1) hash()-ability != immutability (!)


    Proof:


    class X:
         def __hash__(self): return 0


    def pseudo_isimmutable(this):
         try:
             hash(this)
             return True
         except TypeError:
             return False


    shapeshifter = (1, 2, X())
    print pseudo_isimmutable(shapeshifter)
    shapeshifter[2].changed = 4711




    (2) The intended scenario is not described by a fragment such as:


           if isimmutable(obj): x = obj
           else: x = copy.copy(obj)
           function_that_might_modify(x)


    But instead, a more characteristic scenario is


          assert isimmutable(obj)
          # What happens behind the curtain may rely on referencing
          things_behind_the_curtain(obj)


    Or,


         def let_me_know():
              obj = get_what_is_wanted()
              assert isimmutable(obj)
              # The caller may do with it what he wants without risking consistency
              return obj


    where lots of copying










    2013/11/11 <random832@fastmail.us>:
    A built-in function 'isimmutable()' shall tell efficiently whether the
    object
    of concern is mutable or not.
    What's the benefit over attempting to hash() the object?

    copy.deepcopy already has special case for int, string, and tuples
    (including tuples that do and do not have mutable members) - could what
    you need be accomplished by overriding __copy__ and __deepcopy__ in your
    custom class to return itself if it is immutable?

    2013/11/11 <random832@fastmail.us>:
    A built-in function 'isimmutable()' shall tell efficiently whether the
    object
    of concern is mutable or not.
    What's the benefit over attempting to hash() the object?

    copy.deepcopy already has special case for int, string, and tuples
    (including tuples that do and do not have mutable members) - could what
    you need be accomplished by overriding __copy__ and __deepcopy__ in your
    custom class to return itself if it is immutable?
  • Chris Angelico at Nov 12, 2013 at 9:21 am

    On Tue, Nov 12, 2013 at 8:12 PM, Frank-Rene Sch?fer wrote:
    (1) hash()-ability != immutability (!)

    Proof:

    class X:
    def __hash__(self): return 0

    x == y != y == x


    Proof:


    class X:
        def __eq__(self,other): return True
    class Y:
        def __eq__(self,other): return False


    All you've done is proven that you can subvert things. By fiddling
    with __hash__, __eq__, and so on, you can make sets and dicts behave
    very oddly. Means nothing.


    Fundamentally, your mutability check is going to need some form of
    assistance from user-defined classes. That means a class can break
    your rules.


    ChrisA
  • Frank-Rene Schäfer at Nov 12, 2013 at 9:39 am

    All you've done is proven that you can subvert things. By fiddling
    with __hash__, __eq__, and so on, you can make sets and dicts behave
    very oddly. Means nothing.

    To the contrary, it means everything about what 'isimmutable' could
    contribute: security against advert or inadvert insertion of mutable objects.




    2013/11/11 <random832@fastmail.us>:
    A built-in function 'isimmutable()' shall tell efficiently whether the
    object
    of concern is mutable or not.
    What's the benefit over attempting to hash() the object?

    copy.deepcopy already has special case for int, string, and tuples
    (including tuples that do and do not have mutable members) - could what
    you need be accomplished by overriding __copy__ and __deepcopy__ in your
    custom class to return itself if it is immutable?
  • Chris Angelico at Nov 12, 2013 at 9:44 am

    On Tue, Nov 12, 2013 at 8:39 PM, Frank-Rene Sch?fer wrote:
    All you've done is proven that you can subvert things. By fiddling
    with __hash__, __eq__, and so on, you can make sets and dicts behave
    very oddly. Means nothing.
    To the contrary, it means everything about what 'isimmutable' could
    contribute: security against advert or inadvert insertion of mutable objects.

    So how do you figure out whether something's immutable or not? Are you
    going to ask the object itself? If so, stick with __hash__, and just
    follow the rule that mutable objects aren't hashable - which is, if
    I'm not mistaken, how things already are. And if not, then how? How
    will you know if something has mutator methods?


    ChrisA
  • Random832 at Nov 12, 2013 at 3:48 pm

    On Tue, Nov 12, 2013, at 4:39, Frank-Rene Sch?fer wrote:
    All you've done is proven that you can subvert things. By fiddling
    with __hash__, __eq__, and so on, you can make sets and dicts behave
    very oddly. Means nothing.
    To the contrary, it means everything about what 'isimmutable' could
    contribute: security against advert or inadvert insertion of mutable
    objects.

    If an object can lie about its hashability, it can lie to your function
    too... unless you don't intend to provide a way for a _genuinely_
    immutable class to say so.
  • Frank-Rene Schäfer at Nov 12, 2013 at 11:10 am

    So how do you figure out whether something's immutable or not? Are you
    going to ask the object itself? If so, stick with __hash__, and just
    follow the rule that mutable objects aren't hashable - which is, if
    I'm not mistaken, how things already are. And if not, then how? How
    will you know if something has mutator methods?

    Admittedly, I have no knowledge about the python implementation. A possible
    way would be to say:


         def isimmutable(this):
              if isinstance(this, tuple):
                   for x in this:
                       if not isimmutable(x): return False
                   return True
              return isisintance(this, (int, str, ImmutableNester))


    The ImmutableNester special class type would be a feature to help checks
    to avoid recursion. Objects of classes derived from ImmutableNester have no
    mutable access functions and allow insertion of members only at construction
    time. At construction time it checks whether all entered elements are immutable
    in the above sense.


    As said, I have no idea how much this fits into the general python
    implementation.




    2013/11/11 <random832@fastmail.us>:
    A built-in function 'isimmutable()' shall tell efficiently whether the
    object
    of concern is mutable or not.
    What's the benefit over attempting to hash() the object?

    copy.deepcopy already has special case for int, string, and tuples
    (including tuples that do and do not have mutable members) - could what
    you need be accomplished by overriding __copy__ and __deepcopy__ in your
    custom class to return itself if it is immutable?
  • Mark Lawrence at Nov 12, 2013 at 11:06 pm

    On 12/11/2013 11:10, Frank-Rene Sch?fer wrote:
    Admittedly, I have no knowledge about the python implementation.

    There is no "the" regarding Python implementations. Cpython alone is at
    either 2.7.6 or 3.3.3 with 3.4 at alpha, then there's IronPython,
    Jython, PyPy and lots more that I'm sure Steven D'Aprano can probably
    list from the top of his head :)


    --
    Python is the second best programming language in the world.
    But the best has yet to be invented. Christian Tismer


    Mark Lawrence
  • Steven D'Aprano at Nov 12, 2013 at 11:33 am

    On Tue, 12 Nov 2013 08:01:19 +0100, Frank-Rene Sch?fer wrote:


    the existence of a built-in function 'isimmutable' puts the concept of
    immutability some more into the spotlight.

    That is an argument against the proposal, not in favour. The concept of
    immutability doesn't need to be in the spotlight. It is rather
    unimportant. I've been using Python for over 15 years, and have never
    missed an isimmutable function. Once, when I was just starting with
    Python, I thought I'd try writing one. I found it harder than I expected,
    and less useful, and soon gave up. And have never missed it yet.



    You might indeed implement some personal 'policy for copy/deepcopy'.
    But, how can you prevent the insertion of an object into the data tree
    which does not follow your copy/deepcopy convention?

    I don't understand what this "policy" is supposed to be.



    As soon as you
    allow members of type 'tuple' you must either check recursively or only
    allow ints and strings as tuple members.

    Why do you think you need to check at all? I think this is where we are
    talking past each other -- you seem to believe that testing for
    immutability is a critical piece of functionality which is missing from
    Python, as if lists had no way to query their length, or floats had no
    way to do multiplication. But that is not the case. Python has no
    isimmutable built-in function because, for the 20+ years that Python has
    existed, nobody who wanted it was willing to do the work to write it, and
    nobody willing to do the work thought it was important.


    I believe that if you wish this PEP to go anywhere, you need to
    concentrate on two things:


    1) demonstrating that checking for immutability is *necessary*


    2) demonstrating that it is *possible*




    --
    Steven
  • Duncan Booth at Nov 12, 2013 at 3:50 pm

    =?UTF-8?Q?Frank-Rene_Schäfer?= wrote:


    The ImmutableNester special class type would be a feature to help
    checks to avoid recursion. Objects of classes derived from
    ImmutableNester have no mutable access functions and allow insertion
    of members only at construction time. At construction time it checks
    whether all entered elements are immutable in the above sense.

    How does this help anything? If the objects are all immutable the object
    cannot contain any recursive references.


    If you cannot see this think about tuples: a tuple containing immutable
    objects including other tuples can never contain a reference to itself
    because by definition the tuple did not exist at the point where the
    elements it contains were constructed.


    Python already relies on the non-recursive nature of nested tuples when
    handling exceptions: The expression in the 'except' clause "is compatible
    with an exception if it is the class or a base class of the exception
    object or a tuple containing an item compatible with the exception".


    If you try using something like a list in the exception specification you
    get a TypeError; only tuples and exception classes (subclasses of
    BaseException) are permitted. This means the structure can be as deeply
    nested as you wish, but can never be recursive and no checks against
    recursion need to be implemented.
  • Steven D'Aprano at Nov 12, 2013 at 1:38 am
    Hi Frank-Rene, and welcome. Comments below.




    On Mon, 11 Nov 2013 21:47:45 +0100, Frank-Rene Sch?fer wrote:

    I prepared a PEP and was wondering what your thoughts are about it:

    PEP: <pep number>
    Title: ``isimmutable(Obj)`` and/or ``ImmutableNester`` [...]
    * Python-Version: 2.7.1

    That won't do. Python 2.7 is in maintenance mode, it will not gain any
    new functionality. There won't be a Python 2.8 either. If you want to
    propose new functionality, it will have to go into 3.5. (You've missed
    the opportunity for 3.4, since "feature-freeze" is only weeks away.)



    General Idea
    ============

    A built-in function 'isimmutable()' shall tell efficiently whether the
    object of concern is mutable or not. That is it must reflect on the
    whole object tree whether it contains mutable elements or not. For
    example, in the code fragment

    This has been proposed before. It has failed because there is no way to
    tell in general whether an arbitrary object is immutable or not. If you
    only look at the common built-in types, it is quite trivial, e.g.:


    - int, str, bytes, float, frozenset, bool, None are immutable;
    - list, dict, set are not immutable;
    - tuple is immutable if all of its elements are immutable.




    But as soon as you allow arbitrary objects, you're in trouble. How do you
    tell whether an object is immutable?


    I recommend that you start by writing a reference implementation:


    def isimmutable(obj):
         ... # ?


    Some obvious thoughts:


    - It is not enough to include a big list of immutable classes:


         # don't do this
         if isinstance(obj, (float, int, frozenset, ...)):
             return True


       because that list will never be complete and can become out-of-date.


    - You could try writing to the object (how?), and if it succeeds,
       you know it is mutable. But if it fails, that might just mean
       that you haven't tried writing to it in the correct manner.


    - But if the obj is mutable, you've just mutated it. That's bad.


    - You can try hashing the object:


       hash(obj)


       If that fails, then the object *might as well* be mutable, since
       you can't use it in sets or as dict keys. But if that's all
       isimmutable() does, why not just call hash(obj) directly?




    Anyway, I recommend you spend some time on this exercise. The PEP will
    not be accepted without a reference implementation, so you're going to
    have to do it at some point.




    Another thing which your proto-PEP fails to cover in sufficient detail is
    why you think such a function and/all class would be useful. You do say
    this:

    If an object is immutable then copying of it can be safely be replaced
    by a setting of a reference. The principal scenario is when an instance
    A gives an instance B access to some data D under the provision that B
    does not change it. Therefore, B must either clone the data or it must
    be safe to assume that the data cannot change, i.e. is immutable.

    but I really don't think much of this as the principle scenario. I don't
    think I've ever written code that matches this scenario. If possible, you
    should give a real example. If not real, at least a toy example. Either
    way, using code rather than just a vague description is better.



    If the objects are large and/or many there a significant performance
    impact may raise from a deepcopy or manual cloning of objects.
    Therefore, the ``isimmutable()`` built-in function is key for a safe
    implementation of reference-instead-of-copying.

    I don't think this is true. deepcopy (at least sometimes) will avoid
    making a copy if the object is immutable:


    py> import copy
    py> x = (10001, 20002, 30003, (40004, 50005, (60006, 70007)), 80008)
    py> copy.copy(x) is x
    True
    py> copy.deepcopy(x) is x
    True


    so what advantage does isimmutable give you?





    Ensuring immutability is also key for the so called 'Flyweight Design
    Pattern'.

    More details please.




    Ultimately, nothing knows whether an object is immutable or not better
    than the object itself. copy.copy and copy.deepcopy know this, and ask
    the object to copy itself rather than copy it from the outside. Since the
    object knows whether it is immutable, it knows whether or not to make a
    copy or just return itself. It seems to me that isimmutable() *appears*
    to be a useful function to have, but if you look into it in detail the
    apparently uses for it don't hold up. In practice, it would be almost
    impossible to implement (except as below) and even if you could you would
    never need it.


    Really, it seems to me that the only way to implement isimmutable would
    be to delegate to the object:


    def isimmutable(obj):
         return obj.__isimmutable__()


    This gives you:


    if isimmutable(obj):
         x = obj
    else:
         x = copy.copy(obj)
    function_that_might_modify(x)




    but that gives you no advantage at all that the simpler:


    function_that_might_modify(copy.copy(obj))


    doesn't give. So what's the point?




    --
    Steven

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedNov 11, '13 at 8:47p
activeNov 12, '13 at 11:06p
posts20
users8
websitepython.org

People

Translate

site design / logo © 2022 Grokbase