[Python] isimmutable' and 'ImmutableNester

The best place to discuss proposals for changes to the Python language and library is the Python-Ideas mailing list: https://mail.python.org/mailman/listinfo/python-ideas


There you will get in-depth discussion about the details of your proposal. Fair warning: it's very unlikely that your proposal will be adopted (most are not), but you will learn a lot about how Python works in the process. :)


--Ned.

Actually it is recommended to get at least initial feedback here first,
to weed out proposals like:


"Python ought to allow function currying, like in Haskell."


"You mean like functools.partial?"


"Oh, never mind then."

"But I don't want to do it like that, I want..."

aka "Guido's Time Machine" situations. I think this might be one of
them - I read the proposal and thought "Hashability should answer
that", and random832 also posted the same. Love that time machine!


ChrisA

What's the benefit over attempting to hash() the object?


copy.deepcopy already has special case for int, string, and tuples
(including tuples that do and do not have mutable members) - could what
you need be accomplished by overriding __copy__ and __deepcopy__ in your
custom class to return itself if it is immutable?

A tuple is immutable but it may contain mutable objects. In larger
hierarchies of objects it may become less obvious whether down
the lines, there is some mutable object somewhere in the data tree.


One can define a recursive function to check for immutability
manually. However first, it may not be as efficient as if it was
built-in. Second, the existence of a built-in function 'isimmutable'
puts the concept of immutability some more into the spotlight.


You might indeed implement some personal 'policy for copy/deepcopy'.
But, how can you prevent the insertion of an object into the data
tree which does not follow your copy/deepcopy convention? As soon
as you allow members of type 'tuple' you must either check recursively
or only allow ints and strings as tuple members.

-378539185
Traceback (most recent call last):
   File "<pyshell#424>", line 1, in <module>
     hash(x)
TypeError: unhashable type: 'list'


There's your recursive function!


def isimmutable(x):
     try:
         hash(x)
         return True
     except TypeError:
         return False

I'm afraid that doesn't test for immutability. It tests for hashability,
which is different.


No well-behaved mutable object can be hashable, but that's not to say
that badly-behaved mutable objects won't be hashable. And every immutable
object should be hashable, but that's not to say that some immutable
objects might choose, for their own reasons, not to be hashable.


So your function is subject to both false negatives and false positives.

I am going to nitpick below for nitpicking's sake, but I agree with this.


That's not quite true. A well-behaved mutable may be (well-behaved) hashable as
long as the allowed mutations do not affect the equality comparison. For
example, in Python 2, all new classes are mutable by default, but they are also
well-behaved hashable by default because their equality comparison is identity
comparison. None of the mutations affect object identity, so the hash based on
identity remains well-behaved.


I would also dispute this. A tuple itself is immutable, but it may not be
hashable because one of its contained objects is unhashable (whether due to
mutability or something else).


Agreed.

(1) hash()-ability != immutability (!)


Proof:


class X:
     def __hash__(self): return 0


def pseudo_isimmutable(this):
     try:
         hash(this)
         return True
     except TypeError:
         return False


shapeshifter = (1, 2, X())
print pseudo_isimmutable(shapeshifter)
shapeshifter[2].changed = 4711




(2) The intended scenario is not described by a fragment such as:


       if isimmutable(obj): x = obj
       else: x = copy.copy(obj)
       function_that_might_modify(x)


But instead, a more characteristic scenario is


      assert isimmutable(obj)
      # What happens behind the curtain may rely on referencing
      things_behind_the_curtain(obj)


Or,


     def let_me_know():
          obj = get_what_is_wanted()
          assert isimmutable(obj)
          # The caller may do with it what he wants without risking consistency
          return obj


where lots of copying










2013/11/11 <[email protected]>:

2013/11/11 <[email protected]>:

x == y != y == x


Proof:


class X:
    def __eq__(self,other): return True
class Y:
    def __eq__(self,other): return False


All you've done is proven that you can subvert things. By fiddling
with __hash__, __eq__, and so on, you can make sets and dicts behave
very oddly. Means nothing.


Fundamentally, your mutability check is going to need some form of
assistance from user-defined classes. That means a class can break
your rules.


ChrisA

To the contrary, it means everything about what 'isimmutable' could
contribute: security against advert or inadvert insertion of mutable objects.




2013/11/11 <[email protected]>:

So how do you figure out whether something's immutable or not? Are you
going to ask the object itself? If so, stick with __hash__, and just
follow the rule that mutable objects aren't hashable - which is, if
I'm not mistaken, how things already are. And if not, then how? How
will you know if something has mutator methods?


ChrisA

If an object can lie about its hashability, it can lie to your function
too... unless you don't intend to provide a way for a _genuinely_
immutable class to say so.

Admittedly, I have no knowledge about the python implementation. A possible
way would be to say:


     def isimmutable(this):
          if isinstance(this, tuple):
               for x in this:
                   if not isimmutable(x): return False
               return True
          return isisintance(this, (int, str, ImmutableNester))


The ImmutableNester special class type would be a feature to help checks
to avoid recursion. Objects of classes derived from ImmutableNester have no
mutable access functions and allow insertion of members only at construction
time. At construction time it checks whether all entered elements are immutable
in the above sense.


As said, I have no idea how much this fits into the general python
implementation.




2013/11/11 <[email protected]>:

There is no "the" regarding Python implementations. Cpython alone is at
either 2.7.6 or 3.3.3 with 3.4 at alpha, then there's IronPython,
Jython, PyPy and lots more that I'm sure Steven D'Aprano can probably
list from the top of his head :)

That is an argument against the proposal, not in favour. The concept of
immutability doesn't need to be in the spotlight. It is rather
unimportant. I've been using Python for over 15 years, and have never
missed an isimmutable function. Once, when I was just starting with
Python, I thought I'd try writing one. I found it harder than I expected,
and less useful, and soon gave up. And have never missed it yet.




I don't understand what this "policy" is supposed to be.




Why do you think you need to check at all? I think this is where we are
talking past each other -- you seem to believe that testing for
immutability is a critical piece of functionality which is missing from
Python, as if lists had no way to query their length, or floats had no
way to do multiplication. But that is not the case. Python has no
isimmutable built-in function because, for the 20+ years that Python has
existed, nobody who wanted it was willing to do the work to write it, and
nobody willing to do the work thought it was important.


I believe that if you wish this PEP to go anywhere, you need to
concentrate on two things:


1) demonstrating that checking for immutability is *necessary*


2) demonstrating that it is *possible*

How does this help anything? If the objects are all immutable the object
cannot contain any recursive references.


If you cannot see this think about tuples: a tuple containing immutable
objects including other tuples can never contain a reference to itself
because by definition the tuple did not exist at the point where the
elements it contains were constructed.


Python already relies on the non-recursive nature of nested tuples when
handling exceptions: The expression in the 'except' clause "is compatible
with an exception if it is the class or a base class of the exception
object or a tuple containing an item compatible with the exception".


If you try using something like a list in the exception specification you
get a TypeError; only tuples and exception classes (subclasses of
BaseException) are permitted. This means the structure can be as deeply
nested as you wish, but can never be recursive and no checks against
recursion need to be implemented.

Hi Frank-Rene, and welcome. Comments below.




On Mon, 11 Nov 2013 21:47:45 +0100, Frank-Rene Sch?fer wrote:


That won't do. Python 2.7 is in maintenance mode, it will not gain any
new functionality. There won't be a Python 2.8 either. If you want to
propose new functionality, it will have to go into 3.5. (You've missed
the opportunity for 3.4, since "feature-freeze" is only weeks away.)




This has been proposed before. It has failed because there is no way to
tell in general whether an arbitrary object is immutable or not. If you
only look at the common built-in types, it is quite trivial, e.g.:


- int, str, bytes, float, frozenset, bool, None are immutable;
- list, dict, set are not immutable;
- tuple is immutable if all of its elements are immutable.




But as soon as you allow arbitrary objects, you're in trouble. How do you
tell whether an object is immutable?


I recommend that you start by writing a reference implementation:


def isimmutable(obj):
     ... # ?


Some obvious thoughts:


- It is not enough to include a big list of immutable classes:


     # don't do this
     if isinstance(obj, (float, int, frozenset, ...)):
         return True


   because that list will never be complete and can become out-of-date.


- You could try writing to the object (how?), and if it succeeds,
   you know it is mutable. But if it fails, that might just mean
   that you haven't tried writing to it in the correct manner.


- But if the obj is mutable, you've just mutated it. That's bad.


- You can try hashing the object:


   hash(obj)


   If that fails, then the object *might as well* be mutable, since
   you can't use it in sets or as dict keys. But if that's all
   isimmutable() does, why not just call hash(obj) directly?




Anyway, I recommend you spend some time on this exercise. The PEP will
not be accepted without a reference implementation, so you're going to
have to do it at some point.




Another thing which your proto-PEP fails to cover in sufficient detail is
why you think such a function and/all class would be useful. You do say
this:


but I really don't think much of this as the principle scenario. I don't
think I've ever written code that matches this scenario. If possible, you
should give a real example. If not real, at least a toy example. Either
way, using code rather than just a vague description is better.




I don't think this is true. deepcopy (at least sometimes) will avoid
making a copy if the object is immutable:


py> import copy
py> x = (10001, 20002, 30003, (40004, 50005, (60006, 70007)), 80008)
py> copy.copy(x) is x
True
py> copy.deepcopy(x) is x
True


so what advantage does isimmutable give you?






More details please.




Ultimately, nothing knows whether an object is immutable or not better
than the object itself. copy.copy and copy.deepcopy know this, and ask
the object to copy itself rather than copy it from the outside. Since the
object knows whether it is immutable, it knows whether or not to make a
copy or just return itself. It seems to me that isimmutable() *appears*
to be a useful function to have, but if you look into it in detail the
apparently uses for it don't hold up. In practice, it would be almost
impossible to implement (except as below) and even if you could you would
never need it.


Really, it seems to me that the only way to implement isimmutable would
be to delegate to the object:


def isimmutable(obj):
     return obj.__isimmutable__()


This gives you:


if isimmutable(obj):
     x = obj
else:
     x = copy.copy(obj)
function_that_might_modify(x)




but that gives you no advantage at all that the simpler:


function_that_might_modify(copy.copy(obj))


doesn't give. So what's the point?