FAQ
In Python 3 the following two classes should be equivalent:


$ cat foo.py
class Foo:
      def foo():
          pass
      print(callable(foo))


class Foo:
      @staticmethod
      def foo():
          pass
      print(callable(foo))


But they do not:


$ python3 foo.py
True
False


How come the metaclass does not skip the staticmethod decorator?
--
Marco Buttu

Search Discussions

  • Peter Otten at Nov 23, 2013 at 9:01 am

    Marco Buttu wrote:


    In Python 3 the following two classes should be equivalent:

    Says who?

    $ cat foo.py
    class Foo:
    def foo():
    pass
    print(callable(foo))

    class Foo:
    @staticmethod
    def foo():
    pass
    print(callable(foo))

    But they do not:

    $ python3 foo.py
    True
    False

    How come the metaclass does not skip the staticmethod decorator?

    What? Your print()s are executed inside the class body, so the classes don't
    come into play at all.


    Your script is saying that a staticmethod instance is not a callable object.
    It need not be because


    Foo.foo()


    doesn't call the Foo.foo attribute directly, it calls


    Foo.foo.__get__(None, Foo)()

    class D:
    ... def __get__(self, *args): print(args)
    ...
    class Foo:
    ... foo = D()
    ...
    Foo.foo()
    (None, <class '__main__.Foo'>)
    Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
    TypeError: 'NoneType' object is not callable


    Look for "descriptor protocol" to learn the details.
  • Antoon Pardon at Nov 23, 2013 at 3:00 pm
    Op 23-11-13 10:01, Peter Otten schreef:

    Your script is saying that a staticmethod instance is not a callable object.
    It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()

    I think you are burdening the programmer with implemantation details
    that don't matter to him.


    IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
    is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.


    --
    Antoon Pardon
  • Chris Angelico at Nov 23, 2013 at 9:38 pm

    On Sun, Nov 24, 2013 at 2:00 AM, Antoon Pardon wrote:
    IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
    is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.

    I absolutely agree. But isn't that already the case? I seem to be
    missing something here.

    class Foo:
         @staticmethod
         def foo():
             pass

    callable(Foo.foo)
    True
    callable(Foo().foo)
    True


    ChrisA
  • Peter Otten at Nov 23, 2013 at 9:51 pm

    Antoon Pardon wrote:


    Op 23-11-13 10:01, Peter Otten schreef:
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()
    I think you are burdening the programmer with implemantation details
    that don't matter to him.

    IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
    is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.

    If you read the original post -- I think in this case the details do matter.


    What is your highlevel explanation for

    class Foo:
    ... @staticmethod
    ... def foo(): pass
    ... try: foo()
    ... except Exception as err:
    ... print(err)
    ...
    'staticmethod' object is not callable
    Foo.foo()

    or maybe clearer:

    @staticmethod
    ... def foo(): pass
    ...
    def bar(): pass
    ...
    class Foo:
    ... foo = foo
    ... bar = bar
    ...
    Foo.bar is bar
    True
    Foo.foo is foo
    False


    How would you explain that without mentioning the descriptor protocol?
  • Antoon Pardon at Nov 24, 2013 at 10:30 am

    Op 23-11-13 22:51, Peter Otten schreef:
    Antoon Pardon wrote:
    Op 23-11-13 10:01, Peter Otten schreef:
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()
    I think you are burdening the programmer with implemantation details
    that don't matter to him.

    IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
    is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.
    If you read the original post -- I think in this case the details do matter.

    What is your highlevel explanation for

    I don't care about what kind of explanation. I care about a correct answer to
    the question whether a particular object is callable (from a programmers point
    of view). I'm sure you can give a very comprehensive explanation for why in
    this case we get an incorrect answer but that doesn't make the behaviour correct.


    Foo.foo() is legal here. So Foo.foo is callable. So you starting with it needn't
    be callable is using "callable" with a different meaning than the natural
    interpretation. Al the rest is just an attempt in getting others to accept your
    use of "callable" instead of the natural one.


    Foo.foo() being legal and Foo.foo not being callable is IMO a bug in python. No matter
    what explanation you have for the behaviour.


    --
    Antoon Pardon
  • Peter Otten at Nov 24, 2013 at 10:43 am

    Antoon Pardon wrote:


    Foo.foo() being legal and Foo.foo not being callable is IMO a bug in
    python.

    Foo.foo() is legal, and Foo.foo is callable.
  • Antoon Pardon at Nov 24, 2013 at 3:55 pm

    Op 24-11-13 11:43, Peter Otten schreef:
    Antoon Pardon wrote:
    Foo.foo() being legal and Foo.foo not being callable is IMO a bug in
    python.
    Foo.foo() is legal, and Foo.foo is callable.

    Indeed, I had a kink in my brain which made it difficult to see
    where I was going wrong myself. Sorry about that.


    --
    Antoon Pardon
  • Ian Kelly at Nov 24, 2013 at 10:45 am

    On Sun, Nov 24, 2013 at 3:30 AM, Antoon Pardon wrote:
    Op 23-11-13 22:51, Peter Otten schreef:
    Antoon Pardon wrote:
    Op 23-11-13 10:01, Peter Otten schreef:
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()
    I think you are burdening the programmer with implemantation details
    that don't matter to him.

    IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
    is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.
    If you read the original post -- I think in this case the details do matter.

    What is your highlevel explanation for
    I don't care about what kind of explanation. I care about a correct answer to
    the question whether a particular object is callable (from a programmers point
    of view). I'm sure you can give a very comprehensive explanation for why in
    this case we get an incorrect answer but that doesn't make the behaviour correct.

    Foo.foo() is legal here. So Foo.foo is callable. So you starting with it needn't
    be callable is using "callable" with a different meaning than the natural
    interpretation. Al the rest is just an attempt in getting others to accept your
    use of "callable" instead of the natural one.

    Foo.foo() being legal and Foo.foo not being callable is IMO a bug in python. No matter
    what explanation you have for the behaviour.

    Your supposition that Foo.foo is not considered callable by Python is
    false, as Chris already demonstrated, and I don't see where anybody
    here has stated otherwise. What Peter wrote was that "a staticmethod
    instance is not a callable object", which is absolutely correct, and
    these two facts are consistent because Foo.foo is not a staticmethod
    instance (Foo.__dict__['foo'] on the other hand *is* a staticmethod
    instance and *is not* callable, and Foo.__dict__['foo']() will
    correspondingly raise a TypeError).
  • Steven D'Aprano at Nov 24, 2013 at 2:18 pm

    On Sun, 24 Nov 2013 11:30:14 +0100, Antoon Pardon wrote:


    Foo.foo() is legal here. So Foo.foo is callable.

    Incorrect. Foo.foo() is legal for *any* identifiers Foo and foo. Since
    Python is an extremely dynamic language, the compiler cannot (easily, or
    at all) prohibit "illegal" combinations. The only combinations which are
    illegal are those which are not legal identifier, e.g.:


    while.spam
    abc$xyz.eggs


    Otherwise, any pair of legal identifiers are legal and will be accepted
    by the compiler. Only at runtime does the lookup succeed or fail:


    str.length # likely to fail, unless str has been shadowed
    str.find # likely to succeed, unless str has been shadowed


    If the lookup has succeeded, then and only then does a second lookup
    occur, namely:


    type(str.find).__call__


    and if that succeeds it is called.


    It isn't helpful to talk about function or method calls being "legal" or
    "illegal" in Python, since such things are normally determined in terms
    of *success* or *failure*, not permitted versus prohibited. Either the
    full chain of lookups and function call will succeed, or something will
    raise an exception and it will fail.



    Foo.foo() being legal and Foo.foo not being callable is IMO a bug in
    python. No matter what explanation you have for the behaviour.

    I don't know why you keep going on about this point, since this is NOT
    the behaviour the original poster is talking about. With foo decorated as
    a staticmethod in class Foo, Foo.foo *is* callable. It takes 30 seconds
    to try it yourself:




    py> class Foo(object): # inherit from object necessary in Python 2
    ... @staticmethod
    ... def foo():
    ... return "Success!"
    ...
    py>
    py> Foo.foo()
    'Success!'
    py> Foo().foo()
    'Success!'




    This is not what the OP is talking about. Try this instead:


    py> Foo.__dict__['foo']()
    Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
    TypeError: 'staticmethod' object is not callable




    What's going on here? Let's have a look:


    py> Foo.__dict__['foo']
    <sta...@...0x9d83194>
    py> Foo.foo
    <fun...@...0x9d80294>




    You CANNOT understand this behaviour without understanding the descriptor
    protocol, which is *fundamental* to Python, and has been since Python
    2.2. Regular instance methods, class methods, static methods and
    properties are all defined in terms of descriptors. Trying to understand
    them without knowledge of descriptors is like trying to understand
    generators and iterators without knowledge of StopIteration -- you can go
    only so far before you get stuck and confused.


    The OP discovered that you can call a regular function inside a class
    body, outside of a method, during the class statement:




    class MyClass:
         def function():
             return "stuff"


         attr = function()




    This works, but then you can't call MyClass().function() as it will
    raise. So the OP tried making it a staticmethod:


    class MyClass:
         @staticmethod
         def function():
             return "stuff"


         attr = function()




    but that fails, because *staticmethod instances are not callable*. They
    are descriptors. Only after the descriptor protocol gets a chance to run
    do you get a callable function.


    *This is not a bug.* Making staticmethod instances callable is a feature
    enhancement, not a bug fix. Or you could just define your own callable-
    staticmethod descriptor, it's easy enough to do.






    --
    Steven
  • Peter Otten at Nov 24, 2013 at 11:03 am

    Antoon Pardon wrote:


    Op 23-11-13 10:01, Peter Otten schreef:
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()
    I think you are burdening the programmer with implemantation details
    that don't matter to him.

    Replacing "you" in your statement with "python" I was about to suggest to
    make staticmethod(func) callable when I found this had already been
    rejected:


    [Python-Dev] Making staticmethod objects callable?
    Nicolas Fleury nidoizo at yahoo.com
      Wed Mar 1 15:57:12 CET 2006


    https://mail.python.org/pipermail/python-dev/2006-March/061948.html


    If you think you have compelling arguments, go ahead and explain them --
    preferably on python-ideas.
  • Antoon Pardon at Nov 24, 2013 at 4:08 pm

    Op 24-11-13 12:03, Peter Otten schreef:
    Antoon Pardon wrote:
    Op 23-11-13 10:01, Peter Otten schreef:
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()

    doesn't call the Foo.foo attribute directly, it calls

    Foo.foo.__get__(None, Foo)()
    I think you are burdening the programmer with implemantation details
    that don't matter to him.
    Replacing "you" in your statement with "python" I was about to suggest to
    make staticmethod(func) callable when I found this had already been
    rejected:

    [Python-Dev] Making staticmethod objects callable?
    Nicolas Fleury nidoizo at yahoo.com
    Wed Mar 1 15:57:12 CET 2006

    https://mail.python.org/pipermail/python-dev/2006-March/061948.html

    If you think you have compelling arguments, go ahead and explain them --
    preferably on python-ideas.

    I am under no illusion that compelling arguments will make any difference.
    People had argued for a ternary operator for years and it all fell on
    deaf ears until finally a python developer was bitten by a nasty bug
    while using one of the alternative that was always suggested here.


    When you asked that python should behave following the principle of
    least surprise and thus showed consistency where possible, the
    standard trope was that a foolish consistency was the hobgoblin
    of little minds.


    I see no reason why this should go any way differently now.


    --
    Antoon Pardon
  • Marco Buttu at Nov 23, 2013 at 9:39 am

    On 11/23/2013 10:01 AM, Peter Otten wrote:


    In Python 3 the following two classes should be equivalent:
    Says who?
    $ cat foo.py
    class Foo:
    def foo():
    pass
    print(callable(foo))

    class Foo:
    @staticmethod
    def foo():
    pass
    print(callable(foo))

    But they do not:

    $ python3 foo.py
    True
    False
    Your script is saying that a staticmethod instance is not a callable object.
    It need not be because

    Foo.foo()

    Yes, you are right about Python 3. But in Python 2, if I am not going
    wrong, there is not solution, and I need to define a function outside
    the class. For instance:


    $ cat config.py
    class Configuration(object):


          def positiveCheck(value):
              if not value > 0:
                  raise AttributeError('Must be a positive number')


          attributes = {
                  # Attribute name: (type, checkrule)
                  'myattr': (int, positiveCheck),
          }


          def __setattr__(self, name, value):
              if not name in Configuration.attributes:
                  raise AttributeError("Attribute `%s` non allowed." %name)


              expected_type, checkrule = Configuration.attributes[name]
              if not isinstance(value, expected_type):
                  raise TypeError('The value %s is not of type %s' \
                          %(value, expected_type.__name__))
              if callable(checkrule):
                  print('calling %s(%s)' %(checkrule.__name__, value))
                  checkrule(value)


              super(Configuration, self).__setattr__(name, value)


    The positive check works fine:

    from config import Configuration
    c = Configuration()
    c.myattr = -10
    calling positiveCheck(-10)
    Traceback (most recent call last):
          ...
    AttributeError: Must be a positive number


    But I cannot use the method as a function:

    Configuration.positiveCheck(-10)
    Traceback (most recent call last):
          ...
    Configuration instance as first argument (got int instance instead).


    Furthemore, I cannot use the method as a staticmethod, becase otherwise
    it will not be callable inside the class body.


    --
    Marco Buttu
  • Peter Otten at Nov 23, 2013 at 3:23 pm

    Marco Buttu wrote:

    On 11/23/2013 10:01 AM, Peter Otten wrote:

    In Python 3 the following two classes should be equivalent:
    Says who?
    $ cat foo.py
    class Foo:
    def foo():
    pass
    print(callable(foo))

    class Foo:
    @staticmethod
    def foo():
    pass
    print(callable(foo))

    But they do not:

    $ python3 foo.py
    True
    False
    Your script is saying that a staticmethod instance is not a callable
    object. It need not be because

    Foo.foo()
    Yes, you are right about Python 3. But in Python 2, if I am not going
    wrong, there is not solution, and I need to define a function outside
    the class. For instance:

    $ cat config.py
    class Configuration(object):

    def positiveCheck(value):
    if not value > 0:
    raise AttributeError('Must be a positive number')

    attributes = {
    # Attribute name: (type, checkrule)
    'myattr': (int, positiveCheck),
    }

    Just add


            positiveCheck = staticmethod(positiveCheck)


    at this point and everything should work in both Python 2 and 3.
    Alternatively use Steven's callable_staticmethod as a decorator.

    def __setattr__(self, name, value):
    if not name in Configuration.attributes:
    raise AttributeError("Attribute `%s` non allowed." %name)

    expected_type, checkrule = Configuration.attributes[name]
    if not isinstance(value, expected_type):
    raise TypeError('The value %s is not of type %s' \
    %(value, expected_type.__name__))
    if callable(checkrule):
    print('calling %s(%s)' %(checkrule.__name__, value))
    checkrule(value)

    super(Configuration, self).__setattr__(name, value)

    The positive check works fine:
    from config import Configuration
    c = Configuration()
    c.myattr = -10
    calling positiveCheck(-10)
    Traceback (most recent call last):
    ...
    AttributeError: Must be a positive number

    But I cannot use the method as a function:
    Configuration.positiveCheck(-10)
    Traceback (most recent call last):
    ...
    Configuration instance as first argument (got int instance instead).

    Furthemore, I cannot use the method as a staticmethod, becase otherwise
    it will not be callable inside the class body.



    PS: AttributeErrors should be raised when an attribute does not exist or
    cannot be set; I recommend that you raise a ValueError in positiveCheck().
  • Steven D'Aprano at Nov 23, 2013 at 1:48 pm

    On Sat, 23 Nov 2013 09:28:43 +0100, Marco Buttu wrote:


    In Python 3 the following two classes should be equivalent:

    They certainly are not equivalent in *any* version of Python, because
    staticmethods are not equivalent to instance methods.



    $ cat foo.py
    class Foo:
    def foo():
    pass
    print(callable(foo))

    class Foo:
    @staticmethod
    def foo():
    pass
    print(callable(foo))

    But they do not:

    $ python3 foo.py
    True
    False

    And Python 2 gives the same result for staticmethods:


    [steve at ando ~]$ python2.7
    Python 2.7.2 (default, May 18 2012, 18:25:10)
    [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    py>
    py> class Test(object):
    ... @staticmethod
    ... def method():
    ... pass
    ... print callable(method)
    ...
    False



    How come the metaclass does not skip the staticmethod decorator?

    What makes you think the metaclass gets the opportunity to skip the
    decorator? By the time the metaclass gets called, the decorator has
    already run.


    You seem to be conflating behaviour in Python 2 that doesn't actually
    occur. Staticmethods are not directly callable in any version of Python.


    The problem you seem to have is that you want to call a method both
    during and after construction:


    class MyClass(object):
         def helper(arg):
             return arg + 1
         x = helper(10)
         y = helper(20)
         def method(self, arg):
             return self.helper(arg)




    Here, the attributes x and y rely on calling helper as a function, where
    it does not receive a self argument, but when calling helper from inside
    the instance method, or when calling it like MyClass.helper(), it will
    receive a self argument.


    Unfortunately there is no good solution to this using just built-ins.
    staticmethod doesn't work, as it's not callable. However, we can create
    our own callable version of staticmethod:




    class callable_staticmethod(object):
         def __init__(self, func):
             self.func = func
         def __get__(self, obj, cls=None):
             return self.func
         def __call__(self, *args, **kwargs):
             return self.func(*args, **kwargs)




    class MyClass(object):
         @callable_staticmethod
         def helper(arg):
             return arg + 1
         x = helper(10)
         y = helper(20)
         def method(self, arg):
             return self.helper(arg)




    This should now work exactly as you hope.






    --
    Steven

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedNov 23, '13 at 8:28a
activeNov 24, '13 at 4:08p
posts15
users6
websitepython.org

People

Translate

site design / logo © 2022 Grokbase