FAQ
I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

We can fix this in a backward-compatible way, I believe.

Demonstration:
I'd expect these two error messages to be identical, but they are
not.
class C(object): pass
c = C()
sum((c,c))
TypeError: unsupported operand type(s) for +: 'int' and 'C'
c + c
TypeError: unsupported operand type(s) for +: 'C' and 'C'

Search Discussions

  • Buck Golemon at Feb 23, 2012 at 9:23 pm

    On Feb 23, 1:19?pm, Buck Golemon wrote:
    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.

    Given a class that implements __add__ why should sum() not be able to
    operate on that class?

    We can fix this in a backward-compatible way, I believe.

    Demonstration:
    ? ? I'd expect these two error messages to be identical, but they are
    not.

    ? ? ?>>> class C(object): pass
    ? ? ?>>> c = C()
    ? ? ?>>> sum((c,c))
    ? ? TypeError: unsupported operand type(s) for +: 'int' and 'C'
    ? ? >>> c + c
    ? ? TypeError: unsupported operand type(s) for +: 'C' and 'C'
    Proposal:

    def sum(values,
    base=0):
    values =
    iter(values)

    try:
    result = values.next()
    except StopIteration:
    return base

    for value in values:
    result += value
    return result
  • Arnaud Delobelle at Feb 23, 2012 at 9:41 pm

    On 23 February 2012 21:23, Buck Golemon wrote:
    def sum(values,
    base=0):
    ? ? ?values =
    iter(values)

    ? ? ?try:
    ? ? ? ? ?result = values.next()
    ? ? ?except StopIteration:
    ? ? ? ? ?return base

    ? ? ?for value in values:
    ? ? ? ? ?result += value
    ? ? ?return result
    This is definitely not backward compatible. To get something that has
    a better chance of working with existing code, try this (untested):

    _sentinel = object()

    def sum(iterable, start=_sentinel):
    if start is _sentinel:
    iterable = iter(iterable)
    try:
    start = iterable.next()
    except StopIteration:
    return 0
    for x in iterable:
    start += x
    return start

    del _sentinel

    --
    Arnaud
  • Chris Angelico at Feb 23, 2012 at 9:53 pm

    On Fri, Feb 24, 2012 at 8:41 AM, Arnaud Delobelle wrote:
    _sentinel = object()

    def sum(iterable, start=_sentinel):
    ? ?if start is _sentinel:

    del _sentinel
    Somewhat off-topic: Doesn't the if statement there do a lookup for a
    global, which would mean that 'del _sentinel' will cause it to fail?
    Or have I missed something here?

    ChrisA
  • Arnaud Delobelle at Feb 23, 2012 at 9:59 pm

    On 23 February 2012 21:53, Chris Angelico wrote:
    On Fri, Feb 24, 2012 at 8:41 AM, Arnaud Delobelle wrote:
    _sentinel = object()

    def sum(iterable, start=_sentinel):
    ? ?if start is _sentinel:

    del _sentinel
    Somewhat off-topic: Doesn't the if statement there do a lookup for a
    global, which would mean that 'del _sentinel' will cause it to fail?
    Or have I missed something here?
    Yes, you're right :) Change the signature to

    def sum(iterable, start=_sentinel, _sentinel=_sentinel):

    This is not pretty...

    --
    Arnaud
  • Chris Angelico at Feb 23, 2012 at 10:04 pm

    On Fri, Feb 24, 2012 at 8:59 AM, Arnaud Delobelle wrote:
    def sum(iterable, start=_sentinel, _sentinel=_sentinel):
    Is this a reason for Python to introduce a new syntax, such as:

    def foo(blah, optional=del):
    if optional is del: print("No argument was provided")

    Basically, 'del' is treated like a unique non-providable object, only
    possible in an argument list and only if the argument was omitted. No
    more proliferation of individual sentinels... what do you think?

    (I picked "del" because it's an existing keyword. Fairly arbitrary
    choice though.)

    Chris Angelico
  • Arnaud Delobelle at Feb 23, 2012 at 10:09 pm

    On 23 February 2012 22:04, Chris Angelico wrote:
    On Fri, Feb 24, 2012 at 8:59 AM, Arnaud Delobelle wrote:
    def sum(iterable, start=_sentinel, _sentinel=_sentinel):
    Is this a reason for Python to introduce a new syntax, such as:

    def foo(blah, optional=del):
    ? ?if optional is del: print("No argument was provided")

    Basically, 'del' is treated like a unique non-providable object, only
    possible in an argument list and only if the argument was omitted. No
    more proliferation of individual sentinels... what do you think?
    The problem with these proposals is to avoid the leakage of 'del'.
    Here you could do:

    def get_del(x=del):
    return x

    And then you're in trouble again.

    --
    Arnaud
  • Ian Kelly at Feb 23, 2012 at 10:00 pm

    On Thu, Feb 23, 2012 at 2:53 PM, Chris Angelico wrote:
    On Fri, Feb 24, 2012 at 8:41 AM, Arnaud Delobelle wrote:
    _sentinel = object()

    def sum(iterable, start=_sentinel):
    ? ?if start is _sentinel:

    del _sentinel
    Somewhat off-topic: Doesn't the if statement there do a lookup for a
    global, which would mean that 'del _sentinel' will cause it to fail?
    Or have I missed something here?
    I believe you're correct. If you really want to delete the _sentinel
    reference though, you could do:

    def sum(iterable, start=object()):
    if start is sum.func_defaults[0]:
    ...

    Cheers,
    Ian
  • Steven D'Aprano at Feb 23, 2012 at 11:33 pm

    On Fri, 24 Feb 2012 08:53:49 +1100, Chris Angelico wrote:
    On Fri, Feb 24, 2012 at 8:41 AM, Arnaud Delobelle wrote:
    _sentinel = object()

    def sum(iterable, start=_sentinel):
    ? ?if start is _sentinel:

    del _sentinel
    Somewhat off-topic: Doesn't the if statement there do a lookup for a
    global, which would mean that 'del _sentinel' will cause it to fail? Or
    have I missed something here?
    Yes, deleting _sentinel will cause the custom sum to fail, and yes, you
    have missed something.

    If the caller wants to mess with your library and break it, they have
    many, many ways to do so apart from deleting your private variables.


    del _sentinel
    _sentinel = "something else"
    sum.__defaults__ = (42,) # mess with the function defaults
    sum.__code__ = (lambda a, b=None: 100).__code__ # and with func internals
    sum = None # change your custom sum to something else
    del sum # or just delete it completely
    len = 42 # shadow a built-in
    import builtins; del builtins.range # really screw with you


    If your application stops working after you carelessly mess with
    components your application relies on, the right answer is usually:

    "Don't do that then."

    Python doesn't try to prevent people from shooting themselves in the foot.


    Monkey-patching-by-actual-monkeys-for-fun-and-profit-ly y'rs,


    --
    Steven
  • Chris Angelico at Feb 23, 2012 at 11:39 pm

    On Fri, Feb 24, 2012 at 10:33 AM, Steven D'Aprano wrote:
    Yes, deleting _sentinel will cause the custom sum to fail, and yes, you
    have missed something.

    If the caller wants to mess with your library and break it, they have
    many, many ways to do so apart from deleting your private variables.
    I was looking at the module breaking itself, though, not even waiting
    for the caller to do it.

    ChrisA
  • Antoon Pardon at Feb 24, 2012 at 11:41 am

    On 02/24/2012 12:33 AM, Steven D'Aprano wrote:
    If your application stops working after you carelessly mess with
    components your application relies on, the right answer is usually:

    "Don't do that then."

    Python doesn't try to prevent people from shooting themselves in the foot.
    Yes it does! A simple example is None as a keyword to prevent
    assignments to it.

    --
    Antoon Pardon
  • Roy Smith at Feb 24, 2012 at 1:23 pm
    In article <mailman.123.1330083762.3037.python-list at python.org>,
    Antoon Pardon wrote:
    Python doesn't try to prevent people from shooting themselves in the foot.
    Yes it does! A simple example is None as a keyword to prevent
    assignments to it.
    Hmmm. Just playing around with some bizarre things to do with None, and
    discovered this:
    import sys as None
    doesn't give an error, but also doesn't assign the module to the symbol
    'None'. Weird.
  • Terry Reedy at Feb 24, 2012 at 9:58 pm

    On 2/24/2012 8:23 AM, Roy Smith wrote:
    In article<mailman.123.1330083762.3037.python-list at python.org>,
    Antoon Pardonwrote:
    Python doesn't try to prevent people from shooting themselves in the foot.
    Yes it does! A simple example is None as a keyword to prevent
    assignments to it.
    Hmmm. Just playing around with some bizarre things to do with None, and
    discovered this:
    import sys as None
    doesn't give an error, but also doesn't assign the module to the symbol
    'None'. Weird.
    In 3.2
    import sys as None
    SyntaxError: invalid syntax

    --
    Terry Jan Reedy
  • Arnaud Delobelle at Feb 23, 2012 at 9:32 pm

    On 23 February 2012 21:19, Buck Golemon wrote:
    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.

    Given a class that implements __add__ why should sum() not be able to
    operate on that class?
    It can. You need to pass a second argument which will be the start
    value. Try help(sum) for details.

    --
    Arnaud
  • Chris Rebert at Feb 23, 2012 at 9:32 pm

    On Thu, Feb 23, 2012 at 1:19 PM, Buck Golemon wrote:
    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.

    Given a class that implements __add__ why should sum() not be able to
    operate on that class?
    The time machine strikes again! sum() already can. You just need to
    specify an appropriate initial value (the empty list in this example)
    for the accumulator :

    Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
    [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    sum([[1,2],[3,4]], [])
    [1, 2, 3, 4]

    Cheers,
    Chris
  • Stefan Behnel at Feb 23, 2012 at 9:42 pm

    Chris Rebert, 23.02.2012 22:32:
    On Thu, Feb 23, 2012 at 1:19 PM, Buck Golemon wrote:
    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.

    Given a class that implements __add__ why should sum() not be able to
    operate on that class?
    The time machine strikes again! sum() already can. You just need to
    specify an appropriate initial value (the empty list in this example)
    for the accumulator :

    Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
    [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    sum([[1,2],[3,4]], [])
    [1, 2, 3, 4]
    I know that you just meant this as an example, but it's worth mentioning in
    this context that it's not exactly efficient to "sum up" lists this way
    because there is a lot of copying involved. Each adding of two lists
    creates a third one and copies all elements into it. So it eats a lot of
    time and space.

    Stefan
  • Duncan Booth at Feb 24, 2012 at 11:40 am

    Stefan Behnel wrote:

    I know that you just meant this as an example, but it's worth
    mentioning in this context that it's not exactly efficient to "sum up"
    lists this way because there is a lot of copying involved. Each adding
    of two lists creates a third one and copies all elements into it. So
    it eats a lot of time and space.
    If you search back through this group far enough you can find an
    alternative implementation of sum that I suggested which doesn't have the
    same performance problem with lists or strings and also improves the
    accuracy of the result with floats.

    In effect what it does is instead of:
    (((((a + b) + c) + d) + e) + f)
    it calculates the sum as:
    ((a + b) + (c + d)) + (e + f)

    i.e. in as balanced a manner as it can given that it still has to work from
    left to right.

    Of course that could still change the final result for some user defined
    types and never having converted my code to C I have no idea whether or not
    the performance for the intended case would be competitive with the builtin
    sum though I don't see why it wouldn't be.
  • Buck Golemon at Feb 23, 2012 at 9:38 pm

    On Feb 23, 1:32?pm, Chris Rebert wrote:
    On Thu, Feb 23, 2012 at 1:19 PM, Buck Golemon wrote:
    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.
    Given a class that implements __add__ why should sum() not be able to
    operate on that class?
    The time machine strikes again! sum() already can. You just need to
    specify an appropriate initial value (the empty list in this example)
    for the accumulator :

    Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
    [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.>>> sum([[1,2],[3,4]], [])

    [1, 2, 3, 4]

    Cheers,
    Chris
    --http://rebertia.com
    Thanks. I did not know that!

    My proposal is still *slightly* superior in two ways:

    1) It reduces the number of __add__ operations by one
    2) The second argument isn't strictly necessary, if you don't mind
    that the 'null sum' will produce zero.

    def sum(values, base=0):
    values = iter(values)

    try:
    result = values.next()
    except StopIteration:
    return base

    for value in values:
    result += value

    return result
  • Ian Kelly at Feb 23, 2012 at 9:54 pm

    On Thu, Feb 23, 2012 at 2:38 PM, Buck Golemon wrote:
    My proposal is still *slightly* superior in two ways:

    1) It reduces the number of __add__ operations by one
    2) The second argument isn't strictly necessary, if you don't mind
    that the 'null sum' will produce zero.
    It produces the wrong result, though:
    sum([3,4], base)
    7

    If I'm starting with 12 and summing 3 and 4, I expect to get 19.

    Ideally the second argument should be ignored only if it isn't passed
    in at all, and I don't know off-hand why the built-in sum doesn't do
    this. We really don't need to replace it, though. If you want a
    different sum behavior, just write your own.

    def sum(iterable, *args):
    return reduce(operator.add, iterable, *args)
    sum([3,4])
    7
    sum([3,4], 12)
    19
    sum(['hello', 'world'])
    'helloworld'

    Cheers,
    Ian
  • Peter Otten at Feb 24, 2012 at 8:29 am

    Buck Golemon wrote:

    I feel like the design of sum() is inconsistent with other language
    features of python. Often python doesn't require a specific type, only
    that the type implement certain methods.

    Given a class that implements __add__ why should sum() not be able to
    operate on that class?

    We can fix this in a backward-compatible way, I believe.

    Demonstration:
    I'd expect these two error messages to be identical, but they are
    not.
    class C(object): pass
    c = C()
    sum((c,c))
    TypeError: unsupported operand type(s) for +: 'int' and 'C'
    c + c
    TypeError: unsupported operand type(s) for +: 'C' and 'C'
    You could explicitly provide a null object:
    class Null(object):
    ... def __add__(self, other):
    ... return other
    ...
    null = Null()
    class A(object):
    ... def __init__(self, v):
    ... self.v = v
    ... def __add__(self, other):
    ... return A("%s+%s" % (self, other))
    ... def __str__(self):
    ... return self.v
    ... def __repr__(self):
    ... return "A(%r)" % self. v
    ...
    sum(map(A, "abc"))
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    TypeError: unsupported operand type(s) for +: 'int' and 'A'
    sum(map(A, "abc"), null)
    A('a+b+c')

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedFeb 23, '12 at 9:19p
activeFeb 24, '12 at 9:58p
posts20
users12
websitepython.org

People

Translate

site design / logo © 2022 Grokbase