FAQ
Tim:
Do you really find such OverflowErrors more useful than not? I'm asking
because PEP 237 seeks to eradicate them:

http://python.sf.net/peps/pep-0237.html
Despite what I said yesterday, I have used and needed lont ints
a few times in real code. There are two cases:
1) I expect to get large values (>2**31) frequently
2) I expect to get them infrequently

Case #1 is the more common, and is solved by using an 'L' in the
appropriate initialization. There have been many times when the
omission of the L was not caught until the testing phase, which
means automatic conversion would have been prefered.

Case #2 has only occured once, where >98% of the times the
value could fit in a regular integer and I wanted the
faster math performance. In that case I did
try:
operation
except OverflowError:
redo the operation with longs

So I have have no problems with the current 237 PEP, where
int operations convert to long rather than raising an exception.

Andrew
dalke at dalkescientific.com

Search Discussions

  • Erik Max Francis at Aug 20, 2001 at 6:03 am
    I'm working on an application where I need arithmetic to be done modulo
    2^32 (actually, often it will be 2^n with n < 32). Python (very nicely)
    throws OverflowErrors when an arithmetic calculation breaks these
    bounds, but this is one rare case where I'd prefer it not to happen.
    Essentially I'd like things to behave just as if you had a signed 32-bit
    two's-complement number.

    What is the standard Pythonic way to accomplish this? The obvious thing
    (catching the OverflowError and then doing the computation in longs and
    converting back) didn't seem to work, since the obvious bitmask didn't
    do the job, since obviously the bitwise and operator doesn't have quite
    the same meaning for longs:

    max at oxygen:~% python
    Python 2.0 (#2, Apr 4 2001, 19:28:30)
    [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
    Type "copyright", "credits" or "license" for more information.
    i = 2147483648l # one greater than the largest int
    i
    2147483648L
    i & 0xffffffff
    2147483648L # whoops

    --
    Erik Max Francis / max at alcyone.com / http://www.alcyone.com/max/
    __ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
    / \ It's just another day / And nothing's any good
    \__/ Sade
    Alcyone Systems / http://www.alcyone.com/
    Alcyone Systems, San Jose, California.
  • Tim Peters at Aug 20, 2001 at 7:33 am
    [Erik Max Francis]
    I'm working on an application where I need arithmetic to be done modulo
    2^32 (actually, often it will be 2^n with n < 32). Python (very nicely)
    throws OverflowErrors when an arithmetic calculation breaks these
    bounds,
    Do you really find such OverflowErrors more useful than not? I'm asking
    because PEP 237 seeks to eradicate them:

    http://python.sf.net/peps/pep-0237.html

    and "Part A" is a candidate for Python 2.2 (in short, "Part A" *just*
    auto-converts overflowing int + - * / to longs). In full,

    A. Short int operations that currently raise OverflowError return
    a long int value instead. This is the only change in this
    phase. Literals will still distinguish between short and long
    ints. The other semantic differences listed above (including
    the behavior of <<) will remain. Because this phase only
    changes situations that currently raise OverflowError, it is
    assumed that this won't break existing code. (Code that
    depends on this exception would have to be too convoluted to be
    concerned about it.) For those concerned about extreme
    backwards compatibility, a command line option will allow a
    warning to be issued at this point, but this is off by default.
    but this is one rare case where I'd prefer it not to happen.
    Essentially I'd like things to behave just as if you had a signed
    32-bit two's-complement number.
    Python longs are unbounded signed 2's-comp integers, so you're very close.
    What is the standard Pythonic way to accomplish this?
    Use Python longs. For op in + - * ^ | & ^, the last N bits of

    long1 op long2

    are exactly the same as if you had "real" N-bit signed 2's-comp arithmetic.
    "Signed" vs "unsigned" is pretty much a mental conceit here too -- faking
    2's-comp N-bit unsigned arithmetic is just as easy. For signed, it's the
    conversion at the end that's delicate.
    The obvious thing (catching the OverflowError and then doing the
    computation in longs and converting back) didn't seem to work, since
    the obvious bitmask didn't do the job, since obviously the bitwise
    and operator doesn't have quite the same meaning for longs:
    The & operator isn't really the problem.
    max at oxygen:~% python
    Python 2.0 (#2, Apr 4 2001, 19:28:30)
    [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
    Type "copyright", "credits" or "license" for more information.
    i = 2147483648L # one greater than the largest int
    I took the liberty of changing the impossible-to-read trailing "l" to an "L"
    there.
    i
    2147483648L
    i & 0xffffffff
    2147483648L # whoops
    You're getting screwed by the hex literal: you're doing

    long & int

    so the int part gets converted to long first. But 0xffffffff is -1 as a
    short int (on a 32-bit box), so coerces to -1L, i.e. to an infinite string
    of 1 bits.

    i & 0xffffffff == i

    is thus always true on a 32-bit box, regardless of the value of i, and
    regardless of whether i is short or long. Stick an L at the end of the hex
    literal to get 32 1-bits, which is infinitely fewer than you're getting now
    <wink>.

    It's unclear to me whether you want a negative value in the end represented
    as a short int or still as a long. Here's both ways:

    def tolong(i, bits):
    i &= (1L << bits) - 1 # get last "bits" bits, as unsigned
    if i & (1L << (bits - 1)): # is negative in N-bit 2's comp
    i -= 1L << bits # ... so make it negative
    return i

    def toshort(i, bits):
    return int(tolong(i, bits))

    print tolong(1 << 31, 32)
    print tolong(1L << 31, 32)

    print toshort(1 << 31, 32)
    print toshort(1L << 31, 32)

    That prints -2147483648 four times.
  • Erik Max Francis at Aug 20, 2001 at 3:15 pm

    Tim Peters wrote:

    You're getting screwed by the hex literal: you're doing

    long & int

    so the int part gets converted to long first. But 0xffffffff is -1 as
    a
    short int (on a 32-bit box), so coerces to -1L, i.e. to an infinite
    string
    of 1 bits.

    i & 0xffffffff == i

    is thus always true on a 32-bit box, regardless of the value of i, and
    regardless of whether i is short or long. Stick an L at the end of
    the hex
    literal to get 32 1-bits, which is infinitely fewer than you're
    getting now
    <wink>.
    Aha! Longs are unbounded, so I don't want to and it with -1l, I want to
    and it with 1l << 32. I seem to recall trying to get the effect I
    wanted with 0xffffffffl and it didn't seem to work, but I may have just
    goofed it.

    Thanks! (For the record, I did want 32-bit signed ints, so preserving
    the sign is fine.)

    --
    Erik Max Francis / max at alcyone.com / http://www.alcyone.com/max/
    __ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
    / \ But in matters fundamental / We are patterned on an old design
    \__/ Oleta Adams
    Rules for Buh / http://www.alcyone.com/max/projects/cards/buh.html
    The official rules to the betting card game, Buh.
  • Tim Peters at Aug 23, 2001 at 3:54 am
    [Andrew Dalke]
    Despite what I said yesterday, I have used and needed lont ints
    a few times in real code. There are two cases:
    1) I expect to get large values (>2**31) frequently
    2) I expect to get them infrequently

    Case #1 is the more common, and is solved by using an 'L' in the
    appropriate initialization. There have been many times when the
    omission of the L was not caught until the testing phase, which
    means automatic conversion would have been prefered.

    Case #2 has only occured once, where >98% of the times the
    value could fit in a regular integer and I wanted the
    faster math performance. In that case I did
    try:
    operation
    except OverflowError:
    redo the operation with longs
    I've used long ints extensively, and indeed I first used Python "for real"
    precisely because its long int performance was beating the pants off Icon's
    at the time. Exactly the same two cases apply to me, except that I've done
    case #2 hundreds of times. It's great to hear that I'm not completely out
    of tune with your reality <wink>.
    So I have have no problems with the current 237 PEP, where
    int operations convert to long rather than raising an exception.
    PEP 237 doesn't mention this, but there's a flip side to your case #2:

    def chop(i):
    try:
    return int(i)
    except OverflowError:
    return i

    That is, chop back a long to an int "if it fits", again to get the speed
    boost in later calculations. I wonder whether we could get away with that
    too ...

    one-day-one-revolution-ly y'rs - tim

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedAug 20, '01 at 1:40a
activeAug 23, '01 at 3:54a
posts5
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase