FAQ
HISTORY:

In using python 2.7.2 for awhile on a web project (apache/wsgi web.py), I discovered a problem in using decimal.Decimal. A short search revealed that many other people have been having the problem as well, in their own apache/wsgi implementations (django, mostly), but I found no real solutions among the posts I read. So I did some experimentation of my own.

The following code will break unexpectedly on standard python2.7 (and earlier) because of the way that isinstance fails after reload() (which is called by both of the above web frameworks).

This is the error: TypeError("Cannot convert %r to Decimal" % value)

THE TEST CODE

import decimal
from decimal import Decimal

#this works
Decimal(Decimal())

reload(decimal)

#this fails before patching, but works fine afterwards
Decimal(Decimal())


THE SOLUTION:

So, looking into decimal.py I discovered lots if statements using isinstance, and have slightly rearranged the code inside __new__() to mostly remove their use, and for my purposes totally fixes the problem within wsgi.

I am not an official python dev, so would appreciate it if someone who *IS* could just look this over, improve it if necessary, and get it (or some variation) submitted into the library.

Below is a patch for use against python-2.7.2

PATCH:


*** decimal.py 2012-03-02 16:42:51.285964007 -0600
--- /usr/lib/python2.7/decimal.py 2012-03-02 14:36:01.238976461 -0600
***************
*** 535,608 ****
# and the Decimal constructor still deal with tuples of
# digits.

self = object.__new__(cls)

! # From a string
! # REs insist on real strings, so we can too.
! if isinstance(value, basestring):
! m = _parser(value.strip())
! if m is None:
! if context is None:
! context = getcontext()
! return context._raise_error(ConversionSyntax,
! "Invalid literal for Decimal: %r" % value)
!
! if m.group('sign') == "-":
! self._sign = 1
! else:
! self._sign = 0
! intpart = m.group('int')
! if intpart is not None:
! # finite number
! fracpart = m.group('frac') or ''
! exp = int(m.group('exp') or '0')
! self._int = str(int(intpart+fracpart))
! self._exp = exp - len(fracpart)
! self._is_special = False
! else:
! diag = m.group('diag')
! if diag is not None:
! # NaN
! self._int = str(int(diag or '0')).lstrip('0')
! if m.group('signal'):
! self._exp = 'N'
! else:
! self._exp = 'n'
! else:
! # infinity
! self._int = '0'
! self._exp = 'F'
! self._is_special = True
! return self
!
! # From an integer
! if isinstance(value, (int,long)):
! if value >= 0:
! self._sign = 0
! else:
! self._sign = 1
! self._exp = 0
! self._int = str(abs(value))
! self._is_special = False
return self

! # From another decimal
! if isinstance(value, Decimal):
self._exp = value._exp
self._sign = value._sign
self._int = value._int
self._is_special = value._is_special
return self

# From an internal working value
! if isinstance(value, _WorkRep):
self._sign = value.sign
self._int = str(value.int)
self._exp = int(value.exp)
self._is_special = False
return self

# tuple/list conversion (possibly from as_tuple())
if isinstance(value, (list,tuple)):
if len(value) != 3:
raise ValueError('Invalid tuple size in creation of Decimal '
--- 535,582 ----
# and the Decimal constructor still deal with tuples of
# digits.

self = object.__new__(cls)

! # Note about isinstance -- Decimal has been a victim of the
! # isinstance builtin failing after module reload in some
! # environments (e.g. web.py, django) under apache/wsgi, which
! # I determined to be the reason Decimal was causing so many
! # problems in my web deployment. I have re-organized the
! # following code to remove use of isinstance except on
! # native types (int, long, float, list, tuple), since those
! # seem not to break in this regard. -- jdb
!
! # First, assume it's another Decimal or similar(having _exp,
! # _sign, _int and _is_special. Obviously, having these
! # implies it's at least an attempt to represent Decimal
! try:
! self._exp = value._exp
! self._sign = value._sign
! self._int = value._int
! self._is_special = value._is_special
return self
+ except: pass

! # Or it's a float
! try:
! value = Decimal.from_float(value)
self._exp = value._exp
self._sign = value._sign
self._int = value._int
self._is_special = value._is_special
return self
+ except: pass

# From an internal working value
! try:
self._sign = value.sign
self._int = str(value.int)
self._exp = int(value.exp)
self._is_special = False
return self
+ except: pass

# tuple/list conversion (possibly from as_tuple())
if isinstance(value, (list,tuple)):
if len(value) != 3:
raise ValueError('Invalid tuple size in creation of Decimal '
***************
*** 645,661 ****
raise ValueError("The third value in the tuple must "
"be an integer, or one of the "
"strings 'F', 'n', 'N'.")
return self

! if isinstance(value, float):
! value = Decimal.from_float(value)
! self._exp = value._exp
! self._sign = value._sign
! self._int = value._int
! self._is_special = value._is_special
return self

raise TypeError("Cannot convert %r to Decimal" % value)

# @classmethod, but @decorator is not valid Python 2.3 syntax, so
# don't use it (see notes on Py2.3 compatibility at top of file)
--- 619,666 ----
raise ValueError("The third value in the tuple must "
"be an integer, or one of the "
"strings 'F', 'n', 'N'.")
return self

! # From a string, or anything representable as a string
! try:
! value = str(value)
! m = _parser(value.strip())
! if m is None:
! if context is None:
! context = getcontext()
! return context._raise_error(ConversionSyntax,
! "Invalid literal for Decimal: %r" % value)
!
! if m.group('sign') == "-":
! self._sign = 1
! else:
! self._sign = 0
! intpart = m.group('int')
! if intpart is not None:
! # finite number
! fracpart = m.group('frac') or ''
! exp = int(m.group('exp') or '0')
! self._int = str(int(intpart+fracpart))
! self._exp = exp - len(fracpart)
! self._is_special = False
! else:
! diag = m.group('diag')
! if diag is not None:
! # NaN
! self._int = str(int(diag or '0')).lstrip('0')
! if m.group('signal'):
! self._exp = 'N'
! else:
! self._exp = 'n'
! else:
! # infinity
! self._int = '0'
! self._exp = 'F'
! self._is_special = True
return self
+ except: pass

raise TypeError("Cannot convert %r to Decimal" % value)

# @classmethod, but @decorator is not valid Python 2.3 syntax, so
# don't use it (see notes on Py2.3 compatibility at top of file)

Search Discussions

  • Ethan Furman at Mar 2, 2012 at 11:49 pm

    Jeff Beardsley wrote:
    HISTORY:

    In using python 2.7.2 for awhile on a web project (apache/wsgi web.py), I discovered a problem in using decimal.Decimal. A short search revealed that many other people have been having the problem as well, in their own apache/wsgi implementations (django, mostly), but I found no real solutions among the posts I read. So I did some experimentation of my own.

    The following code will break unexpectedly on standard python2.7 (and earlier) because of the way that isinstance fails after reload() (which is called by both of the above web frameworks).

    This is the error: TypeError("Cannot convert %r to Decimal" % value)

    THE TEST CODE

    import decimal
    from decimal import Decimal

    #this works
    Decimal(Decimal())

    reload(decimal)

    #this fails before patching, but works fine afterwards
    Decimal(Decimal())
    Patching decimal.py to make it work with reload() is probably not going
    to happen.

    What you should be doing is:

    import decimal
    from decimal import Decimal

    reload(decimal)
    Decimal = decimal.Decimal # (rebind 'Decimal' to the reloaded code)

    ~Ethan~
  • Jeff Beardsley at Mar 3, 2012 at 12:18 am
    The problem with that though is: I am not the one calling reload(). That
    is actually being called for me by web.py (or django, or some other
    framework, take your pick). More than that, I believe it's called (or
    caused, anyway) by something happening in WSGI under apache. (And I don't
    really want to start digging around in there either)

    The patch in this case is very limited in scope, and all it inflicts on the
    subject code inside of decimal.Decimal.__new__(), is better programming
    practices.

    --jeff
    On Fri, Mar 2, 2012 at 5:49 PM, Ethan Furman wrote:

    Jeff Beardsley wrote:
    HISTORY:
    In using python 2.7.2 for awhile on a web project (apache/wsgi web.py), I
    discovered a problem in using decimal.Decimal. A short search revealed
    that many other people have been having the problem as well, in their own
    apache/wsgi implementations (django, mostly), but I found no real solutions
    among the posts I read. So I did some experimentation of my own.

    The following code will break unexpectedly on standard python2.7 (and
    earlier) because of the way that isinstance fails after reload() (which is
    called by both of the above web frameworks).

    This is the error: TypeError("Cannot convert %r to Decimal" % value)

    THE TEST CODE

    import decimal
    from decimal import Decimal

    #this works
    Decimal(Decimal())

    reload(decimal)

    #this fails before patching, but works fine afterwards
    Decimal(Decimal())
    Patching decimal.py to make it work with reload() is probably not going to
    happen.

    What you should be doing is:

    import decimal
    from decimal import Decimal

    reload(decimal)
    Decimal = decimal.Decimal # (rebind 'Decimal' to the reloaded code)

    ~Ethan~
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-list/attachments/20120302/13457e33/attachment.html>
  • Ethan Furman at Mar 4, 2012 at 3:44 pm

    Jeff Beardsley wrote:
    The problem with that though: I am not calling reload(), except to
    recreate the error as implemented by the web frameworks.

    I am also unlikely to get a patch accepted into several different
    projects, where this is ONE project, and it's a simple change
    Simple -- maybe.

    Appropriate -- no.

    It is unfortunate that those frameworks have that bug, but it is not up
    to Decimal to fix it for them.

    ~Ethan~
  • A. Lloyd Flanagan at Mar 4, 2012 at 12:37 pm

    On Friday, March 2, 2012 6:49:39 PM UTC-5, Ethan Furman wrote:
    Jeff Beardsley wrote:
    HISTORY:
    ...
    What you should be doing is:

    import decimal
    from decimal import Decimal

    reload(decimal)
    Decimal = decimal.Decimal # (rebind 'Decimal' to the reloaded code)

    ~Ethan~
    Agree that's how the import should be done. On the other hand, removing gratuitous use of isinstance() is generally a Good Thing.
  • Ethan Furman at Mar 4, 2012 at 3:38 pm

    A. Lloyd Flanagan wrote:
    On Friday, March 2, 2012 6:49:39 PM UTC-5, Ethan Furman wrote:
    Jeff Beardsley wrote:
    HISTORY:
    ...
    What you should be doing is:

    import decimal
    from decimal import Decimal

    reload(decimal)
    Decimal = decimal.Decimal # (rebind 'Decimal' to the reloaded code)

    ~Ethan~
    Agree that's how the import should be done. On the other hand, removing gratuitous use of isinstance() is generally a Good Thing.
    Gratuitous use? How is it gratuitous for an class to check that one of
    its arguments is its own type?

    class Frizzy(object):
    def __add__(self, other):
    if not isinstance(other, Frizzy):
    return NotImplemented
    do_stuff_with(self, other)

    This is exactly what isinstance() is for, and this is how it is being
    used in Decimal.__init__().

    ~Ethan~

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMar 2, '12 at 10:46p
activeMar 4, '12 at 3:44p
posts6
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase