FAQ
When I run pychecker through my modules I get the message that
comparisons with "False" is not necessary and that it might yield
unexpected results.

Yet in some situations I need to specifically check whether False was
returned or None was returned. Why is comparison with False so bad?

# example code which matches both False and None
if not var:
# do something

# specifically check if False is returned
# I don't want None
if var == False:
# do something

So how do you get around this? My functions return False and None under
different circumstances. Should I raise exceptions instead? I feel it's
unnecessary clutter to use exceptions unless absolutely no other solution
is available and yet I have doubts about the "False" value.

--
Harishankar (http://harishankar.org http://lawstudentscommunity.com)

Search Discussions

  • Alice Bevan–McGregor at Dec 2, 2010 at 8:15 am
    Howdy!
    When I run pychecker through my modules I get the message that
    comparisons with "False" is not necessary and that it might yield
    unexpected results.
    Comparisons against False -are- dangerous, demonstrated below.
    Yet in some situations I need to specifically check whether False was
    returned or None was returned. Why is comparison with False so bad?
    (False == 0) is True
    (True == 1) is True

    The bool type is a subclass of int! (Run those lines in a Python
    interpreter to see. ;)
    if var == False:
    if var is False: ?
    So how do you get around this? My functions return False and None under
    different circumstances. Should I raise exceptions instead? I feel it's
    unnecessary clutter to use exceptions unless absolutely no other
    solution is available and yet I have doubts about the "False" value.
    If you want to check not just for value equivelance (False == 0) but
    literal type, use the "is" comparator. "is" checks, and others correct
    me if I'm wrong, the literal memory address of an object against
    another. E.g. False, being a singleton, will always have the same
    memory address. (This is true of CPython, possibly not of Python
    implementations like Jython or IronPython.) Using "is" will be
    effective for checking for literal None as well.

    When ever I need to test for None, I always use the "is" comparator.
    It's also more English-like. (None, evaluating to False when using
    '==', is useful when all you care about is having a blank default
    value, for example.)

    ? Alice.
  • Harishankar at Dec 2, 2010 at 10:02 am

    On Thu, 02 Dec 2010 00:15:42 -0800, Alice Bevan?McGregor wrote:

    Howdy!
    Good day to you!
    (False == 0) is True
    (True == 1) is True
    I see. Thanks for this. I suspected this, but wasn't sure.
    The bool type is a subclass of int! (Run those lines in a Python
    interpreter to see. ;)
    if var == False:
    if var is False: ?
    So "var is False" is safer to use when I want to specifically check
    whether var is set to False and not 0 or None?
    If you want to check not just for value equivelance (False == 0) but
    literal type, use the "is" comparator. "is" checks, and others correct
    me if I'm wrong, the literal memory address of an object against
    another. E.g. False, being a singleton, will always have the same
    memory address. (This is true of CPython, possibly not of Python
    implementations like Jython or IronPython.) Using "is" will be
    effective for checking for literal None as well.
    Thanks, it makes sense to me now. Literal equivalence is what I was
    looking for. I didn't quite understand whether == achieved this or not.
    Now I guess I know.
    When ever I need to test for None, I always use the "is" comparator.
    It's also more English-like. (None, evaluating to False when using
    '==', is useful when all you care about is having a blank default value,
    for example.)
    Yes, but in my function I don't want to confuse False with 0 or anything
    else except False. Thanks again for explaining this clearly.
    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Stephen Hansen at Dec 2, 2010 at 10:49 am

    On 12/2/10 2:02 AM, Harishankar wrote:
    On Thu, 02 Dec 2010 00:15:42 -0800, Alice Bevan?McGregor wrote:
    The bool type is a subclass of int! (Run those lines in a Python
    interpreter to see. ;)
    if var == False:
    if var is False: ?
    So "var is False" is safer to use when I want to specifically check
    whether var is set to False and not 0 or None?
    Equality is a somewhat fuzzy concept.

    By convention and habit, its usually fine and clear: but its still fuzzy
    and up to each individual object involved to answer the question of
    equality.

    Now, its generally suggested in Python to do fairly vague truth testing:
    "if x" and "if not x" as opposed to concepts like "if x == True" and "if
    x == False" because the former is broad but usually more correct, and
    the latter can lead to some unexpected semantics.

    But that doesn't mean you must never check if something is False or
    True: there are times when you really do want or need to see if _False_
    is what's being returned, or _True_, or _None_. In this case, use "is",
    yes, indeed.

    The "is" operator checks absolute object identity, and so is how you
    should do that check in cases where you want to test the distinction
    between "Is it /False/, or just something false-ish or not-true or
    nothing-ish?"

    Generally speaking in Python, you usually want to do tests as "if x" or
    "if not x".

    But sometimes you need to know if "x" is a specific singleton value:
    True, False or None. In that case, its okay to do "if x is True", "if x
    is False" or "if x is None".

    But you should only do that after the simple test is deemed
    inappropriate in your API or situation.

    And-- here's the rub-- while "is" is absolutely OK and right for you to
    use to test if an object is one of those singletons, its *probably* NOT
    what you want to do in any other situation*.

    Outside of the True/False/None singletons, and places where you're doing
    some special OOP-stuff, you almost certainly don't want to use 'is', but
    use equality checking (even if its fuzzy, because its fuzzy) instead.

    This demonstrates why "is" should be avoided except when in those
    singleton situations (except when you need to, of course):
    a = 2
    b = 2
    a is b
    True
    a == b
    True
    a = 20000
    b = 20000
    a is b
    False
    a == b
    True

    (If you're wondering why that's happening: Python makes very little in
    the way of promises with regard to object identity. It may choose to
    make a whole new int object of value 2 every time you type 2, or use the
    same old int object each time: sure, presently it tends to only share
    "small" integers for re-use, but that's not a promise, not a documented
    feature, but a function of the current implementation. It could happen
    tomorrow, in theory, that where a = 10000; b = 10000; become the same
    object as far as "is" is concerned even though today they are
    different... "is" should only be used in situations where you care about
    absolute object identity, not *value*.)
    --

    Stephen Hansen
    ... Also: Ixokai
    ... Mail: me+list/python (AT) ixokai (DOT) io
    ... Blog: http://meh.ixokai.io/

    * P.S. I'm not saying its never right to use "is" outside of The
    Singletons. Just that its probably not, for most people, what they
    actually should do in most code. There are numerous counter-examples, of
    course. Its just a general guideline to follow. Until a need arises that
    demonstrates otherwise.

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: signature.asc
    Type: application/pgp-signature
    Size: 487 bytes
    Desc: OpenPGP digital signature
    URL: <http://mail.python.org/pipermail/python-list/attachments/20101202/506efd5c/attachment.pgp>
  • Harishankar at Dec 2, 2010 at 2:18 pm

    On Thu, 02 Dec 2010 02:49:50 -0800, Stephen Hansen wrote:
    ...
    ...
    ...
    * P.S. I'm not saying its never right to use "is" outside of The
    Singletons. Just that its probably not, for most people, what they
    actually should do in most code. There are numerous counter-examples, of
    course. Its just a general guideline to follow. Until a need arises that
    demonstrates otherwise.
    Here I'm using it to compare the result of a function where I
    specifically return False on error condition, so I think it's better I
    check it against the literal False rather than the fuzzy False produced
    by the boolean operation.

    I wouldn't do this in most situations though, but I did need to
    distinguish between the the empty list and False and using the broader
    form as in "if not a" did not work as expected.

    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Tim Chase at Dec 2, 2010 at 2:44 pm

    On 12/02/2010 08:18 AM, Harishankar wrote:
    Here I'm using it to compare the result of a function where I
    specifically return False on error condition,
    This sounds exactly like the reason to use exceptions...you have
    an exceptional error condition.

    -tkc
  • Harishankar at Dec 2, 2010 at 2:56 pm

    On Thu, 02 Dec 2010 08:44:11 -0600, Tim Chase wrote:
    On 12/02/2010 08:18 AM, Harishankar wrote:
    Here I'm using it to compare the result of a function where I
    specifically return False on error condition,
    This sounds exactly like the reason to use exceptions...you have an
    exceptional error condition.

    -tkc
    There are some reasons why I hate exceptions but that is a different
    topic. However, in short I can say that personally:

    1. I hate try blocks which add complexity to the code when none is
    needed. Try blocks make code much more unreadable in my view and I use it
    only for the built-in exceptions when absolutely needed.

    2. I prefer the less irksome True or False to do error checking.
    Exceptions seem too heavyweight for simple problems.

    3. Philosophically I think exception handling is the wrong approach to
    error management. I have never grown up programming with exceptions in C
    and I couldn't pick up the habit with python either. Did I mention that I
    detest try blocks? try blocks seem ugly and destroy code clarity at least
    in my view. And enclosing single statements under separate try blocks
    seem to add a lot of clutter.

    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Steve Holden at Dec 2, 2010 at 3:21 pm

    On 12/2/2010 9:56 AM, Harishankar wrote:
    3. Philosophically I think exception handling is the wrong approach to
    error management. I have never grown up programming with exceptions in C
    and I couldn't pick up the habit with python either. Did I mention that I
    detest try blocks? try blocks seem ugly and destroy code clarity at least
    in my view. And enclosing single statements under separate try blocks
    seem to add a lot of clutter.
    Whereas lots of nested if statements to test that multiple errors have
    all not occurred is a model of clarity? This sounds to me like a
    prejudice that will harm your Python development.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
    See Python Video! http://python.mirocommunity.org/
    Holden Web LLC http://www.holdenweb.com/
  • Stephen Hansen at Dec 2, 2010 at 3:35 pm

    On 12/2/10 6:56 AM, Harishankar wrote:
    On Thu, 02 Dec 2010 08:44:11 -0600, Tim Chase wrote:
    On 12/02/2010 08:18 AM, Harishankar wrote:
    Here I'm using it to compare the result of a function where I
    specifically return False on error condition,
    This sounds exactly like the reason to use exceptions...you have an
    exceptional error condition.

    -tkc
    There are some reasons why I hate exceptions but that is a different
    topic. However, in short I can say that personally:
    To each his/her own, of course; but --
    3. Philosophically I think exception handling is the wrong approach to
    error management.
    Exceptions aren't about "error management"; they are about exceptional
    conditions: some are errors, others are entirely normal situations you
    know are going to happen (such as reaching the end of a sequence as you
    iterate over it: that's not an error, but it is special). To be
    philosophically opposed to them seems to me to be philosophically in
    favor of race conditions.
    I have never grown up programming with exceptions in C
    and I couldn't pick up the habit with python either. Did I mention that I
    detest try blocks? try blocks seem ugly and destroy code clarity at least
    in my view. And enclosing single statements under separate try blocks
    seem to add a lot of clutter.
    ? How do they destroy clarity or add clutter, since presumably you have
    to deal with that "False" in some way with logical structure -- which
    always does whitespace in Python. If not immediately, then up the call
    stack (which seems to imply you should just not use a try/except and let
    the exception unwind the stack to wherever in your code someone wants to
    deal with it).

    if is_correct():
    result = do_thing()
    else:
    do_error_handling()

    try:
    result = do_thing()
    except KeyError:
    do_error_handling()

    And as an aside, the more statements one puts into a try/except block:
    and the more complicated a statement at that-- the more likely it is
    they are going to mess up and do something Wrong.

    Now, all that said: sure, in some situations I do prefer the "check
    first" style of programming where exceptions don't end up being used.
    Its not *wrong* to return False on an "error condition": its going
    against the grain, though, and makes your code harder to deal with
    long-term if only because now there's two separate mechanisms that
    "errors" happen in it.

    You can't totally do away with exceptions in Python, even if you try
    very hard. So with that in mind, IMHO, the best approach is to just...
    get over it, and learn to appreciate them :)

    But, to each his or her own. :)

    --

    Stephen Hansen
    ... Also: Ixokai
    ... Mail: me+list/python (AT) ixokai (DOT) io
    ... Blog: http://meh.ixokai.io/

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: signature.asc
    Type: application/pgp-signature
    Size: 487 bytes
    Desc: OpenPGP digital signature
    URL: <http://mail.python.org/pipermail/python-list/attachments/20101202/100a24c9/attachment.pgp>
  • Harishankar at Dec 2, 2010 at 3:58 pm

    On Thu, 02 Dec 2010 07:35:18 -0800, Stephen Hansen wrote:

    Exceptions aren't about "error management"; they are about exceptional
    conditions: some are errors, others are entirely normal situations you
    know are going to happen (such as reaching the end of a sequence as you
    iterate over it: that's not an error, but it is special). To be
    philosophically opposed to them seems to me to be philosophically in
    favor of race conditions.
    Maybe I worded that part wrongly. Of course I get that error handling is
    only a part of exception mechanism. I agree that the built-in exceptions
    are useful but I prefer most times not to catch them. :-)
    ? How do they destroy clarity or add clutter, since presumably you have
    to deal with that "False" in some way with logical structure -- which
    always does whitespace in Python. If not immediately, then up the call
    stack (which seems to imply you should just not use a try/except and let
    the exception unwind the stack to wherever in your code someone wants to
    deal with it).
    The reason why I said they remove clarity is because it's not always
    obvious at which point the exception may be raised. In other words,
    within a try block there may be multiple statements that generate the
    exception. Of course, as I said before, one way would be to wrap single
    statements with the try block and avoid this issue.
    if is_correct():
    result = do_thing()
    else:
    do_error_handling()

    try:
    result = do_thing()
    except KeyError:
    do_error_handling()
    Of course, to me the if statement would make more sense because I
    immediately figure out the exact condition being tested against while the
    exception object is not always so clear and maybe ambiguous in some
    cases.

    Also if that same type of exception is raised by another statement within
    a function that is called within the try block then it would be handled
    by the same except block right? This is where it gets a bit confusing to
    me and the flow of code is not always obvious. That's why I prefer atomic
    error handling where I know exactly which part of the code led to the
    result.
    And as an aside, the more statements one puts into a try/except block:
    and the more complicated a statement at that-- the more likely it is
    they are going to mess up and do something Wrong.

    Now, all that said: sure, in some situations I do prefer the "check
    first" style of programming where exceptions don't end up being used.
    Its not *wrong* to return False on an "error condition": its going
    against the grain, though, and makes your code harder to deal with
    long-term if only because now there's two separate mechanisms that
    "errors" happen in it.
    I realize that it is impossible for me to avoid exception mechanism
    myself. So I restrict it only to situations where I cannot avoid it (e.g.
    inbuilt exceptions in some cases where I want to handle it myself)

    You can't totally do away with exceptions in Python, even if you try
    very hard. So with that in mind, IMHO, the best approach is to just...
    get over it, and learn to appreciate them :)
    Finding it hard to appreciate exceptions myself. But I am used to
    thinking linearly. A piece of code which does not explain itself in the
    most obvious way even if I wrote it always worries me.

    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Terry Reedy at Dec 2, 2010 at 6:31 pm

    On 12/2/2010 9:56 AM, Harishankar wrote:

    There are some reasons why I hate exceptions but that is a different
    topic. However, in short I can say that personally:

    1. I hate try blocks which add complexity to the code when none is
    needed. Try blocks make code much more unreadable in my view and I use it
    only for the built-in exceptions when absolutely needed.

    2. I prefer the less irksome True or False to do error checking.
    Exceptions seem too heavyweight for simple problems.
    It turns out that try block are computationally lighter weight (faster)
    for normal execution ;-)
    3. Philosophically I think exception handling is the wrong approach to
    error management. I have never grown up programming with exceptions in C
    and I couldn't pick up the habit with python either. Did I mention that I
    detest try blocks? try blocks seem ugly and destroy code clarity at least
    in my view. And enclosing single statements under separate try blocks
    seem to add a lot of clutter.
    Having also come to Python directly from C, I can sympathize. It took me
    a while to adjust.

    --
    Terry Jan Reedy
  • Steve Holden at Dec 2, 2010 at 7:50 pm

    On 12/2/2010 1:31 PM, Terry Reedy wrote:
    It turns out that try block are computationally lighter weight (faster)
    for normal execution ;-)
    Though that alone would hardly be sufficient reason to use them.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
    See Python Video! http://python.mirocommunity.org/
    Holden Web LLC http://www.holdenweb.com/
  • Tim Harig at Dec 2, 2010 at 3:25 pm

    On 2010-12-02, Harishankar wrote:
    There are some reasons why I hate exceptions but that is a different
    topic. However, in short I can say that personally:

    1. I hate try blocks which add complexity to the code when none is
    needed. Try blocks make code much more unreadable in my view and I use it
    only for the built-in exceptions when absolutely needed.
    Actually, exceptions remove a ton of complexity and almost universally
    remove a ton of redundant error checking code. Second, they aleviate a
    ton of design complexity about designing a clean and unified method of
    error handling throughout the program. Using exceptions, the only real
    questions are where to handle various errors. It is a godsend for having
    to clean up intermediary results when an error occurs as part of a complex
    set of dependant sequential operations.
    2. I prefer the less irksome True or False to do error checking.
    Exceptions seem too heavyweight for simple problems.
    Error handling in C huge source of bugs. First, error handling code is
    scattered throughout the codebase and second it is hard to test the error
    handling code for a number of failures. Being able to raise exceptions
    within your test code makes it much easier to write tests capable of
    detecting error handling bugs.
    3. Philosophically I think exception handling is the wrong approach to
    error management. I have never grown up programming with exceptions in C
    and I couldn't pick up the habit with python either. Did I mention that I
    detest try blocks? try blocks seem ugly and destroy code clarity at least
    in my view. And enclosing single statements under separate try blocks
    seem to add a lot of clutter.
    Perhaps you should take a look at how Erlang appoaches exception handling.
    Being message passing and concurrency oriented, Erlang encourages ignoring
    error conditions within worker processes. Errors instead cause the worker
    processes to be killed and a supervisory process is notified, by message,
    so that it can handle the error and respawn the worker process. Since it
    doesn't use try/exept blocks, maybe that will be more to your liking.
  • Harishankar at Dec 2, 2010 at 3:44 pm

    On Thu, 02 Dec 2010 15:25:55 +0000, Tim Harig wrote:
    ...
    ...

    Perhaps you should take a look at how Erlang appoaches exception
    handling. Being message passing and concurrency oriented, Erlang
    encourages ignoring error conditions within worker processes. Errors
    instead cause the worker processes to be killed and a supervisory
    process is notified, by message, so that it can handle the error and
    respawn the worker process. Since it doesn't use try/exept blocks,
    maybe that will be more to your liking.
    Thanks for the reply.

    I understand that the error vs exception debate is quite a big one in the
    programming community as a whole and I don't consider myself very
    knowledgeable in these issues. However, I will try to approach this with
    an open mind and see whether I can work with exceptions comfortably in
    Python. I do understand both sides of the issue. Exceptions seem to be
    generally more reliable but I feel they add a lot of complexity
    particular when a lot of code is placed in a try block.

    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Tim Harig at Dec 2, 2010 at 4:12 pm

    On 2010-12-02, Harishankar wrote:
    I understand that the error vs exception debate is quite a big one in the
    programming community as a whole and I don't consider myself very
    Actually, I thought that debate was resolved years ago. I cannot think of
    a single recently developed programming language that does not provide
    exception handling mechanisms because they have been proven more reliable.
    Python. I do understand both sides of the issue. Exceptions seem to be
    generally more reliable but I feel they add a lot of complexity
    particular when a lot of code is placed in a try block.
    Lines of code is one measure of complexity. Each line of code has a small
    chance of containing a bug. The more lines of code that you have, then the
    more likely that one of them contains a bug. Exceptions, by placing error
    handling code in fewer places, requires much fewer lines of code then
    requiring error handling code after each call that might produce an error
    condition.

    The operations of propogating an error up to the higher level logic of
    the program is another measure. In languages without exception handling,
    careful planning is needed to pass error conditions up through the call
    stack until they reach a high enough level in the logic that decisions
    can be made about how to handle them. Even further planning must be
    taken so that when the error condition reaches level where it needs to
    be handled, that enough information about the error is present to know
    exactly what went wrong so that it can figure out what to do about it.
    This usually involves using globals like errorno to pass out of band
    information about the error. Sometimes you even need to know about how
    the error affect intermediate levels, did the intermediate code attempt
    to handle the condtion and fail? The Openssl error handling system,
    that creates an error logging chain is an example of just how complex
    this can become. You gain all of this functionality automatically
    through exception mechanisms; without all of the complexity.
  • MRAB at Dec 2, 2010 at 5:57 pm

    On 02/12/2010 16:12, Tim Harig wrote:
    On 2010-12-02, Harishankarwrote:
    I understand that the error vs exception debate is quite a big one in the
    programming community as a whole and I don't consider myself very
    Actually, I thought that debate was resolved years ago. I cannot think of
    a single recently developed programming language that does not provide
    exception handling mechanisms because they have been proven more reliable.
    Python. I do understand both sides of the issue. Exceptions seem to be
    generally more reliable but I feel they add a lot of complexity
    particular when a lot of code is placed in a try block.
    Lines of code is one measure of complexity. Each line of code has a small
    chance of containing a bug. The more lines of code that you have, then the
    more likely that one of them contains a bug. Exceptions, by placing error
    handling code in fewer places, requires much fewer lines of code then
    requiring error handling code after each call that might produce an error
    condition.

    The operations of propogating an error up to the higher level logic of
    the program is another measure. In languages without exception handling,
    careful planning is needed to pass error conditions up through the call
    stack until they reach a high enough level in the logic that decisions
    can be made about how to handle them. Even further planning must be
    taken so that when the error condition reaches level where it needs to
    be handled, that enough information about the error is present to know
    exactly what went wrong so that it can figure out what to do about it.
    This usually involves using globals like errorno to pass out of band
    information about the error. Sometimes you even need to know about how
    the error affect intermediate levels, did the intermediate code attempt
    to handle the condtion and fail? The Openssl error handling system,
    that creates an error logging chain is an example of just how complex
    this can become. You gain all of this functionality automatically
    through exception mechanisms; without all of the complexity.
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.

    Automatic garbage collection would also have been nice.

    You don't realise how nice it is to have such things until you have to
    go without them.
  • Paul Rubin at Dec 2, 2010 at 6:09 pm

    MRAB <python at mrabarnett.plus.com> writes:
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.
    That's called longjmp.
    Automatic garbage collection would also have been nice.
    alloca might help.
  • Tim Harig at Dec 2, 2010 at 6:16 pm

    On 2010-12-02, Paul Rubin wrote:
    MRAB <python at mrabarnett.plus.com> writes:
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.
    That's called longjmp.
    The problem is that you might have partially allocated data structures that
    you need to free before you can go anywhere. Jumping up several levels,
    without letting the intermediate levels do their cleanup could easily lead
    to a memory leak or worse.
  • Paul Rubin at Dec 2, 2010 at 6:33 pm

    Tim Harig <usernet at ilthio.net> writes:
    That's called longjmp.
    The problem is that you might have partially allocated data structures
    that you need to free before you can go anywhere.
    Alloca can help with that since the stack stuff gets released by the
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    that, you're halfway towards reinventing exceptions.
  • Tim Harig at Dec 2, 2010 at 6:51 pm

    On 2010-12-02, Paul Rubin wrote:
    Tim Harig <usernet at ilthio.net> writes:
    That's called longjmp.
    The problem is that you might have partially allocated data structures
    that you need to free before you can go anywhere.
    Alloca can help with that since the stack stuff gets released by the
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    alloca() only helps if you actually *want* the data stored on the stack.
    There are many reasons one might prefer or need that the data in the heap.
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    Only if you already have pointers to *all* of the data structures at
    the point where you put your setjmp(). This approach is error prone.
    records that the longjmp handler walks through. Of course if you do
    that, you're halfway towards reinventing exceptions.
    Exactly.
  • Paul Rubin at Dec 2, 2010 at 7:04 pm

    Tim Harig <usernet at ilthio.net> writes:
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    Only if you already have pointers to *all* of the data structures at
    the point where you put your setjmp().
    The setjmp point only has to know where the aux stack is and its depth
    when the longjmp happens. The cleanup records contain any necessary
    pointers to data structures that need freeing. That is basically how
    try/finally would do it too. This is pretty standard stuff.
  • Tim Harig at Dec 2, 2010 at 7:15 pm

    On 2010-12-02, Paul Rubin wrote:
    Tim Harig <usernet at ilthio.net> writes:
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    Only if you already have pointers to *all* of the data structures at
    the point where you put your setjmp().
    The setjmp point only has to know where the aux stack is and its depth
    when the longjmp happens. The cleanup records contain any necessary
    pointers to data structures that need freeing. That is basically how
    try/finally would do it too. This is pretty standard stuff.
    I am not talking about what setjmp() has to do, I am talking about what
    *you* have to do after setjmp() returns. If you have allocated memory in
    intermediate functions and you don't have a reference to them outside of
    the functions that longjmp() bypasses from returning properly (and thus
    either not clearning data structures or returning a reference to those data
    structures as it normally would) then you have potential memory leaks,
    dangling pointers, etc.

    I am not saying that this cannot be done. What I am saying is that it
    is inherently error prone.
  • MRAB at Dec 2, 2010 at 7:54 pm

    On 02/12/2010 19:15, Tim Harig wrote:
    On 2010-12-02, Paul Rubinwrote:
    Tim Harig<usernet at ilthio.net> writes:
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    Only if you already have pointers to *all* of the data structures at
    the point where you put your setjmp().
    The setjmp point only has to know where the aux stack is and its depth
    when the longjmp happens. The cleanup records contain any necessary
    pointers to data structures that need freeing. That is basically how
    try/finally would do it too. This is pretty standard stuff.
    I am not talking about what setjmp() has to do, I am talking about what
    *you* have to do after setjmp() returns. If you have allocated memory in
    intermediate functions and you don't have a reference to them outside of
    the functions that longjmp() bypasses from returning properly (and thus
    either not clearning data structures or returning a reference to those data
    structures as it normally would) then you have potential memory leaks,
    dangling pointers, etc.

    I am not saying that this cannot be done. What I am saying is that it
    is inherently error prone.
    Automatic garbage collection is nice to have when using exceptions
    precisely because it's automatic, so unwinding the stack is much safer.
  • Paul Rubin at Dec 2, 2010 at 8:21 pm

    Tim Harig <usernet at ilthio.net> writes:
    I am not talking about what setjmp() has to do, I am talking about what
    *you* have to do after setjmp() returns. If you have allocated memory in
    intermediate functions and you don't have a reference to them outside of
    the functions that longjmp() bypasses from returning properly (and thus
    either not clearning data structures or returning a reference to those data
    structures as it normally would) then you have potential memory leaks,
    dangling pointers, etc.
    Sure, that's what the aux stack is for--you put any such references into
    it, for the setjmp handler to find later. You do that BEFORE setjmp
    returns, of course.
    I am not saying that this cannot be done. What I am saying is that it
    is inherently error prone.
    I suppose so, but so is everything else in C. On the overall scale of
    C-related hazards, this particular one isn't so bad if you code in a
    consistent style and are disciplined about recording the cleanups.

    You could also use something like an obstack, which is a stack allocated
    on the heap, so it persists after the control stack returns, but you can
    release the whole thing in one operation.
  • Tim Harig at Dec 2, 2010 at 8:51 pm

    On 2010-12-02, Paul Rubin wrote:
    Tim Harig <usernet at ilthio.net> writes:
    I am not talking about what setjmp() has to do, I am talking about what
    *you* have to do after setjmp() returns. If you have allocated memory in
    intermediate functions and you don't have a reference to them outside of
    the functions that longjmp() bypasses from returning properly (and thus
    either not clearning data structures or returning a reference to those data
    structures as it normally would) then you have potential memory leaks,
    dangling pointers, etc.
    Sure, that's what the aux stack is for--you put any such references into
    it, for the setjmp handler to find later. You do that BEFORE setjmp
    returns, of course.
    If you miss something, you are in trouble.

    There is a concept of variable life that is measured by how many lines
    separate the use of variable from its first use to its last. By using
    setjmp/longjmp, you effectively extend the life of these variables,
    potentially through several files, to at least as long as the jump. If
    there are several function calls in depth, there may be quite a lot of
    space that you have to check to make sure that you have not missed
    anything.
    I am not saying that this cannot be done. What I am saying is that it
    is inherently error prone.
    I suppose so, but so is everything else in C. On the overall scale of
    C-related hazards, this particular one isn't so bad if you code in a
    consistent style and are disciplined about recording the cleanups.

    You could also use something like an obstack, which is a stack allocated
    on the heap, so it persists after the control stack returns, but you can
    release the whole thing in one operation.
    By working the error back up through the call stack, you can keep track of
    the variables and allocations in each function isolated to that function.
    The smaller each function is, the easier and less error prone it will be
    to theck it is to check. That makes it much easier to make sure that
    you have not missed anything. Essentially, you can validate that each
    function correctly handles is allocations rather then having to validate
    the setjmp/longjmp structure as a whole. To use Joe Armstrong's phrase,
    "it makes the impossible merely difficult."

    Back to the topic, by using Python with its exceptions and garbage
    collection, all of this is a moot point.
  • Tim Harig at Dec 2, 2010 at 8:32 pm

    On 2010-12-02, MRAB wrote:
    On 02/12/2010 19:15, Tim Harig wrote:
    On 2010-12-02, Paul Rubinwrote:
    Tim Harig<usernet at ilthio.net> writes:
    longjmp. Alternatively you can have an auxiliary stack of cleanup
    records that the longjmp handler walks through. Of course if you do
    Only if you already have pointers to *all* of the data structures at
    the point where you put your setjmp().
    The setjmp point only has to know where the aux stack is and its depth
    when the longjmp happens. The cleanup records contain any necessary
    pointers to data structures that need freeing. That is basically how
    try/finally would do it too. This is pretty standard stuff.
    I am not talking about what setjmp() has to do, I am talking about what
    *you* have to do after setjmp() returns. If you have allocated memory in
    intermediate functions and you don't have a reference to them outside of
    the functions that longjmp() bypasses from returning properly (and thus
    either not clearning data structures or returning a reference to those data
    structures as it normally would) then you have potential memory leaks,
    dangling pointers, etc.

    I am not saying that this cannot be done. What I am saying is that it
    is inherently error prone.
    Automatic garbage collection is nice to have when using exceptions
    precisely because it's automatic, so unwinding the stack is much safer.
    Indeed.
  • Grant Edwards at Dec 2, 2010 at 6:36 pm

    On 2010-12-02, Paul Rubin wrote:
    MRAB <python at mrabarnett.plus.com> writes:
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.
    That's called longjmp.
    In theory.

    In practice, using longjump for that without blowing your foot off
    isn't easy.
    Automatic garbage collection would also have been nice.
    alloca might help.
    And that's almost as easy to screw up. :)

    --
    Grant Edwards grant.b.edwards Yow! I want EARS! I want
    at two ROUND BLACK EARS
    gmail.com to make me feel warm
    'n secure!!
  • MRAB at Dec 2, 2010 at 6:41 pm

    On 02/12/2010 18:09, Paul Rubin wrote:
    MRAB<python at mrabarnett.plus.com> writes:
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.
    That's called longjmp.
    The problem with that is that the caller might have to do some tidying
    up, such as deallocation.

    Exceptions give the caller the chance of catching it (ideally in a
    'finally' block), tidying up, and then propagating.
    Automatic garbage collection would also have been nice.
    alloca might help.
    I didn't know about that.

    It looks like that's allocated on the stack, and the allocation I'm
    talking must be on the heap, so it's not suitable anyway.
  • John Nagle at Dec 6, 2010 at 6:50 pm

    On 12/2/2010 10:09 AM, Paul Rubin wrote:
    MRAB<python at mrabarnett.plus.com> writes:
    When writing the C code for the new regex module I thought that it
    would've been easier if I could've used exceptions to propagate errors
    and unwind the stack, instead of having to return an error code which
    had to be checked by the caller, and then have the caller explicitly
    return an error code to /its/ caller.
    That's called longjmp.
    Automatic garbage collection would also have been nice.
    alloca might help.
    If you want proper exception unwinding, use C++, which
    has it. "longjmp" is a hack from the PDP-11 era.

    John Nagle
  • Alex23 at Dec 4, 2010 at 2:10 am

    On Dec 3, 2:12?am, Tim Harig wrote:
    Actually, I thought that debate was resolved years ago. ?I cannot think of
    a single recently developed programming language that does not provide
    exception handling mechanisms because they have been proven more reliable.
    Google's Go lacks exceptions and I believe that was a deliberate
    design choice.
  • Tim Harig at Dec 4, 2010 at 4:14 am

    On 2010-12-04, alex23 wrote:
    On Dec 3, 2:12?am, Tim Harig wrote:
    Actually, I thought that debate was resolved years ago. ?I cannot think of
    a single recently developed programming language that does not provide
    exception handling mechanisms because they have been proven more reliable.
    Google's Go lacks exceptions and I believe that was a deliberate
    design choice.
    1. The debate that I was referring to was between simple function checking
    vs. everything else. I didn't mean to automatically proclude any
    newer methodologies of which I might not even be aware.

    2. I would consider the defer/panic/recovery mechanism functionally similar
    to exceptions in most ways. It allows the error handling
    code to be placed at a higher level and panics tranverse the stack
    until they are handled by a recovery. This is basically equivilent
    to how exceptions work using different names. The change is basically the defer
    function which solves the problem of any cleanup work that the
    function needs to do before the panic is raised. I like it, its
    nice. It formalizes the pattern of cleaning up within an exception
    block and re-raising the exception.

    I do have to wonder what patterns will emerge in the object given
    to panic(). Since it takes anything, and since Go doesn't have an
    object hierarchy, much less an exception hierarchy, the panic value
    raised may or may not contain the kind of detailed information that
    can be obtained about the error that we are able to get from the
    Exception objects in Python.
  • Mark Wooding at Dec 2, 2010 at 4:35 pm

    Harishankar <v.harishankar at gmail.com> writes:

    There are some reasons why I hate exceptions but that is a different
    topic. However, in short I can say that personally:

    1. I hate try blocks which add complexity to the code when none is
    needed. Try blocks make code much more unreadable in my view and I use it
    only for the built-in exceptions when absolutely needed.
    Very little code actually needs `try' blocks.
    2. I prefer the less irksome True or False to do error checking.
    Exceptions seem too heavyweight for simple problems.
    Just write simple programs as if errors never happen; if an exception is
    raised, the program prints a vaguely useful message and exits.

    Besides, this complaint is incoherent. One needs at least as much
    structure to check an error code as to catch an exception.
    3. Philosophically I think exception handling is the wrong approach to
    error management.
    There are better ways to handle errors than Python's exception system.
    Passing error codes around manually is most definitely not one of them.

    (One of the main reasons I ditched Perl in favour of Python is the
    former's insistence on returning error codes for I/O and system calls.)

    -- [mdw]
  • Steven D'Aprano at Dec 3, 2010 at 6:46 am

    On Thu, 02 Dec 2010 16:35:08 +0000, Mark Wooding wrote:

    3. Philosophically I think exception handling is the wrong approach to
    error management.
    There are better ways to handle errors than Python's exception system.
    I'm curious -- what ways would they be?

    I'm aware of three general exception handling techniques:

    1. return a sentinel value or error code to indicate an exceptional case
    (e.g. str.find returns -1);

    2. raise an exception (e.g. nearly everything else in Python);

    3. set an error code somewhere (often a global variable) and hope the
    caller remembers to check it;

    plus some de facto techniques sadly in common use:

    4. dump core;

    5. do nothing and produce garbage output.


    What else is there?


    --
    Steven
  • Paul Rubin at Dec 3, 2010 at 7:25 am

    Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
    There are better ways to handle errors than Python's exception system.
    I'm curious -- what ways would they be?
    I'm aware of three general exception handling techniques: ...
    What else is there?
    The Erlang approach is to chop the application into a lot of very
    lightweight processes, and let any process that encounters an error
    simply crash. A monitoring process notices the crashed process and
    restarts it. There is a "supervision tree" of uber-monitor processes
    that restart crashed monitor proceses. I haven't programmed in that
    style myself and I'm not persuaded that it's better than what Python
    does, but I think it's different from the stuff on your list, which is
    an answer to your "what else is there". I do know that they write some
    complex, very high reliability systems (phone switches) in Erlang.
  • Tim Harig at Dec 3, 2010 at 8:43 am

    On 2010-12-03, Paul Rubin wrote:
    Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
    There are better ways to handle errors than Python's exception system.
    I'm curious -- what ways would they be?
    I'm aware of three general exception handling techniques: ...
    What else is there?
    The Erlang approach is to chop the application into a lot of very
    lightweight processes, and let any process that encounters an error
    simply crash. A monitoring process notices the crashed process and
    restarts it. There is a "supervision tree" of uber-monitor processes
    that restart crashed monitor proceses. I haven't programmed in that
    style myself and I'm not persuaded that it's better than what Python
    does, but I think it's different from the stuff on your list, which is
    Erlang also offers an exception syntax almost identical to Python's for use
    within a single process.

    What makes Erlang's supervisor mode of error handling superior is that it
    works for more then just the current vm. If a process throws an exception,
    the supervisor catches and handles it. If a vm dies, a supervisor from
    another vm takes over. If an entire computer dies, a supervisor on another
    computer takes over. OTP provides some extremely advanced support for
    supervisory structures.
    an answer to your "what else is there". I do know that they write some
    complex, very high reliability systems (phone switches) in Erlang.
    Erlang isn't what I would call a very general purpose programming language
    like Python; but, if you want to build highly scalable and/or highly
    available systemes, there really isn't anything else that comes close
    to it. I am not really a huge fan of the purely functional nature of
    the language; but, light weight processes using the actor model is the
    way to go for concurrent processing.

    The BEAM virtual machine is also a powerful system with its ability to
    patch systems on the fly. It has start to become the target for other
    languages. I know of two that are in current developement. I wouldn't
    mind seeing a version of Python that could leverage the power of the
    BEAM vm.
  • Mark Wooding at Dec 3, 2010 at 2:31 pm

    Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
    On Thu, 02 Dec 2010 16:35:08 +0000, Mark Wooding wrote:
    There are better ways to handle errors than Python's exception system.
    I'm curious -- what ways would they be?
    The most obvious improvement is resumable exceptions.

    In general, recovering from an exceptional condition requires three
    activities:

    * doing something about the condition so that the program can continue
    running;

    * identifying some way of rejoining the program's main (unexceptional)
    flow of control; and

    * actually performing that transfer, ensuring that any necessary
    invariants are restored.

    Python's `try ... finally' helps with the last; but Python intertwines
    the first two, implementing both with `try ... except'. The most
    important consequence of this is that the logic which contains knowledge
    about how to fix the condition must be closer to the code that
    encountered the condition than the resume point is. It's therefore hard
    to factor out high-level policy about fixing conditions from the
    relatively tedious business of providing safe points at which to resume
    main execution. (Effectively, each section of your program which wants
    to avail itself of some high-level condition-fixing policy has to
    provide its own protocol for expressing and implementing them.)

    Phew. That was abstract. Can I come up with some examples?

    I've been writing a (sort of) compiler recently. When it encounters
    errors, it reports a message to the user containing the position it had
    reached in the source, updates a counter so that it can report a summary
    at the end of the run and produce a sensible exit status, and then
    attempts to carry on compiling as best it can.

    The last bit -- attempting to carry on as best it can -- needs local
    knowledge of what the compiler's doing and what actually went wrong. If
    the parser expected to find a delimiter, maybe it should pretend that it
    found one, for example.

    The other stuff, printing messages, updating counters, and so on, is all
    done with some condition handlers wrapped around the meat of the
    compiler. That's written only once. Everything that signals errors,
    including standard I/O functions like open-a-file, gets the same
    treatment.

    (The remaining missing ingredient is a fluid variable which tracks the
    current position in the source and is updated by the scanner; bits of
    the compiler's semantic analysis machinery will temporarily focus
    attention on other parts of the source using locations they saved during
    the parse. Implementing fluids in Python can be done with a context
    manager: if you don't care about concurrency then you can use simple
    variables; otherwise it's little fiddly and the syntax isn't very
    comfortable, but it's still possible.)

    A more familiar example, maybe, is the good old DOS `abort/retry/fail'
    query. Implementing such a thing in Python, as a general policy for
    handling I/O errors, isn't possible. Viewed narrowly, this is probably
    a good thing: the old DOS query was very annoying. But the difficulty
    of implementing this policy illustrates the missing functionality. And,
    of course, if DOS had a proper resumable exception system, programs
    could have overridden the annoying query.

    In general, the code which discovers an exceptional condition may have
    several options for how to resume. It might be possible to ignore the
    situation entirely and carry on regardless (`false alarm!'). It might
    be possible to try again (`transient failure'). Alas, the logic which
    is capable of implementing these options is usually too low-level and
    too close to the action to be able to decide among them sensibly.
    (Maybe a human user should be consulted -- but that can drag in user
    interface baggage into a low-level networking library or whatever.)
    Resumable exceptions provide a way out of this mess by separating the
    mechanics of resumption from policy of which resumption option to
    choose.

    It's easy to show that a resumable exception system can do everything
    that a nonresumable system (like Python's) can do (simply put all of the
    recovery logic at the resume point); but the converse is not true.

    There are some other fringe benefits to resumable exceptions.

    * It's usual to report a stack backtrace or similar if an exception
    occurs but nothing manages to resume execution. If unwinding the
    stack is intertwined with working out how to resume execution, then
    whenever you /try/ to run an applicable handler, you have to reify
    the stack context and stash it somewhere in case the handler doesn't
    complete the job. This makes raising exceptions more expensive than
    they need to be.

    * You can use the same mechanism for other kinds of communication with
    surrounding context. For example, Python occasionally emits
    `warnings', which have their own rather simple management system
    (using global state, so it's hard to say `don't issue MumbleWarnings
    while we frob the widgets' in a concurrent program). A resumable
    exceptions system could easily integrate warnings fully with other
    kinds of conditions (and avoid the concurrency problems). You could
    also use it for reporting progress indications, for example.

    Of course, there's a downside. Resumable exceptions aren't the usual
    kind, so people aren't used to them. I'm not sure whether resumable
    exceptions are actually more complicated to understand: there are more
    named concepts involved, but they do less and their various roles are
    clearer and less tangled. (The `handler' for an exceptional condition
    can be called just like a function. Python doesn't have nonlocal flow
    control distinct from its exception system, but a nonlocal transfer to a
    resumption point isn't conceptually very complicated.)
    1. return a sentinel value or error code to indicate an exceptional case
    (e.g. str.find returns -1);
    This works fine if you consider failure as being unexceptional. If
    you're not actually expecting to find that substring, and have something
    sensible to do if it wasn't there, a report that it wasn't there isn't
    really exceptional.

    (I think I use str.find more frequently than str.index, so it's nice
    that there are both.)
    2. raise an exception (e.g. nearly everything else in Python);
    Raising exceptions is a more complicated business than this suggests.
    Python made some specific design decisions regarding its exception
    system; they're pretty common choices, but not, I think, the best ones.
    3. set an error code somewhere (often a global variable) and hope the
    caller remembers to check it;
    That's less common than you might think. Usually there's some sentinel
    value to tell you to look at the global error code. (Obvious examples
    where there isn't a clear sentinel: strtol and friends, math.h.)

    I think we can agree that this sucks.

    Someone else mentioned Erlang. An Erlang system is structured as a
    collection of `processes' (they don't have any shared state, so they
    aren't really `threads') which communicate by sending messages to each
    other. If an Erlang process encounters a problem, it dies, and a
    message is sent to report its demise. Erlang processes are very cheap,
    it's not unusual for a system to have tens of thousands of them.
    plus some de facto techniques sadly in common use:

    4. dump core;

    5. do nothing and produce garbage output.

    What else is there?
    You missed `6. assume that erroneous input is actually executable code
    and transfer control to it', which is a popular approach in C.

    -- [mdw]
  • Harishankar at Dec 3, 2010 at 5:16 pm

    On Fri, 03 Dec 2010 14:31:43 +0000, Mark Wooding wrote:

    The most obvious improvement is resumable exceptions.
    This is probably what I had in mind but I just couldn't explain it the
    way you did below.
    In general, recovering from an exceptional condition requires three
    activities:

    * doing something about the condition so that the program can continue
    running;

    * identifying some way of rejoining the program's main (unexceptional)
    flow of control; and

    * actually performing that transfer, ensuring that any necessary
    invariants are restored.
    This really sums up my thoughts about exceptions better than I could have
    explained! I just felt instinctively that I had missed something, but it
    appears to be a break in logic of the code somewhere which I thought was
    my fault. Seems that exception handling requires a lot of forethought
    since the control of program execution breaks at the point of exception
    with no obvious way to rejoin it seamlessly whereas with an error, a
    simple if condition could handle the error state and resume execution
    from that point forward. This is the main reason why I think I used
    simple error codes to handle certain recoverable conditions and avoided
    exceptions.

    I quite enjoyed your post. Thank you for explaining a lot of issues which
    I probably could not have figured out on my own.
    --
    Harishankar (http://harishankar.org http://lawstudentscommunity.com)
  • Emile van Sebille at Dec 3, 2010 at 6:53 pm
    On 12/3/2010 6:31 AM Mark Wooding said...
    It's easy to show that a resumable exception system can do everything
    that a nonresumable system (like Python's) can do (simply put all of the
    recovery logic at the resume point); but the converse is not true.

    There are some other fringe benefits to resumable exceptions.
    I do a lot of work in a variant of Business Basic that has always
    offered resumable exceptions. The closest I get in python is using
    import pdb;pdb.set_trace(). I wonder what it would take to allow for
    any exceptions occurring outside a try/except context to dump the
    traceback, then invoke pdb.set_trace() before bailing to allow for both
    investigation and possible recovery and continuance?

    Emile
  • Tim Harig at Dec 4, 2010 at 4:25 am

    On 2010-12-03, Harishankar wrote:
    On Fri, 03 Dec 2010 14:31:43 +0000, Mark Wooding wrote:
    In general, recovering from an exceptional condition requires three
    activities:

    * doing something about the condition so that the program can continue
    running;

    * identifying some way of rejoining the program's main (unexceptional)
    flow of control; and

    * actually performing that transfer, ensuring that any necessary
    invariants are restored.
    my fault. Seems that exception handling requires a lot of forethought
    since the control of program execution breaks at the point of exception
    with no obvious way to rejoin it seamlessly whereas with an error, a
    simple if condition could handle the error state and resume execution
    from that point forward. This is the main reason why I think I used
    simple error codes to handle certain recoverable conditions and avoided
    exceptions.
    If you are returning an error code to the above function, then there is
    nothing that you cannot do with with the exception. Basically, you resolve
    the issue in your except block just as you would in the block of your if
    statement after returning the error code. If you try and fail to handle
    the exception or just needed to do some cleanup before allowing the
    exception to continue, then you just re-raise the exception.
  • Paul Rubin at Dec 6, 2010 at 8:14 am

    mdw at distorted.org.uk (Mark Wooding) writes:
    The most obvious improvement is resumable exceptions.
    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue. Are there any languages out there with
    resumable exceptions? Escaping to a debugger doesn't really count as
    that. I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns, but
    doing it in a single-threaded system just seems full of hazards.
  • Steve Holden at Dec 6, 2010 at 8:40 am

    On 12/6/2010 9:14 AM, Paul Rubin wrote:
    mdw at distorted.org.uk (Mark Wooding) writes:
    The most obvious improvement is resumable exceptions.
    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue. Are there any languages out there with
    resumable exceptions? Escaping to a debugger doesn't really count as
    that. I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns, but
    doing it in a single-threaded system just seems full of hazards.
    I seem to remember PL/1 has resumable exceptions, but I don't ever
    remember finding a real use for them. And it's so long since I used PL/1
    I may be mistaken.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
    See Python Video! http://python.mirocommunity.org/
    Holden Web LLC http://www.holdenweb.com/
  • Mel at Dec 6, 2010 at 1:32 pm

    Paul Rubin wrote:

    mdw at distorted.org.uk (Mark Wooding) writes:
    The most obvious improvement is resumable exceptions.
    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue. Are there any languages out there with
    resumable exceptions? Escaping to a debugger doesn't really count as
    that. I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns, but
    doing it in a single-threaded system just seems full of hazards.
    Apparently, at the end of his research, Alan Turing was trying out the idea
    of 'oracles', where a computable process would have access to an
    uncomputable process to get particular results. I would imagine that the
    idea here was to clarify what this would do to the computable process. If
    he had lived, I doubt that Turing would have built an oracle, but the idea
    does live on in interactive debuggers.

    It would seem if some situation has arisen that can be fixed by code, then
    you can just run that code there and then. Then 'resumable exceptions' just
    become a kind of subroutine call, perhaps like the triggers in SQL.

    Mel.
  • Nobody at Dec 7, 2010 at 2:26 am

    On Mon, 06 Dec 2010 08:32:18 -0500, Mel wrote:

    Apparently, at the end of his research, Alan Turing was trying out the idea
    of 'oracles', where a computable process would have access to an
    uncomputable process to get particular results. I would imagine that the
    idea here was to clarify what this would do to the computable process. If
    he had lived, I doubt that Turing would have built an oracle, but the idea
    does live on in interactive debuggers.
    The "oracle" concept was introduced quite early on in Turing's work, late
    1930s. The idea is to examine the complexity of problems relative to other
    problems. E.g. if you have a Turing machine with access to an oracle which
    can solve some NP-complete problem, you can analyse the complexity of
    solving other NP-complete problems in that context.
  • Martin Gregorie at Dec 6, 2010 at 6:59 pm

    On Mon, 06 Dec 2010 09:54:46 -0800, Dennis Lee Bieber wrote:

    On Mon, 06 Dec 2010 00:14:11 -0800, Paul Rubin <no.email at nospam.invalid>
    declaimed the following in gmane.comp.python.general:

    exceptions that fixed the issue. Are there any languages out there
    with resumable exceptions? Escaping to a debugger doesn't really count
    as
    Visual BASIC 6

    -=-=-=-=-
    On Error GoTo line
    9999 REM Actions to sort out the error
    RESUME
    Enables the error-handling routine that starts at line
    specified in the required line argument. The line argument is any line
    label or line number. If a run-time error occurs, control branches to
    line, making the error handler active. The specified line must be in the
    same procedure as the On Error statement; otherwise, a compile-time
    error occurs.
    Any BASIC that implements ON ERROR (i.e. just about all of them) will do
    this, not just VB.


    --
    martin@ | Martin Gregorie
    gregorie. | Essex, UK
    org |
  • John Nagle at Dec 6, 2010 at 7:23 pm

    On 12/6/2010 12:40 AM, Steve Holden wrote:
    On 12/6/2010 9:14 AM, Paul Rubin wrote:
    mdw at distorted.org.uk (Mark Wooding) writes:
    The most obvious improvement is resumable exceptions.
    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue. Are there any languages out there with
    resumable exceptions? Escaping to a debugger doesn't really count as
    that. I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns, but
    doing it in a single-threaded system just seems full of hazards.
    I seem to remember PL/1 has resumable exceptions, but I don't ever
    remember finding a real use for them. And it's so long since I used PL/1
    I may be mistaken.
    Resumable exceptions were a popular idea in the early days of
    programming. LISP, PL/I, and early COBOL had constructs which
    could be considered resumable exceptions. They didn't work out
    well, because the exception handler gets control in an ambiguous
    situation, perhaps in the middle of an expression. Changing
    the state of the execution, then returning, can leave the program
    in an invalid state.

    Signal handling has many of the same problems. A POSIX signal
    is a forced subroutine call while something else is going on,
    which is in itself a weird concept. That's what a resumable
    exception looks like. CPython has a terrible time with signal
    handling. See "http://www.dabeaz.com/python/UnderstandingGIL.pdf"
    for the whole ugly mess. That's why control-C won't terminate
    multi-thread programs, among other things.

    Unwinding cleanly from a signal is difficult, but possible
    with proper CPU and compiler design. It's done right in Ada, and
    in Visual C++ for x86 on Windows. Only some CPUs support "exact"
    floating point exceptions, where you're guaranteed that the
    exception comes in at the point where the problem occurred.
    In modern superscalar CPUs, the exception comes in several
    instructions after the problem was detected. In x86 type
    CPUs, the CPU hardware in the "retirement unit" backs up the
    CPU state to the point at which the exception was detected.
    PowerPC and SPARC CPUs do not do this; if you need exactness
    in exception position on them, you have to put in "fence" instructions
    to stop lookahead. This costs performance.

    As a result, C code which unwinds from signals via "longjmp"
    is not portable.
    See
    "https://www.securecoding.cert.org/confluence/display/seccode/SIG32-C.+Do+not+call+longjmp%28%29+from+inside+a+signal+handler"
    Nor is changing the program state from inside a signal handler.
    You're not entirely sure, on many CPUs, where control is at the
    point the signal came in.

    (In a physics simulator, I once had to handle floating point
    overflow, which indicated that the computation had to be backed
    up and rerun with a smaller time step. It's possible to do this
    safely under Windows on x86 if you read all the appropriate documents.
    It's not portable. That's why I'm aware of this mess.)

    So that's why resumable exceptions are a bad idea.

    John Nagle
  • Mark Wooding at Dec 6, 2010 at 9:23 pm

    John Nagle <nagle at animats.com> writes:

    Resumable exceptions were a popular idea in the early days of
    programming. LISP, PL/I, and early COBOL had constructs which could
    be considered resumable exceptions. They didn't work out well,
    because the exception handler gets control in an ambiguous situation,
    perhaps in the middle of an expression. Changing the state of the
    execution, then returning, can leave the program in an invalid state.
    Right, but that's not the important really important trick. The
    important bit is separating out the `how should I fix this?' logic from
    the point where execution should resume. There's no good reason why the
    former should have to come from a dynamic context smaller than the
    latter: it's just an unnecessary conflation.

    [Snip stuff about signals. I agree that Unix signals are a disaster.]
    So that's why resumable exceptions are a bad idea.
    That's why a primitive resumable exception system, used naively, is a
    bad idea. Now look at the good ones.

    -- [mdw]
  • Mark Wooding at Dec 6, 2010 at 8:58 pm

    Paul Rubin <no.email at nospam.invalid> writes:

    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue.
    That seems very surprising to me.
    Are there any languages out there with resumable exceptions?
    Common Lisp and Smalltalk spring to mind. It's fairly straightforward
    to write one in Scheme. (Actually, implementing the Common Lisp one in
    terms of fluids, closures and blocks isn't especially difficult.)
    Escaping to a debugger doesn't really count as that.
    Indeed not.
    I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns,
    but doing it in a single-threaded system just seems full of hazards.
    It seems pretty straightforward to me. Handlers are simply closures;
    the registered handlers are part of the prevailing dynamic context.
    When an exception occurs, you invoke the handlers, most-recently
    registered first. A handler that returns normally can be thought of as
    `declining' to handle the exception; a handler that explicitly transfers
    control elsewhere can be thought of as having handled it.

    To make this work, all you need is:

    * a fluid list (i.e., one which is part of the dynamic context) of
    handlers, which you can build in pure Python if you try hard enough
    (see below);

    * closures to represent handlers, which Python has already, and;

    * a nonlocal transfer mechanism, and a mechanism like try ... finally
    to allow functions to clean up if they're unwound.

    We can actually come up with a nonlocal transfer if we try, by abusing
    exceptions.

    [The code in this article is lightly tested, but probably contains
    stupid bugs. Be careful.]

    class block (object):
    """
    Context manager for escapable blocks.

    Write

    with block() as escape:
    ...

    Invoking the `escape' function causes the context body to exit
    immediately. Invoking the `escape' function outside of the
    block's dynamic context raises a ValueError.
    """
    def __init__(me):
    me._tag = None
    def _escape(me, value = None):
    if me._tag is None:
    raise ValueError, 'defunct block'
    me.result = value
    raise me._tag
    def __enter__(me, value = None):
    if me._tag:
    raise ValueError, 'block already active'
    me._tag = type('block tag', (BaseException,), {})
    me.result = value
    return me._escape
    def __exit__(me, ty, val, tb):
    tag, me._tag = me._tag, None
    return ty is tag

    This is somewhat brittle, since some intervening context might capture
    the custom exception we're using, but I don't think we can do
    significantly better.

    Implementing fluids badly is easy. Effectively what we'd do to bind a
    fluid dynamically is

    try:
    old, fluid = fluid, new
    ...
    finally:
    fluid = old

    but this is visible in other threads. The following will do the job in
    a multithreaded environment.

    import threading as T
    import weakref as W

    class FluidBinding (object):
    """Context object for fluid bindings."""
    def __init__(me, fluid, value):
    me.fluid = fluid
    me.value = value
    def __enter__(me):
    me.fluid._bind(me.value)
    def __exit__(me, ty, val, tb):
    me.fluid._unbind()

    class Fluid (object):
    """
    Represents a fluid variable, i.e., one whose binding respects
    the dynamic context rather than the lexical context.

    Read and write the Fluid through the `value' property.

    The global value is shared by all threads. To dynamically
    bind the fluid, use the context manager `binding':

    with myfluid.binding(NEWVALUE):
    ...

    The binding is visible in functions called MAP within the
    context body, but not in other threads.
    """

    _TLS = T.local()
    _UNBOUND = ['fluid unbound']
    _OMIT = ['fluid omitted']

    def __init__(me, value = _UNBOUND):
    """
    Iinitialze a fluid, optionally setting the global value.
    """
    me._value = value

    @property
    def value(me):
    """
    Return the current value of the fluid.

    Raises AttributeError if the fluid is currently unbound.
    """
    try:
    value, _ = me._TLS.map[me]
    except (AttributeError, KeyError):
    value = me._value
    if value == me._UNBOUND:
    raise AttributeError, 'unbound fluid'
    return value
    @value.setter
    def value(me, value):
    try:
    map = me._TLS.map
    _, stack = map[me]
    map[me] = value, stack
    except (AttributeError, KeyError):
    me._value = value
    @value.deleter
    def value(me):
    me.value = me._UNBOUND

    def binding(me, value = _OMIT, unbound = False):
    """
    Bind the fluid dynamically.

    If UNBOUND is true then make the fluid be `unbound', i.e.,
    not associated with a value. Otherwise, if VALUE is unset,
    then preserve the current value. Otherwise, set it to
    VALUE.

    The fluid can be modified and deleted. This will not affect
    the value outside of the dynamic extent of the context
    (e.g., in other threads, or when the context is unwound).
    """
    if unbound:
    value = me._UNBOUND
    elif value == me._OMIT:
    value = me.value
    return _FluidBinding(me, value)

    def _bind(me, value):
    try:
    map = me._TLS.map
    except AttributeError:
    me._TLS.map = map = W.WeakKeyDictionary()
    try:
    old, stack = map[me]
    stack.append(old)
    map[me] = value, stack
    except KeyError:
    map[me] = value, []

    def _unbind(me):
    map = me._TLS.map
    _, stack = map[me]
    if stack:
    map[me] = stack.pop(), stack
    else:
    del map[me]

    Now we can say

    with fluid.binding(new):
    ...

    and all is well.

    So, how do we piece all of this together to make a resumable exception
    system?

    We're going to need to keep a list of handlers. We're going to be
    adding and removing stuff a lot; and we want to make use of the fluid
    mechanism we've already built, which will restore old values
    automatically when we leave a dynamic context. So maintaining a linked
    list seems like a good idea. The nodes in the list will look somewhat
    like this.

    class Link (object):
    def __init__(me, item, next):
    me.item = item
    me.next = next

    Our handlers are going to be simple functions which take exception
    objects as arguments. A more advanced handler might filter exceptions
    based on their classes. That's not especially difficult to do badly,
    but it's fiddly to do well and it doesn't shed much light on the overall
    mechanism, so I'll omit that complication.

    We'll want a fluid for the handler list.

    HANDLERS = Fluid(None)

    Now we want to run a chunk of code with a handler attached. This seems
    like another good use for a context manager.

    class handler (object):
    def __init__(me, func):
    me._func = func
    def __enter__(me):
    me._bind = FluidBinding(HANDLERS,
    Link(me.func, HANDLERS.value)
    me._bind.__enter__()
    def __exit__(me, ty, val, tb):
    return me._bind.__exit__(ty, val, tb)

    (Context managers don't compose very nicely. It'd be prettier with the
    contextmanager decorator.)

    Let's say that we `signal' resumable exceptions rather than `raising'
    them. How do we do that?

    def signal(exc):
    with HANDLERS.binding():
    while HANDLERS.value:
    h = HANDLERS.value
    HANDLERS.value = h.next
    h.item(exc)

    Yes, if all of the handlers decline, we just return. This is Bad for
    errors, but good for other kinds of situations, so `signal' is a
    convenient substrate to build on.

    def error(exc):
    signal(exc)
    raise RuntimeError, 'unhandled resumable exception'

    def warning(exc):
    signal(exc)
    ## Crank up python's usual warning stuff

    Note also that handlers are invoked in a dynamic environment which
    doesn't include them or any handlers added since. Obviously they can
    install their own handlers just fine.

    Cool. Now how about recovery? This is where nonlocal transfer comes
    in. If a handler wants to take responsibility for the exception, it has
    to make a nonlocal transfer. Where should it go? Let's maintain a
    table of restart points. Again, it'll be a linked list.

    RESTARTS = Fluid(None)

    class restart (block):
    def __init__(me, name):
    me.name = name
    super(restart, me).__init__(me)
    def invoke(me, value = None):
    me._escape(value)
    def __enter__(me):
    me._bind = FluidBinding(RESTARTS, Link(me, RESTARTS.value))
    me._bind.__enter__()
    return super(restart, me).__enter__()
    def __exit__(me, ty, val, tb):
    ## Poor man's PROG1.
    try:
    return super(restart, me).__exit__(ty, val, tb)
    finally:
    me._bind.__exit__(ty, val, tb)

    def find_restart(name):
    r = RESTARTS.value
    while r:
    if r.item.name == name:
    return r.item
    r = r.next
    return None

    Using all of this is rather cumbersome, and Python doesn't allow
    syntactic abstraction so there isn't really much we can do to sweeten
    the pill. But I ought to provide an example of this machinery in
    action.

    def toy(x, y):
    r = restart('use-value')
    with r:
    if y == 0:
    error(ZeroDivisionError())
    r.result = x/y
    return r.result

    def example():
    def zd(exc):
    if not isinstance(exc, ZeroDivisionError):
    return
    r = find_restart('use-value')
    if not r: return
    r.invoke(42)
    print toy(5, 2)
    with handler(zd):
    print toy(1, 0)

    Does any of that help?

    -- [mdw]
  • Carl Banks at Dec 6, 2010 at 10:42 pm

    On Dec 6, 12:58?pm, m... at distorted.org.uk (Mark Wooding) wrote:
    Paul Rubin <no.em... at nospam.invalid> writes:
    You know, I've heard the story from language designers several times
    over, that they tried putting resumable exceptions into their languages
    and it turned out to be a big mess, so they went to termination
    exceptions that fixed the issue.
    That seems very surprising to me.
    Are there any languages out there with resumable exceptions?
    Common Lisp and Smalltalk spring to mind. ?It's fairly straightforward
    to write one in Scheme. ?(Actually, implementing the Common Lisp one in
    terms of fluids, closures and blocks isn't especially difficult.)
    Escaping to a debugger doesn't really count as that.
    Indeed not.
    I guess one way to do it would be call a coroutine to handle the
    exception, and either continue or unwind after the continue returns,
    but doing it in a single-threaded system just seems full of hazards.
    It seems pretty straightforward to me. ?Handlers are simply closures;
    the registered handlers are part of the prevailing dynamic context.
    When an exception occurs, you invoke the handlers, most-recently
    registered first. ?A handler that returns normally can be thought of as
    `declining' to handle the exception; a handler that explicitly transfers
    control elsewhere can be thought of as having handled it.

    To make this work, all you need is:

    ? * a fluid list (i.e., one which is part of the dynamic context) of
    ? ? handlers, which you can build in pure Python if you try hard enough
    ? ? (see below);

    ? * closures to represent handlers, which Python has already, and;

    ? * a nonlocal transfer mechanism, and a mechanism like try ... finally
    ? ? to allow functions to clean up if they're unwound.

    We can actually come up with a nonlocal transfer if we try, by abusing
    exceptions.

    [The code in this article is lightly tested, but probably contains
    stupid bugs. ?Be careful.]

    ? ? ? ? class block (object):
    ? ? ? ? ? """
    ? ? ? ? ? Context manager for escapable blocks.

    ? ? ? ? ? Write

    ? ? ? ? ? ? ? ? with block() as escape:
    ? ? ? ? ? ? ? ? ? ...

    ? ? ? ? ? Invoking the `escape' function causes the context body to exit
    ? ? ? ? ? immediately. ?Invoking the `escape' function outside of the
    ? ? ? ? ? block's dynamic context raises a ValueError.
    ? ? ? ? ? """
    ? ? ? ? ? def __init__(me):
    ? ? ? ? ? ? me._tag = None
    ? ? ? ? ? def _escape(me, value = None):
    ? ? ? ? ? ? if me._tag is None:
    ? ? ? ? ? ? ? raise ValueError, 'defunct block'
    ? ? ? ? ? ? me.result = value
    ? ? ? ? ? ? raise me._tag
    ? ? ? ? ? def __enter__(me, value = None):
    ? ? ? ? ? ? if me._tag:
    ? ? ? ? ? ? ? raise ValueError, 'block already active'
    ? ? ? ? ? ? me._tag = type('block tag', (BaseException,), {})
    ? ? ? ? ? ? me.result = value
    ? ? ? ? ? ? return me._escape
    ? ? ? ? ? def __exit__(me, ty, val, tb):
    ? ? ? ? ? ? tag, me._tag = me._tag, None
    ? ? ? ? ? ? return ty is tag

    This is somewhat brittle, since some intervening context might capture
    the custom exception we're using, but I don't think we can do
    significantly better.

    Implementing fluids badly is easy. ?Effectively what we'd do to bind a
    fluid dynamically is

    ? ? ? ? try:
    ? ? ? ? ? old, fluid = fluid, new
    ? ? ? ? ? ...
    ? ? ? ? finally:
    ? ? ? ? ? fluid = old

    but this is visible in other threads. ?The following will do the job in
    a multithreaded environment.

    ? ? ? ? import threading as T
    ? ? ? ? import weakref as W

    ? ? ? ? class FluidBinding (object):
    ? ? ? ? ? """Context object for fluid bindings."""
    ? ? ? ? ? def __init__(me, fluid, value):
    ? ? ? ? ? ? me.fluid = fluid
    ? ? ? ? ? ? me.value = value
    ? ? ? ? ? def __enter__(me):
    ? ? ? ? ? ? me.fluid._bind(me.value)
    ? ? ? ? ? def __exit__(me, ty, val, tb):
    ? ? ? ? ? ? me.fluid._unbind()

    ? ? ? ? class Fluid (object):
    ? ? ? ? ? """
    ? ? ? ? ? Represents a fluid variable, i.e., one whose binding respects
    ? ? ? ? ? the dynamic context rather than the lexical context.

    ? ? ? ? ? Read and write the Fluid through the `value' property.

    ? ? ? ? ? The global value is shared by all threads. ?To dynamically
    ? ? ? ? ? bind the fluid, use the context manager `binding':

    ? ? ? ? ? ? ? ? ? with myfluid.binding(NEWVALUE):
    ? ? ? ? ? ? ? ? ? ? ...

    ? ? ? ? ? The binding is visible in functions called MAP within the
    ? ? ? ? ? context body, but not in other threads.
    ? ? ? ? ? """

    ? ? ? ? ? _TLS = T.local()
    ? ? ? ? ? _UNBOUND = ['fluid unbound']
    ? ? ? ? ? _OMIT = ['fluid omitted']

    ? ? ? ? ? def __init__(me, value = _UNBOUND):
    ? ? ? ? ? ? """
    ? ? ? ? ? ? Iinitialze a fluid, optionally setting the global value.
    ? ? ? ? ? ? """
    ? ? ? ? ? ? me._value = value

    ? ? ? ? ? @property
    ? ? ? ? ? def value(me):
    ? ? ? ? ? ? """
    ? ? ? ? ? ? Return the current value of the fluid.

    ? ? ? ? ? ? Raises AttributeError if the fluid is currently unbound.
    ? ? ? ? ? ? """
    ? ? ? ? ? ? try:
    ? ? ? ? ? ? ? value, _ = me._TLS.map[me]
    ? ? ? ? ? ? except (AttributeError, KeyError):
    ? ? ? ? ? ? ? value = me._value
    ? ? ? ? ? ? if value == me._UNBOUND:
    ? ? ? ? ? ? ? raise AttributeError, 'unbound fluid'
    ? ? ? ? ? ? return value
    ? ? ? ? ? @value.setter
    ? ? ? ? ? def value(me, value):
    ? ? ? ? ? ? try:
    ? ? ? ? ? ? ? map = me._TLS.map
    ? ? ? ? ? ? ? _, stack = map[me]
    ? ? ? ? ? ? ? map[me] = value, stack
    ? ? ? ? ? ? except (AttributeError, KeyError):
    ? ? ? ? ? ? ? me._value = value
    ? ? ? ? ? @value.deleter
    ? ? ? ? ? def value(me):
    ? ? ? ? ? ? me.value = me._UNBOUND

    ? ? ? ? ? def binding(me, value = _OMIT, unbound = False):
    ? ? ? ? ? ? """
    ? ? ? ? ? ? Bind the fluid dynamically.

    ? ? ? ? ? ? If UNBOUND is true then make the fluid be `unbound', i.e.,
    ? ? ? ? ? ? not associated with a value. ?Otherwise, if VALUE is unset,
    ? ? ? ? ? ? then preserve the current value. ?Otherwise, set it to
    ? ? ? ? ? ? VALUE.

    ? ? ? ? ? ? The fluid can be modified and deleted. ?This will not affect
    ? ? ? ? ? ? the value outside of the dynamic extent of the context
    ? ? ? ? ? ? (e.g., in other threads, or when the context is unwound).
    ? ? ? ? ? ? """
    ? ? ? ? ? ? if unbound:
    ? ? ? ? ? ? ? value = me._UNBOUND
    ? ? ? ? ? ? elif value == me._OMIT:
    ? ? ? ? ? ? ? value = me.value
    ? ? ? ? ? ? return _FluidBinding(me, value)

    ? ? ? ? ? def _bind(me, value):
    ? ? ? ? ? ? try:
    ? ? ? ? ? ? ? map = me._TLS.map
    ? ? ? ? ? ? except AttributeError:
    ? ? ? ? ? ? ? me._TLS.map = map = W.WeakKeyDictionary()
    ? ? ? ? ? ? try:
    ? ? ? ? ? ? ? old, stack = map[me]
    ? ? ? ? ? ? ? stack.append(old)
    ? ? ? ? ? ? ? map[me] = value, stack
    ? ? ? ? ? ? except KeyError:
    ? ? ? ? ? ? ? map[me] = value, []

    ? ? ? ? ? def _unbind(me):
    ? ? ? ? ? ? map = me._TLS.map
    ? ? ? ? ? ? _, stack = map[me]
    ? ? ? ? ? ? if stack:
    ? ? ? ? ? ? ? map[me] = stack.pop(), stack
    ? ? ? ? ? ? else:
    ? ? ? ? ? ? ? del map[me]

    Now we can say

    ? ? ? ? with fluid.binding(new):
    ? ? ? ? ? ...

    and all is well.

    So, how do we piece all of this together to make a resumable exception
    system?

    We're going to need to keep a list of handlers. ?We're going to be
    adding and removing stuff a lot; and we want to make use of the fluid
    mechanism we've already built, which will restore old values
    automatically when we leave a dynamic context. ?So maintaining a linked
    list seems like a good idea. ?The nodes in the list will look somewhat
    like this.

    ? ? ? ? class Link (object):
    ? ? ? ? ? def __init__(me, item, next):
    ? ? ? ? ? ? me.item = item
    ? ? ? ? ? ? me.next = next

    Our handlers are going to be simple functions which take exception
    objects as arguments. ?A more advanced handler might filter exceptions
    based on their classes. ?That's not especially difficult to do badly,
    but it's fiddly to do well and it doesn't shed much light on the overall
    mechanism, so I'll omit that complication.

    We'll want a fluid for the handler list.

    ? ? ? ? HANDLERS = Fluid(None)

    Now we want to run a chunk of code with a handler attached. ?This seems
    like another good use for a context manager.

    ? ? ? ? class handler (object):
    ? ? ? ? ? def __init__(me, func):
    ? ? ? ? ? ? me._func = func
    ? ? ? ? ? def __enter__(me):
    ? ? ? ? ? ? me._bind = FluidBinding(HANDLERS,
    ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Link(me.func, HANDLERS.value)
    ? ? ? ? ? ? me._bind.__enter__()
    ? ? ? ? ? def __exit__(me, ty, val, tb):
    ? ? ? ? ? ? return me._bind.__exit__(ty, val, tb)

    (Context managers don't compose very nicely. ?It'd be prettier with the
    contextmanager decorator.)

    Let's say that we `signal' resumable exceptions rather than `raising'
    them. ?How do we do that?

    ? ? ? ? def signal(exc):
    ? ? ? ? ? with HANDLERS.binding():
    ? ? ? ? ? ? while HANDLERS.value:
    ? ? ? ? ? ? ? h = HANDLERS.value
    ? ? ? ? ? ? ? HANDLERS.value = h.next
    ? ? ? ? ? ? ? h.item(exc)

    Yes, if all of the handlers decline, we just return. ?This is Bad for
    errors, but good for other kinds of situations, so `signal' is a
    convenient substrate to build on.

    ? ? ? ? def error(exc):
    ? ? ? ? ? signal(exc)
    ? ? ? ? ? raise RuntimeError, 'unhandled resumable exception'

    ? ? ? ? def warning(exc):
    ? ? ? ? ? signal(exc)
    ? ? ? ? ? ## Crank up python's usual warning stuff

    Note also that handlers are invoked in a dynamic environment which
    doesn't include them or any handlers added since. ?Obviously they can
    install their own handlers just fine.

    Cool. ?Now how about recovery? ?This is where nonlocal transfer comes
    in. ?If a handler wants to take responsibility for the exception, it has
    to make a nonlocal transfer. ?Where should it go? ?Let's maintain a
    table of restart points. ?Again, it'll be a linked list.

    ? ? ? ? RESTARTS = Fluid(None)

    ? ? ? ? class restart (block):
    ? ? ? ? ? def __init__(me, name):
    ? ? ? ? ? ? me.name = name
    ? ? ? ? ? ? super(restart, me).__init__(me)
    ? ? ? ? ? def invoke(me, value = None):
    ? ? ? ? ? ? me._escape(value)
    ? ? ? ? ? def __enter__(me):
    ? ? ? ? ? ? me._bind = FluidBinding(RESTARTS, Link(me, RESTARTS.value))
    ? ? ? ? ? ? me._bind.__enter__()
    ? ? ? ? ? ? return super(restart, me).__enter__()
    ? ? ? ? ? def __exit__(me, ty, val, tb):
    ? ? ? ? ? ? ## Poor man's PROG1.
    ? ? ? ? ? ? try:
    ? ? ? ? ? ? ? return super(restart, me).__exit__(ty, val, tb)
    ? ? ? ? ? ? finally:
    ? ? ? ? ? ? ? me._bind.__exit__(ty, val, tb)

    ? ? ? ? def find_restart(name):
    ? ? ? ? ? r = RESTARTS.value
    ? ? ? ? ? while r:
    ? ? ? ? ? ? if r.item.name == name:
    ? ? ? ? ? ? ? return r.item
    ? ? ? ? ? ? r = r.next
    ? ? ? ? ? return None

    Using all of this is rather cumbersome, and Python doesn't allow
    syntactic abstraction so there isn't really much we can do to sweeten
    the pill. ?But I ought to provide an example of this machinery in
    action.

    ? ? ? ? def toy(x, y):
    ? ? ? ? ? r = restart('use-value')
    ? ? ? ? ? with r:
    ? ? ? ? ? ? if y == 0:
    ? ? ? ? ? ? ? error(ZeroDivisionError())
    ? ? ? ? ? ? r.result = x/y
    ? ? ? ? ? return r.result

    ? ? ? ? def example():
    ? ? ? ? ? def zd(exc):
    ? ? ? ? ? ? if not isinstance(exc, ZeroDivisionError):
    ? ? ? ? ? ? ? return
    ? ? ? ? ? ? r = find_restart('use-value')
    ? ? ? ? ? ? if not r: return
    ? ? ? ? ? ? r.invoke(42)
    ? ? ? ? ? print toy(5, 2)
    ? ? ? ? ? with handler(zd):
    ? ? ? ? ? ? print toy(1, 0)

    Does any of that help?
    You could do that.

    Or, you could just put your try...finally inside a loop.


    Carl Banks
  • Carl Banks at Dec 6, 2010 at 10:43 pm

    On Dec 6, 2:42?pm, Carl Banks wrote:
    Or, you could just put your try...finally inside a loop.

    er, try...except


    Carl Banks
  • Mark Wooding at Dec 6, 2010 at 11:03 pm

    Carl Banks <pavlovevidence at gmail.com> writes:
    On Dec 6, 12:58?pm, m... at distorted.org.uk (Mark Wooding) wrote:
    ? ? ? ? def toy(x, y):
    ? ? ? ? ? r = restart('use-value')
    ? ? ? ? ? with r:
    ? ? ? ? ? ? if y == 0:
    ? ? ? ? ? ? ? error(ZeroDivisionError())
    ? ? ? ? ? ? r.result = x/y
    ? ? ? ? ? return r.result

    ? ? ? ? def example():
    ? ? ? ? ? def zd(exc):
    ? ? ? ? ? ? if not isinstance(exc, ZeroDivisionError):
    ? ? ? ? ? ? ? return
    ? ? ? ? ? ? r = find_restart('use-value')
    ? ? ? ? ? ? if not r: return
    ? ? ? ? ? ? r.invoke(42)
    ? ? ? ? ? print toy(5, 2)
    ? ? ? ? ? with handler(zd):
    ? ? ? ? ? ? print toy(1, 0)
    You could do that.

    Or, you could just put your try...finally inside a loop.
    [You correct `finally' to `except' in a follow-up.]

    I think you've missed the point almost entirely.

    Any code called from within the `with handler' context will (unless
    overridden) cause a call `toy(x, 0)' to return 42. Even if the `with
    handler' block calls other functions and so on. Note also that the
    expression of this is dynamically further from where the error is
    signalled than the resume point (which is within the same function).
    You can't do this with `try' ... `except'. Which was, of course, the
    point.

    -- [mdw]
  • OKB (not okblacke) at Dec 8, 2010 at 8:09 pm

    Mark Wooding wrote:
    Any code called from within the `with handler' context will (unless
    overridden) cause a call `toy(x, 0)' to return 42. Even if the `with
    handler' block calls other functions and so on. Note also that the
    expression of this is dynamically further from where the error is
    signalled than the resume point (which is within the same function).
    You can't do this with `try' ... `except'. Which was, of course, the
    point.
    This is an interesting setup, but I'm not sure I see why you need
    it. If you know that, in a particular context, you want toy(x, 0) to
    result in 42 instead of ZeroDivisionError, why not just define

    safeToy(x, y):
    try:
    retVal = toy(x, y)
    except ZeroDivisionError:
    retVal = 42
    return retVal

    . . . and then call safeToy instead of toy in those contexts?

    --
    --OKB (not okblacke)
    Brendan Barnwell
    "Do not follow where the path may lead. Go, instead, where there is
    no path, and leave a trail."
    --author unknown

Related Discussions

People

Translate

site design / logo © 2022 Grokbase