FAQ
Howdy all,

I'm slowly developing PEP 3143 and, in parallel, its reference
implementation, the ?python-daemon? library
<URL:http://pypi.python.org/pypi/python-daemon/>. Feedback continues to
be welcome from people wanting to make a program become a daemon; please
try it out and critique the specification and/or the implementation.

Today a flaw has been uncovered by Pavol Babin??k. In diagnosing an
issue, we doscovered that PEP 3143 currently leads, in some common
circumstances, to exiting a context manager twice.

The current specification of the `DaemonContext` class says:

=====
close()
Return: None

Close the daemon context. This performs the following step:

* If the pidfile attribute is not None, exit its context
manager.
=====

and:

=====
__exit__(exc_type, exc_value, exc_traceback)
Return: True or False as defined by the context manager
protocol

Call the instance's close() method, then return True if the
exception was handled or False if it was not.
=====

but also:

=====
open()
Return: None

Open the daemon context, turning the current program into a daemon
process. This performs the following steps:
[?]

* Register the close method to be called during Python's exit
processing.
=====


This leads to the situation where the `DaemonContext.close` method can
be called twice:

pidfile = FunkyLockFile()
daemon_context = daemon.DaemonContext(pidfile=pidfile)
with daemon_context:
main_processing()

According to the current specification, this will cause
`daemon_context.close()` to be called on exiting its context manager (at
the end of the `with` suite) *and* on program exit (via `atexit`
processing).

This will cause the `pidfile` context manager to be exited twice in
succession, which surely can't be good and is at least undefined AFAICT.


How should a context manager be implemented to handle situations like
this? One possible way is to raise an exception when trying to close a
not-open context manager. Another is to return immediately if the
attempt is made, making it safe to try closing a context manager
multiple times.

The ?Register the close method to be called during Python's exit
processing? is there to ensure clean-up occurs even if the program
isn't using the DaemonContext as a context manager. Ideally, what I'd
like to do is *un*-register the `close` method after having already
closed it, so that exit processing *won't* call it. The `atexit` module,
though, doesn't appear to give me that option.

Ideas? How should this be addressed both Pythonically and respecting the
intent of PEP 3143?

--
\ ?For fast acting relief, try slowing down.? ?Jane Wagner, via |
`\ Lily Tomlin |
_o__) |
Ben Finney

Search Discussions

  • Carl Banks at May 17, 2009 at 3:01 am

    On May 16, 5:50?pm, Ben Finney wrote:
    Ideas? How should this be addressed both Pythonically and respecting the
    intent of PEP 3143?

    There's already precedent for what to do in the Python library.

    Python 2.5.2 (r252:60911, Jan 4 2009, 17:40:26)
    [GCC 4.3.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    f = open('somefile')
    f.close()
    f.close()
    f.close()
    f.close()

    Is a context manager really necessary here? It's not like daemon
    processes can reattach themselves to the terminal when they're done
    being a daemon. What's the use case for being able to partially clean
    up before program exit?


    Carl Banks
  • Ben Finney at May 17, 2009 at 3:20 am

    Carl Banks <pavlovevidence at gmail.com> writes:

    There's already precedent for what to do in the Python library.

    Python 2.5.2 (r252:60911, Jan 4 2009, 17:40:26)
    [GCC 4.3.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    f = open('somefile')
    f.close()
    f.close()
    f.close()
    f.close()
    Yes, that's a precedent in favour of ?silently return immediately when
    closing an already-closed DaemonContext?. I'm not about to assume that
    it's the *only* relevant precedent.
    Is a context manager really necessary here? It's not like daemon
    processes can reattach themselves to the terminal when they're done
    being a daemon. What's the use case for being able to partially clean
    up before program exit?
    The use case is::

    pidfile = MyChoiceOfLockFile()
    daemon_context = daemon.DaemonContext(pidfile=pidfile)

    with daemon_context:
    do_main_processing()

    which could easily become something like::

    beans_resource = crunchy_init()

    with beans_resource:

    spam_resource = crispy_init()

    pidfile = MyChoiceOfLockFile()
    daemon_context = daemon.DaemonContext(pidfile=pidfile)

    with daemon_context:
    do_main_processing()

    spam_resource.cleanup()

    In other words, the DaemonContext has a specific job, and is easier to
    use if it can be relied on to clean up its own stuff via a context
    manager; but the larger program could easily have other things (in the
    example above, both of `spam_resource` and `beans_resource`) it needs to
    clean up apart from just the daemon context.

    --
    \ ?We reserve the right to serve refuse to anyone. ? ?restaurant, |
    `\ Japan |
    _o__) |
    Ben Finney
  • Carl Banks at May 17, 2009 at 5:33 am

    On May 16, 8:20?pm, Ben Finney wrote:
    Carl Banks <pavlovevide... at gmail.com> writes:
    There's already precedent for what to do in the Python library.
    Python 2.5.2 (r252:60911, Jan ?4 2009, 17:40:26)
    [GCC 4.3.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    f = open('somefile')
    f.close()
    f.close()
    f.close()
    f.close()
    Yes, that's a precedent in favour of ?silently return immediately when
    closing an already-closed DaemonContext?. I'm not about to assume that
    it's the *only* relevant precedent.
    I don't think this is anything more than a trivial consideration,
    which should warrant nothing more than the simplest solution possible,
    which is to simply allow multiple clean up calls.

    That one of Python's most fundamental types allows its clean-up
    operation to be invoked safely on an already-cleaned up object is
    precedent enough, given the triviality of the issue.

    You're welcome to your own assessment of the problem's importance.


    Carl Banks
  • Ben Finney at May 17, 2009 at 5:46 am

    Carl Banks <pavlovevidence at gmail.com> writes:

    I don't think this is anything more than a trivial consideration,
    Okay, thank you.

    Anyone else?

    --
    \ ?All television is educational television. The question is: |
    `\ what is it teaching?? ?Nicholas Johnson |
    _o__) |
    Ben Finney
  • Gunter Henriksen at May 18, 2009 at 5:14 pm
    Anyone else?
    If there is a function which triggers a one-shot switch, I
    like to have a way to find out if it has already been triggered,
    I prefer to have the function tell me if it triggered the switch
    or not, but I would not want that to be by raising an exception.
  • MRAB at May 18, 2009 at 5:43 pm

    Gunter Henriksen wrote:
    Anyone else?
    If there is a function which triggers a one-shot switch, I
    like to have a way to find out if it has already been triggered,
    I prefer to have the function tell me if it triggered the switch
    or not, but I would not want that to be by raising an exception.
    The 'file' class also has a 'closed' method, so that's a good model to
    follow.
  • Ben Finney at May 18, 2009 at 9:39 pm

    MRAB <google at mrabarnett.plus.com> writes:

    Gunter Henriksen wrote:
    If there is a function which triggers a one-shot switch, I like to
    have a way to find out if it has already been triggered, I prefer to
    have the function tell me if it triggered the switch or not, but I
    would not want that to be by raising an exception.
    The 'file' class also has a 'closed' method, so that's a good model to
    follow.
    Yes, I find that convincing as well. (I might prefer it to instead
    answer the question ?is it currently open??, i.e. an ?opened? query,
    but that makes for more awkward English than ?closed?.)

    --
    \ ?Are you thinking what I'm thinking, Pinky?? ?Uh... yeah, |
    `\ Brain, but where are we going to find rubber pants our size?? |
    _o__) ?_Pinky and The Brain_ |
    Ben Finney
  • Duncan Booth at May 19, 2009 at 7:56 am

    Ben Finney wrote:

    MRAB <google at mrabarnett.plus.com> writes:
    Gunter Henriksen wrote:
    If there is a function which triggers a one-shot switch, I like to
    have a way to find out if it has already been triggered, I prefer
    to have the function tell me if it triggered the switch or not, but
    I would not want that to be by raising an exception.
    The 'file' class also has a 'closed' method, so that's a good model
    to follow.
    Yes, I find that convincing as well. (I might prefer it to instead
    answer the question ???is it currently open????, i.e. an ???opened???
    query, but that makes for more awkward English than ???closed???.)
    In the general case you might have an object which you create then open as
    an explicit step. Later you close it explicitly before it is discarded. In
    that case methods 'opened' and 'closed' both make sense but mean different
    things. For a file object, where construction always opens the file, an
    opened method (if it existed) should always return true so isn't useful.

    Or in other words, I take the word 'opened' as meaning 'has been opened'
    which is not the same as 'is currently open'.
  • MRAB at May 19, 2009 at 11:37 am

    Duncan Booth wrote:
    Ben Finney wrote:
    MRAB <google at mrabarnett.plus.com> writes:
    Gunter Henriksen wrote:
    If there is a function which triggers a one-shot switch, I like to
    have a way to find out if it has already been triggered, I prefer
    to have the function tell me if it triggered the switch or not, but
    I would not want that to be by raising an exception.
    The 'file' class also has a 'closed' method, so that's a good model
    to follow.
    Yes, I find that convincing as well. (I might prefer it to instead
    answer the question ???is it currently open????, i.e. an ???opened???
    query, but that makes for more awkward English than ???closed???.)
    In the general case you might have an object which you create then open as
    an explicit step. Later you close it explicitly before it is discarded. In
    that case methods 'opened' and 'closed' both make sense but mean different
    things. For a file object, where construction always opens the file, an
    opened method (if it existed) should always return true so isn't useful.

    Or in other words, I take the word 'opened' as meaning 'has been opened'
    which is not the same as 'is currently open'.
    Equally you could say that "closed" means "has been closed", which is
    not the same as "is currently closed". Unusual, but possible! :-)

    Borrowing from the string class, it would be "isopen" (like "isdigit",
    probably influenced by C), although these days we would write "is_open".
  • Ben Finney at May 19, 2009 at 8:27 am

    Gunter Henriksen <gunterhenriksen at gmail.com> writes:

    If there is a function which triggers a one-shot switch, I like to
    have a way to find out if it has already been triggered, I prefer to
    have the function tell me if it triggered the switch or not, but I
    would not want that to be by raising an exception.
    In this case, though, we're talking about a class intended to implement
    the context manager API (in addition to other API).

    So, following your argument above, we might expect a generic context
    manager object to be able to tell us whether we're currently inside that
    context or not. Does such a thing exist?

    --
    \ ?If consumers even know there's a DRM, what it is, and how it |
    `\ works, we've already failed.? ?Peter Lee, Disney corporation, |
    _o__) 2005 |
    Ben Finney
  • Gunter Henriksen at May 19, 2009 at 10:22 pm

    If there is a function which triggers a one-shot switch, I like
    to have a way to find out if it has already been triggered, I
    prefer to have the function tell me if it triggered the switch
    or not, but I would not want that to be by raising an exception.
    In this case, though, we're talking about a class intended to
    implement the context manager API (in addition to other API).

    So, following your argument above, we might expect a generic
    context manager object to be able to tell us whether we're
    currently inside that context or not. Does such a thing exist?
    By a "one-shot switch" I meant as something which
    changes once atomically in isolation, not as
    something whose state comprises a transaction during
    which the state of multiple externally visible
    objects might change.

    I imagine if there is an object used as a context
    manager whose context state ("currently inside
    that context or not") is not strictly under the
    control of the single "with" statement where it is
    fulfilling the "context manager" contract, would not
    be a generic context manager.

    In other words, to me it seems like the concept of
    "currently inside a context or not" is not something
    visible external to the object except in a case
    where there is another thread of execution, which is
    not a case I would expect to be part of the generic
    context manager protocol, but can be added by an
    object which implements the context manager protocol
    and also keeps track of its state by setting flags
    in the __enter__ and __exit__ functions.

    Does this make sense?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMay 17, '09 at 12:50a
activeMay 19, '09 at 10:22p
posts12
users5
websitepython.org

People

Translate

site design / logo © 2022 Grokbase