FAQ
I'm using the python mailbox class in a script that processes incoming
mail and delivers it to various mbox format mailboxes. It appears
that, although I am calling the lock method on the destination before
writing to the mbox and calling unlock afterwards the locking isn't
working correctly.

I am reading the mail with mutt, if I deliver the mail direct using
Postfix (i.e. bypassing my python script) mutt works perfectly and
shows new mail arriving in the mailbox as expected with an 'N' flag
beside it. If I deliver the mail with my python script I get an error
from mutt saying "Mailbox was externally modified. Flags may be
wrong.", or if I'm reading the mbox via NFS I get all sorts of strange
errors (though again it works perfectly if Postfix writes to the mbox).

So it seems that python's mailbox class locking isn't playing nicely
with mutt's mailbox locking whereas postfix's locking does work
correctly.

Has anyone seen this problem before, and/or do I need to anything more
than the following for the locking to work correctly:-

#
#
# set up the mb for adding the new message, will create if it doesn't exist
#
dest = mailbox.mbox(mbName, factory=None)

dest.lock()
dest.add(m) # add the new message
dest.flush()
dest.unlock()

Any help/comments most welcome

--
Chris Green

Search Discussions

  • Tim Roberts at Aug 10, 2010 at 6:28 am

    tinnews at isbd.co.uk wrote:
    I'm using the python mailbox class in a script that processes incoming
    mail and delivers it to various mbox format mailboxes. It appears
    that, although I am calling the lock method on the destination before
    writing to the mbox and calling unlock afterwards the locking isn't
    working correctly.
    ...
    So it seems that python's mailbox class locking isn't playing nicely
    with mutt's mailbox locking whereas postfix's locking does work
    correctly.
    Correct. The "dest.flush()" method creates a temporary file, copies the
    entire modified mailbox into it, removed the original file, and renames the
    temp file into place.

    The Postfix MDA, like most MDAs, just opens the existing file and appends
    the new data to it.
    Has anyone seen this problem before, and/or do I need to anything more
    than the following for the locking to work correctly:-
    It's not the locking. It's the flush mechanism. The mbox class doesn't
    know that the ONLY thing you did was an append. You might have modified
    other messages in the middle. If you want to do an append, you'll need to
    write your own subclass of mbox.
    --
    Tim Roberts, timr at probo.com
    Providenza & Boekelheide, Inc.
  • Tinnews at Aug 10, 2010 at 9:01 am

    Tim Roberts wrote:
    tinnews at isbd.co.uk wrote:
    I'm using the python mailbox class in a script that processes incoming
    mail and delivers it to various mbox format mailboxes. It appears
    that, although I am calling the lock method on the destination before
    writing to the mbox and calling unlock afterwards the locking isn't
    working correctly.
    ...
    So it seems that python's mailbox class locking isn't playing nicely
    with mutt's mailbox locking whereas postfix's locking does work
    correctly.
    Correct. The "dest.flush()" method creates a temporary file, copies the
    entire modified mailbox into it, removed the original file, and renames the
    temp file into place.
    Yes, I just took a look at the mailbox.py code and it does exactly
    that which of course screws up just about any normal MUA looking at
    the mbox. Grrrrrr!

    The Postfix MDA, like most MDAs, just opens the existing file and appends
    the new data to it.
    Has anyone seen this problem before, and/or do I need to anything more
    than the following for the locking to work correctly:-
    It's not the locking. It's the flush mechanism. The mbox class doesn't
    know that the ONLY thing you did was an append. You might have modified
    other messages in the middle. If you want to do an append, you'll need to
    write your own subclass of mbox.
    OK, thanks. In reality I can probably just use straightforward file
    reading and writing as the *only* thing I will ever be doing is to
    append a message to a mailbox file.

    I think there should be a big warning in the mailbox documentation to
    this effect as doing it the way that Python's mailbox class does it
    will break all sorts of things. There should maybe be a specific
    'append' method.

    --
    Chris Green
  • Chris Rebert at Aug 10, 2010 at 9:25 am

    On Tue, Aug 10, 2010 at 2:01 AM, wrote:
    Tim Roberts wrote:
    tinnews at isbd.co.uk wrote:
    I'm using the python mailbox class in a script that processes incoming
    mail and delivers it to various mbox format mailboxes. ?It appears
    that, although I am calling the lock method on the destination before
    writing to the mbox and calling unlock afterwards the locking isn't
    working correctly.
    ...
    So it seems that python's mailbox class locking isn't playing nicely
    with mutt's mailbox locking whereas postfix's locking does work
    correctly.
    Correct. ?The "dest.flush()" method creates a temporary file, copies the
    entire modified mailbox into it, removed the original file, and renames the
    temp file into place.
    Yes, I just took a look at the mailbox.py code and it does exactly
    that which of course screws up just about any normal MUA looking at
    the mbox. ?Grrrrrr!

    The Postfix MDA, like most MDAs, just opens the existing file and appends
    the new data to it.
    Has anyone seen this problem before, and/or do I need to anything more
    than the following for the locking to work correctly:-
    It's not the locking. ?It's the flush mechanism. ?The mbox class doesn't
    know that the ONLY thing you did was an append. ?You might have modified
    other messages in the middle. ?If you want to do an append, you'll need to
    write your own subclass of mbox.
    OK, thanks. ?In reality I can probably just use straightforward file
    reading and writing as the *only* thing I will ever be doing is to
    append a message to a mailbox file.

    I think there should be a big warning in the mailbox documentation to
    this effect as doing it the way that Python's mailbox class does it
    will break all sorts of things. ?There should maybe be a specific
    'append' method.
    File a documentation and/or library bug:
    http://bugs.python.org/

    Cheers,
    Chris
  • John Nagle at Aug 27, 2010 at 8:09 pm
    On 8/10/2010 2:25 AM, Chris Rebert wrote:
    On Tue, Aug 10, 2010 at 2:01 AM,wrote:
    Tim Robertswrote:
    tinnews at isbd.co.uk wrote:
    I'm using the python mailbox class in a script that processes incoming
    mail and delivers it to various mbox format mailboxes. It appears
    that, although I am calling the lock method on the destination before
    writing to the mbox and calling unlock afterwards the locking isn't
    working correctly.
    ...
    So it seems that python's mailbox class locking isn't playing nicely
    with mutt's mailbox locking whereas postfix's locking does work
    correctly.
    Correct. The "dest.flush()" method creates a temporary file, copies the
    entire modified mailbox into it, removed the original file, and renames the
    temp file into place.
    Yes, I just took a look at the mailbox.py code and it does exactly
    that which of course screws up just about any normal MUA looking at
    the mbox. Grrrrrr!

    The Postfix MDA, like most MDAs, just opens the existing file and appends
    the new data to it.
    Has anyone seen this problem before, and/or do I need to anything more
    than the following for the locking to work correctly:-
    It's not the locking. It's the flush mechanism. The mbox class doesn't
    know that the ONLY thing you did was an append. You might have modified
    other messages in the middle. If you want to do an append, you'll need to
    write your own subclass of mbox.
    OK, thanks. In reality I can probably just use straightforward file
    reading and writing as the *only* thing I will ever be doing is to
    append a message to a mailbox file.

    I think there should be a big warning in the mailbox documentation to
    this effect as doing it the way that Python's mailbox class does it
    will break all sorts of things. There should maybe be a specific
    'append' method.
    File a documentation and/or library bug:
    http://bugs.python.org/

    Cheers,
    Chris
  • Chris Jewell at Aug 27, 2010 at 6:22 pm
    Even if you replace the python mbox code with something that uses
    fcntl.flock() to protect against concurrent updating, you should also
    understand that NFS does *not* provide full Unix filesystem semantics.
    In particular, Unix flock(2) (which Python's fcntl.flock() wraps)
    doesn't work over NFS.

    That's why if you want to access mail over NFS, you should use maildir,
    rather than mbox, no matter what your programming language.

    --
    Chris Jewell chrisj at puffin.com PO Box 1396 Gualala CA USA 95445-1396

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedAug 9, '10 at 9:52p
activeAug 27, '10 at 8:09p
posts6
users5
websitepython.org

People

Translate

site design / logo © 2022 Grokbase