FAQ
Hello list,

A week or so ago I ran across this problem on one of our mailing lists: A user
had sent an email with an apparently bad subject line. This has caused
mailman to stop delivering emails for that specific mailning list. Here's the
error from the logfile:

---

Jan 05 21:25:10 2009 (23055) Uncaught runner exception: unknown encoding: Re:
DDS 26073 : Imap - installation d
Jan 05 21:25:10 2009 (23055) Traceback (most recent call last):
File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
self._onefile(msg, msgdata)
File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
_dispose
more = self._dopipeline(mlist, msg, msgdata, pipeline)
File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
_dopipeline
sys.modules[modname].process(mlist, msg, msgdata)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
send_digests(mlist, mboxfp)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in
send_digests
send_i18n_digests(mlist, mboxfp)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in
send_i18n_digests
msg = scrubber(mlist, msg)
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 240, in process
url = save_attachment(mlist, part, dir)
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in
save_attachment
fnext = os.path.splitext(msg.get_filename(''))[1]
File "/usr/lib/python2.3/email/Message.py", line 731, in get_filename
return unicode(newvalue[2], newvalue[0] or 'us-ascii')
LookupError: unknown encoding: Re: DDS 26073 : Imap - installation d

Jan 05 21:25:10 2009 (23055) SHUNTING:
1231190705.198385+f75914e7ce7a9daf6e70914e0ed49923a47d2dd1


---

The full subject line should be:

RE: DDS 26073 : Imap - installation d'application 'vacation'

so I think the apostrophe is causing havoc here...

I have searched mailing list and the search engines, and it looks like this
error was a problem for people but has been solved in subsequent versions of
mailman (we are running 2.1.5 on Debian Sarge). What I haven't been able to
find out is how to purge out the offending email, and continue life as normal
until we upgrade to Etch!

I should say that I have tried removing the offending file from
$MAILMANDIR/qfiles/shunt and running "unshunt", but nothing changed.

Any suggestions are most appreciated.
/Khosrow
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20090105/7fccf84c/attachment.pgp>

Search Discussions

  • Mark Sapiro at Jan 6, 2009 at 4:36 pm

    Khosrow Ebrahimpour wrote:
    A week or so ago I ran across this problem on one of our mailing lists: A user
    had sent an email with an apparently bad subject line. This has caused
    mailman to stop delivering emails for that specific mailning list. Here's the
    error from the logfile:

    ---

    Jan 05 21:25:10 2009 (23055) Uncaught runner exception: unknown encoding: Re:
    DDS 26073 : Imap - installation d
    Jan 05 21:25:10 2009 (23055) Traceback (most recent call last):
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
    self._onefile(msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
    _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
    _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
    send_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in
    send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in
    send_i18n_digests
    msg = scrubber(mlist, msg)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 240, in process
    url = save_attachment(mlist, part, dir)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in
    save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
    File "/usr/lib/python2.3/email/Message.py", line 731, in get_filename
    return unicode(newvalue[2], newvalue[0] or 'us-ascii')
    LookupError: unknown encoding: Re: DDS 26073 : Imap - installation d

    Jan 05 21:25:10 2009 (23055) SHUNTING:
    1231190705.198385+f75914e7ce7a9daf6e70914e0ed49923a47d2dd1

    The message that causes this problem is not the message that is shunted.
    All the shunted messages are (presumably) valid messages that arrived
    after the issue started.

    The problem is that a message was posted to the list and probably
    delivered normally and stored in the digest.mbox file for the list.
    Then, eventually, the digest.mbox grew big enough that every subsequent
    message triggers a digest delivery on size, but the bad message in
    digest.mbox throws the above exception in the ToDigest handler, and the
    digest is not produced and the current message is shunted.

    The full subject line should be:

    RE: DDS 26073 : Imap - installation d'application 'vacation'

    so I think the apostrophe is causing havoc here...

    Yes, it is an apostrophe that is the problem, but it is in a 'name' or
    'filename' parameter of a Content-Type: or Content-Disposition: header.

    I have searched mailing list and the search engines, and it looks like this
    error was a problem for people but has been solved in subsequent versions of
    mailman (we are running 2.1.5 on Debian Sarge). What I haven't been able to
    find out is how to purge out the offending email, and continue life as normal
    until we upgrade to Etch!

    I should say that I have tried removing the offending file from
    $MAILMANDIR/qfiles/shunt and running "unshunt", but nothing changed.

    You don't want to remove the shunted files as they are not the problem,
    and you can't successfully unshunt until you fix the digest.mbox.

    You need to edit the lists/LISTNAME/digest.mbox. You can edit it by hand
    or with an MUA like mutt for example.

    The offending message has an RFC 2231 enncoded 'name' or 'filename'
    parameter with an apostrophe in the name. It will look something like

    Content-Disposition: attachment;
    filename*0*='charset'lang'text....................................
    filename*1=more.............................

    And text will have an apostrophe.

    You can simply remove those parameters (filename and/or name) or delete
    the entire message.

    You can also just delete the lists/LISTNAME/digest.mbox file, but then
    you'd lose those messages from the digest.

    After fixing the digest.mbox, you can run bin/unshunt to finish
    processing the shunted messages.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Khosrow Ebrahimpour at Jan 8, 2009 at 10:23 pm

    On 6 January 2009 11:36, Mark Sapiro wrote:

    You need to edit the lists/LISTNAME/digest.mbox. You can edit it by hand
    or with an MUA like mutt for example.
    Thank you Mark. That solution worked perfectly. I modified the digest.mbox and
    changed only the offending attachment content-types. After running "unshunt"
    everything behaved nicely.

    A more general question. Has this issue been fixed in subsequent releases of
    mailman? We are running 2.1.5 on Sarge at the moment. I suspect that we will
    be seeing this error occasionally, although I'm surprised it hasn't been an
    issue so far.

    Cheers,
    Khosrow
  • Mark Sapiro at Jan 8, 2009 at 11:30 pm

    Khosrow Ebrahimpour wrote:
    A more general question. Has this issue been fixed in subsequent releases of
    mailman? We are running 2.1.5 on Sarge at the moment. I suspect that we will
    be seeing this error occasionally, although I'm surprised it hasn't been an
    issue so far.

    Yes. The problem of unanticipated exceptions in digest processing
    stopping a list was fixed in 2.1.7.

    The problem of apostrophes in RFC 2231 encoded parameters was fixed in
    the underlying Python email library. I don't remember exactly when,
    but I'm pretty sure it was fixed in email 2.5.8 which has shipped with
    Mailman starting with 2.1.9.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedJan 5, '09 at 10:12p
activeJan 8, '09 at 11:30p
posts4
users2
websitelist.org

People

Translate

site design / logo © 2022 Grokbase