FAQ
We run 159 lists for an organisation on Mailman 2.1.5 (Debian Stable).
All other lists appear to be behaving fine, but one list seems to be
having issues. This list (and others) have run fine for several years,
and I don't think

I have done as much debugging as I know how, but I can't get this one
list to behave. I've checked for oversize files, the permissions of
the config directory entries, and even restored the list config.pck
from backups to a version that was working before it stopped. But it
seems the problem isn't any of these, as it persists in gobbling any
incoming mail on that list and putting it in the shunt file.

I'd appreciate any suggestions in how to usefully debug this list from
here - it's knocked out one of our organisation's more active
discussion systems.

Here's an example of what I see in the mailman error log for any
message to this list:

---

Aug 17 15:39:16 2007 (29254) Uncaught runner exception: unpack list of
wrong size
Aug 17 15:39:16 2007 (29254) Traceback (most recent call last):
File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
self._onefile(msg, msgdata)
File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
more = self._dopipeline(mlist, msg, msgdata, pipeline)
File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153,
in _dopipeline
sys.modules[modname].process(mlist, msg, msgdata)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
send_digests(mlist, mboxfp)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in
send_digests
send_i18n_digests(mlist, mboxfp)
File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in
send_i18n_digests
msg = scrubber(mlist, msg)
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 273, in process
url = save_attachment(mlist, part, dir)
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in
save_attachment
fnext = os.path.splitext(msg.get_filename(''))[1]
File "/usr/lib/python2.3/email/Message.py", line 725, in get_filename
filename = self.get_param('filename', missing, 'content-disposition')
File "/usr/lib/python2.3/email/Message.py", line 608, in get_param
for k, v in self._get_params_preserve(failobj, header):
File "/usr/lib/python2.3/email/Message.py", line 555, in _get_params_preserve
params = Utils.decode_params(params)
File "/usr/lib/python2.3/email/Utils.py", line 337, in decode_params
charset, language, value = decode_rfc2231(EMPTYSTRING.join(value))
File "/usr/lib/python2.3/email/Utils.py", line 284, in decode_rfc2231
charset, language, s = parts
ValueError: unpack list of wrong size

---

This is the resulting pickle file in qfiles/shunt for the same message:

---

[----- start pickle file -----]
<----- start object 1 ----->
From m.m at domain.com Fri Aug 17 15:39:14 2007
Received: from localhost ([127.0.0.1])
by domain.com with smtp (Exim 4.50 #1 (Debian))
id 1ILsfb-0007gx-Gn
for <cc at domain.com>; Fri, 17 Aug 2007 15:39:14 +1200
From: m.m at domain.com
To: cc at domain.com
X-SA-Exim-Connect-IP: 127.0.0.1
X-SA-Exim-Mail-From: m.m at domain.com
X-SA-Exim-Scanned: No (on domain.com); SAEximRunCond expanded to false
Subject: [CC] test message
X-BeenThere: cc at domain.com
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: Campaign Committee <cc at domain.com>
List-Id: Campaign Committee <cc.domain.com>
List-Unsubscribe: <http://domain.com/cgi-bin/mailman/listinfo/cc>,
<mailto:cc-request at domain.com?subject=unsubscribe>
List-Archive: <http://domain.com/cgi-bin/mailman/private/cc>
List-Post: <mailto:cc at domain.com>
List-Help: <mailto:cc-request at domain.com?subject=help>
List-Subscribe: <http://domain.com/cgi-bin/mailman/listinfo/cc>,
<mailto:cc-request at domain.com?subject=subscribe>

Hi, test here


<----- start object 2 ----->
{ '_parsemsg': False,
'lang': 'en',
'listname': 'cc',
'original_sender': 'm.m at domain.com',
'origsubj': 'test message',
'pipeline': [ 'ToDigest',
'ToArchive',
'ToUsenet',
'AfterDelivery',
'Acknowledge',
'ToOutgoing'],
'received_time': 1187321954.8111341,
'recips': [ 'example at domain.com' ],
'tolist': 1,
'version': 3,
'whichq': '/var/lib/mailman/qfiles/in'}
[----- end pickle file -----]

Search Discussions

  • Mark Sapiro at Aug 19, 2007 at 1:51 am

    chris burgess wrote:
    Here's an example of what I see in the mailman error log for any
    message to this list:

    ---

    Aug 17 15:39:16 2007 (29254) Uncaught runner exception: unpack list of
    wrong size
    Aug 17 15:39:16 2007 (29254) Traceback (most recent call last):
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
    self._onefile(msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153,
    in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
    send_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in
    send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in
    send_i18n_digests
    msg = scrubber(mlist, msg)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 273, in process
    url = save_attachment(mlist, part, dir)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in
    save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
    File "/usr/lib/python2.3/email/Message.py", line 725, in get_filename
    filename = self.get_param('filename', missing, 'content-disposition')
    File "/usr/lib/python2.3/email/Message.py", line 608, in get_param
    for k, v in self._get_params_preserve(failobj, header):
    File "/usr/lib/python2.3/email/Message.py", line 555, in _get_params_preserve
    params = Utils.decode_params(params)
    File "/usr/lib/python2.3/email/Utils.py", line 337, in decode_params
    charset, language, value = decode_rfc2231(EMPTYSTRING.join(value))
    File "/usr/lib/python2.3/email/Utils.py", line 284, in decode_rfc2231
    charset, language, s = parts
    ValueError: unpack list of wrong size

    ---

    This is the resulting pickle file in qfiles/shunt for the same message:

    The shunted message is not the one causing the problem. The problem is
    one of the older messages in the lists/<listname>/digest.mbox file.

    Every post to the list attempts to produce a digest based on size and
    scrubber encounters this error (which is most likely due to a known and
    recently fixed bug in RFC 2231 header decoding).

    See
    <http://mail.python.org/pipermail/mailman-users/2007-July/057768.html>
    for additional info on what to look for in the digest.mbox to fix this.

    - --
    Mark Sapiro <msapiro at value.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Chris burgess at Aug 19, 2007 at 3:03 am
    Thanks, Mark.

    It was as you suggested - a message with an attachment containing an
    apostrophe in the filename. This had also knocked out a list a year
    ago, and the cause was identical (but we didn't find the solution for
    that list until now).

    Is this bug #1518281 on SF, or if not, which release of Mailman fixes
    this? I'd like to be able to get my Mailman install (currently Debian
    stable's 2.1.5-8sarge5) fixed for that specific bug - it's the only
    thing which has knocked service out on our listserver in several years
    :)

    At least I'll know what to look for next time - much appreciated, Mark!
    On 8/19/07, Mark Sapiro wrote:
    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    chris burgess wrote:
    Here's an example of what I see in the mailman error log for any
    message to this list:

    ---

    Aug 17 15:39:16 2007 (29254) Uncaught runner exception: unpack list of
    wrong size
    Aug 17 15:39:16 2007 (29254) Traceback (most recent call last):
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
    self._onefile(msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
    File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153,
    in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 91, in process
    send_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 132, in
    send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 306, in
    send_i18n_digests
    msg = scrubber(mlist, msg)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 273, in process
    url = save_attachment(mlist, part, dir)
    File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 373, in
    save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
    File "/usr/lib/python2.3/email/Message.py", line 725, in get_filename
    filename = self.get_param('filename', missing, 'content-disposition')
    File "/usr/lib/python2.3/email/Message.py", line 608, in get_param
    for k, v in self._get_params_preserve(failobj, header):
    File "/usr/lib/python2.3/email/Message.py", line 555, in _get_params_preserve
    params = Utils.decode_params(params)
    File "/usr/lib/python2.3/email/Utils.py", line 337, in decode_params
    charset, language, value = decode_rfc2231(EMPTYSTRING.join(value))
    File "/usr/lib/python2.3/email/Utils.py", line 284, in decode_rfc2231
    charset, language, s = parts
    ValueError: unpack list of wrong size

    ---

    This is the resulting pickle file in qfiles/shunt for the same message:

    The shunted message is not the one causing the problem. The problem is
    one of the older messages in the lists/<listname>/digest.mbox file.

    Every post to the list attempts to produce a digest based on size and
    scrubber encounters this error (which is most likely due to a known and
    recently fixed bug in RFC 2231 header decoding).

    See
    <http://mail.python.org/pipermail/mailman-users/2007-July/057768.html>
    for additional info on what to look for in the digest.mbox to fix this.

    - --
    Mark Sapiro <msapiro at value.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (MingW32)

    iD8DBQFGx6I0VVuXXpU7hpMRAu7oAKDH3P0pn4ZhUxlgO/3TL14uwGZSZACfRt9T
    h8+azlACjBJ7tjclze/zIJ8=
    =1z+H
    -----END PGP SIGNATURE-----

    --
    -- chris burgess, giant robot ltd
    -- emailing you from gmail.com
    --
  • Mark Sapiro at Aug 19, 2007 at 5:30 am

    chris burgess wrote:
    It was as you suggested - a message with an attachment containing an
    apostrophe in the filename. This had also knocked out a list a year
    ago, and the cause was identical (but we didn't find the solution for
    that list until now).

    Is this bug #1518281 on SF, or if not, which release of Mailman fixes
    this? I'd like to be able to get my Mailman install (currently Debian
    stable's 2.1.5-8sarge5) fixed for that specific bug - it's the only
    thing which has knocked service out on our listserver in several years
    :)

    Yes, it is bug #1518281, but this bug is not completely/correctly fixed
    until Mailman 2.1.9 and it's associated email 2.5.8 library.

    Mailman 2.1.7 and up contain a fix to prevent this type of error,
    regardless of cause, from blocking regular delivery, although errors
    can still block digest delivery.

    --
    Mark Sapiro <msapiro at value.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedAug 17, '07 at 4:08a
activeAug 19, '07 at 5:30a
posts4
users2
websitelist.org

2 users in discussion

Mark Sapiro: 2 posts Chris burgess: 2 posts

People

Translate

site design / logo © 2022 Grokbase