We are running Mailman 2.1.12 on Solaris 9 SPARC, Python 2.4.2. What causes the following "decoding Unicode is not supported" error message? I've looked at digest.mbox and didn't find any Unicode.

Thanks, Larry

-----Original Message-----
From: GNU Mailman [mailto:mailman at ornl71.ornl.gov]
Sent: Tuesday, September 01, 2009 12:00 PM
To: mailman at ornl71.ornl.gov
Subject: [Mailman Site List] Output from "cron" command

Your "cron" job on ornl71
/usr/local/bin/python -S /usr/local/mailman/cron/senddigests

produced the following output:

List: dcaplusplus: problem processing /usr/local/mailman/lists/dcaplusplus/digest.mbox:
decoding Unicode is not supported

Search Discussions

  • Mark Sapiro at Sep 2, 2009 at 12:04 am

    Rosenbaum, Larry M. wrote:
    We are running Mailman 2.1.12 on Solaris 9 SPARC, Python 2.4.2. What causes the following "decoding Unicode is not supported" error message? I've looked at digest.mbox and didn't find any Unicode. [...]
    Your "cron" job on ornl71
    /usr/local/bin/python -S /usr/local/mailman/cron/senddigests

    produced the following output:

    List: dcaplusplus: problem processing /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported

    I think this may be the "Mailman 2.1.12 is not compatible with python
    2.4.x (x>3)" issue. See the March 2009 note at the end of the FAQ at
    <http://wiki.list.org/x/pYA9>.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Rosenbaum, Larry M. at Sep 2, 2009 at 1:05 pm

    From: Mark Sapiro [mailto:mark at msapiro.net]

    Rosenbaum, Larry M. wrote:
    We are running Mailman 2.1.12 on Solaris 9 SPARC, Python 2.4.2. What
    causes the following "decoding Unicode is not supported" error message?
    I've looked at digest.mbox and didn't find any Unicode.
    [...]
    Your "cron" job on ornl71
    /usr/local/bin/python -S /usr/local/mailman/cron/senddigests

    produced the following output:

    List: dcaplusplus: problem processing
    /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported

    I think this may be the "Mailman 2.1.12 is not compatible with python
    2.4.x (x>3)" issue. See the March 2009 note at the end of the FAQ at
    <http://wiki.list.org/x/pYA9>.
    My original message was incorrect (I was looking at the wrong system). The system generating the error message is running python 2.5, which is stated as one of the recommended Python versions. What else can I check?

    Thanks, L
    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Mark Sapiro at Sep 2, 2009 at 2:24 pm

    Rosenbaum, Larry M. wrote:

    From: Mark Sapiro [mailto:mark at msapiro.net]

    Rosenbaum, Larry M. wrote:
    We are running Mailman 2.1.12 on Solaris 9 SPARC, Python 2.4.2. What
    causes the following "decoding Unicode is not supported" error message?
    I've looked at digest.mbox and didn't find any Unicode.
    [...]
    Your "cron" job on ornl71
    /usr/local/bin/python -S /usr/local/mailman/cron/senddigests

    produced the following output:

    List: dcaplusplus: problem processing
    /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported

    I think this may be the "Mailman 2.1.12 is not compatible with python
    2.4.x (x>3)" issue. See the March 2009 note at the end of the FAQ at
    <http://wiki.list.org/x/pYA9>.
    My original message was incorrect (I was looking at the wrong system). The system generating the error message is running python 2.5, which is stated as one of the recommended Python versions. What else can I check?

    cp cron/senddigests cron/senddigestsx

    Then edit cron/senddigestsx

    Where you see (without the linewrap)

    for listname in listnames:
    mlist = MailList.MailList(listname, lock=0)
    if mlist.digest_send_periodic:
    mlist.Lock()
    try:
    try:
    mlist.send_digest_now()
    mlist.Save()
    # We are unable to predict what exception may occur in
    digest
    # processing and we don't want to lose the other
    digests, so
    # we catch everything.
    except Exception, errmsg:
    print >> sys.stderr, \
    'List: %s: problem processing %s:\n%s' % \
    (listname,
    os.path.join(mlist.fullpath(), 'digest.mbox'),
    errmsg)
    finally:
    mlist.Unlock()


    Add the two lines
    import traceback
    traceback.print_exc()
    to make it

    for listname in listnames:
    mlist = MailList.MailList(listname, lock=0)
    if mlist.digest_send_periodic:
    mlist.Lock()
    try:
    try:
    mlist.send_digest_now()
    mlist.Save()
    # We are unable to predict what exception may occur in
    digest
    # processing and we don't want to lose the other
    digests, so
    # we catch everything.
    except Exception, errmsg:
    print >> sys.stderr, \
    'List: %s: problem processing %s:\n%s' % \
    (listname,
    os.path.join(mlist.fullpath(), 'digest.mbox'),
    errmsg)
    import traceback
    traceback.print_exc()
    finally:
    mlist.Unlock()

    make sure the added lines are indented exactly the same amount (20
    spaces) as the 'print' above and don't use tabs.

    Then run

    cron/senddigestsx -l dcaplusplus

    This should produce a traceback in addition to the above message. Post
    that.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Rosenbaum, Larry M. at Sep 2, 2009 at 3:12 pm

    From: Mark Sapiro [mailto:mark at msapiro.net]

    Rosenbaum, Larry M. wrote:
    From: Mark Sapiro [mailto:mark at msapiro.net]

    Rosenbaum, Larry M. wrote:
    We are running Mailman 2.1.12 on Solaris 9 SPARC, Python 2.4.2.
    What
    causes the following "decoding Unicode is not supported" error
    message?
    I've looked at digest.mbox and didn't find any Unicode.
    [...]
    Your "cron" job on ornl71
    /usr/local/bin/python -S /usr/local/mailman/cron/senddigests

    produced the following output:

    List: dcaplusplus: problem processing
    /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported

    I think this may be the "Mailman 2.1.12 is not compatible with
    python
    2.4.x (x>3)" issue. See the March 2009 note at the end of the FAQ at
    <http://wiki.list.org/x/pYA9>.
    My original message was incorrect (I was looking at the wrong system).
    The system generating the error message is running python 2.5, which is
    stated as one of the recommended Python versions. What else can I
    check?
    ...
    Add the two lines
    import traceback
    traceback.print_exc() ...
    cron/senddigestsx -l dcaplusplus

    This should produce a traceback in addition to the above message. Post
    that.
    Thank you for the excellent instructions. Here is the output:

    ornl71# cron/senddigestsx -l dcaplusplus
    List: dcaplusplus: problem processing /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported
    Traceback (most recent call last):
    File "cron/senddigestsx", line 89, in main
    mlist.send_digest_now()
    File "/usr/local/mailman/Mailman/Digester.py", line 60, in send_digest_now
    ToDigest.send_digests(self, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 339, in send_i18n_digests
    mcset = msg.get_content_charset('')
    File "/usr/local/lib/python2.5/email/message.py", line 759, in get_content_charset
    charset = unicode(charset, 'us-ascii').encode('us-ascii')
    TypeError: decoding Unicode is not supported
  • Mark Sapiro at Sep 3, 2009 at 12:55 am

    Rosenbaum, Larry M. wrote:
    Thank you for the excellent instructions. Here is the output:

    ornl71# cron/senddigestsx -l dcaplusplus
    List: dcaplusplus: problem processing /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported
    Traceback (most recent call last):
    File "cron/senddigestsx", line 89, in main
    mlist.send_digest_now()
    File "/usr/local/mailman/Mailman/Digester.py", line 60, in send_digest_now
    ToDigest.send_digests(self, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 339, in send_i18n_digests
    mcset = msg.get_content_charset('')
    File "/usr/local/lib/python2.5/email/message.py", line 759, in get_content_charset
    charset = unicode(charset, 'us-ascii').encode('us-ascii')
    TypeError: decoding Unicode is not supported

    Now I'm a bit confused. That line 759 in
    /usr/local/lib/python2.5/email/message.py doesn't correlate with any
    of the email/message.py modules I have. Instead of the line

    charset = unicode(charset, 'us-ascii').encode('us-ascii')

    all the email 4.0.x packages I have have

    try:
    if isinstance(charset, str):
    charset = unicode(charset, 'us-ascii')
    charset = charset.encode('us-ascii')
    except UnicodeError:
    return failobj

    This should work in your case. What do you get if you invoke python,
    import email and display email.__version__? e.g.,

    [msapiro at msapiro ~]$ python
    Python 2.5.2 (r252:60911, Dec 2 2008, 09:26:14)
    [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
    Type "help", "copyright", "credits" or "license" for more information.
    import email
    email.__version__
    '4.0.2'
    >>>

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Rosenbaum, Larry M. at Sep 3, 2009 at 2:35 pm

    From: Mark Sapiro [mailto:mark at msapiro.net]

    Rosenbaum, Larry M. wrote:
    Thank you for the excellent instructions. Here is the output:

    ornl71# cron/senddigestsx -l dcaplusplus
    List: dcaplusplus: problem processing
    /usr/local/mailman/lists/dcaplusplus/digest.mbox:
    decoding Unicode is not supported
    Traceback (most recent call last):
    File "cron/senddigestsx", line 89, in main
    mlist.send_digest_now()
    File "/usr/local/mailman/Mailman/Digester.py", line 60, in
    send_digest_now
    ToDigest.send_digests(self, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 142, in
    send_digests
    send_i18n_digests(mlist, mboxfp)
    File "/usr/local/mailman/Mailman/Handlers/ToDigest.py", line 339, in
    send_i18n_digests
    mcset = msg.get_content_charset('')
    File "/usr/local/lib/python2.5/email/message.py", line 759, in
    get_content_charset
    charset = unicode(charset, 'us-ascii').encode('us-ascii')
    TypeError: decoding Unicode is not supported

    Now I'm a bit confused. That line 759 in
    /usr/local/lib/python2.5/email/message.py doesn't correlate with any
    of the email/message.py modules I have. Instead of the line

    charset = unicode(charset, 'us-ascii').encode('us-ascii')

    all the email 4.0.x packages I have have

    try:
    if isinstance(charset, str):
    charset = unicode(charset, 'us-ascii')
    charset = charset.encode('us-ascii')
    except UnicodeError:
    return failobj

    This should work in your case. What do you get if you invoke python,
    import email and display email.__version__? e.g.,

    [msapiro at msapiro ~]$ python
    Python 2.5.2 (r252:60911, Dec 2 2008, 09:26:14)
    [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
    Type "help", "copyright", "credits" or "license" for more information.
    import email
    email.__version__
    '4.0.2'
    ornl71# python
    Python 2.5 (r25:51908, Sep 20 2006, 06:18:53)
    [GCC 3.4.6] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    import email
    email.__version__
    '4.0.1'
  • Mark Sapiro at Sep 3, 2009 at 3:37 pm

    Rosenbaum, Larry M. wrote:
    ornl71# python
    Python 2.5 (r25:51908, Sep 20 2006, 06:18:53)
    [GCC 3.4.6] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    import email
    email.__version__
    '4.0.1'
    I don't know if there was a different email 4.0.1 distributed with
    Python 2.5 as opposed to Python 2.5.1, or if yours is modified by Sun
    in some way (if it is a Sun package), but the problem is in your
    email/message.py get_content_charset method.

    All the email 4.0.x versions I have define this method as in the
    attached message.get_content_charset.txt file.

    In your case, the statement

    charset = unicode(charset, 'us-ascii').encode('us-ascii')

    is attempting to convert charset to unicode without first testing if it
    is already a unicode, which it is in the problem case.

    It appears there may be an additional incompatibility between Mailman
    2.1.12 and Python 2.5 as opposed to Python 2.5.x. I'll not this in the
    FAQ.

    If you can easily upgrade to a later Python 2.5.x, I think that will
    solve the problem. If not, you could patch
    /usr/local/lib/python2.5/email/message.py by replacing the definition
    of get_content_charset with that in the attached file.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan

    -------------- next part --------------
    An embedded and charset-unspecified text was scrubbed...
    Name: message.get_content_charset.txt
    URL: <http://mail.python.org/pipermail/mailman-users/attachments/20090903/88cc82b9/attachment.txt>
  • Rosenbaum, Larry M. at Sep 3, 2009 at 4:14 pm

    From: Mark Sapiro [mailto:mark at msapiro.net]
    Sent: Thursday, September 03, 2009 11:38 AM
    To: Rosenbaum, Larry M.; mailman-users at python.org
    Subject: RE: [Mailman-Users] What causes "decoding Unicode is not
    supported"?

    Rosenbaum, Larry M. wrote:
    ornl71# python
    Python 2.5 (r25:51908, Sep 20 2006, 06:18:53)
    [GCC 3.4.6] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    import email
    email.__version__
    '4.0.1'
    I don't know if there was a different email 4.0.1 distributed with
    Python 2.5 as opposed to Python 2.5.1, or if yours is modified by Sun
    in some way (if it is a Sun package), but the problem is in your
    email/message.py get_content_charset method.

    All the email 4.0.x versions I have define this method as in the
    attached message.get_content_charset.txt file.

    In your case, the statement

    charset = unicode(charset, 'us-ascii').encode('us-ascii')

    is attempting to convert charset to unicode without first testing if it
    is already a unicode, which it is in the problem case.

    It appears there may be an additional incompatibility between Mailman
    2.1.12 and Python 2.5 as opposed to Python 2.5.x. I'll not this in the
    FAQ.

    If you can easily upgrade to a later Python 2.5.x, I think that will
    solve the problem. If not, you could patch
    /usr/local/lib/python2.5/email/message.py by replacing the definition
    of get_content_charset with that in the attached file.
    Thanks. Upgrading Python to v2.5.1 fixed the problem.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedSep 1, '09 at 7:21p
activeSep 3, '09 at 4:14p
posts9
users2
websitelist.org

People

Translate

site design / logo © 2022 Grokbase