[Mailman-Users] [Fwd: archive emails are garbled]

Mark Sapiro

at Sep 16, 2006 at 4:05 am

Justin Zygmont wrote:
Is one of the other lists a better place to ask this?

Probably not. Did you see the reply to your original post at
<http://mail.python.org/pipermail/mailman-users/2006-September/053291.html>?

--
Mark Sapiro <msapiro at value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan

reply | permalink

Justin Zygmont

at Sep 19, 2006 at 12:37 am

⇧

Is one of the other lists a better place to ask this?

Probably not. Did you see the reply to your original post at
<http://mail.python.org/pipermail/mailman-users/2006-September/053291.html>?

I must have missed your message somehow, thanks a lot for the reply.

Justin Zygmont wrote:

I have a noticed a problem with emails stored in the list archives if
they have been uuencoded. When HTML emails are distributed to all the
list members they just show the raw HTML code,
Messages as sent to message subscribers or messages in the archives?
You should be able to send HTML to individual subscribers if you don't
filter content or if you allow the appropriate MIME types in content
filtering. How such messages appear in archives depends on sitewide
archiving options set in mm_cfg.py.
so I used uuencode and it
fixed that, but when I take a look at the message in the archives it
looks mostly garbled. Here's an example: >
begin 644 AgentCommissions.html
M/"%$3T-465!%($A434P at 4%5"3$E#("(M+R]7,T,O+T141"!(5$U,(#0N,#$@
M5')A;G-I=&EO;F%L+R]%3B(^#0H\:'1M;#X-"CQH96%D/@T*/'1I=&QE/D-I
M='EF;VYE($-O;6UI(2TM
M#0IT9"YL:71T;&5T97AT("![(&9O;G0M9F%M:6QY.B!!<FEA;#L at 9F]N="US
M:7IE.B X<'0@?0T*:#$@(" @(" @(" @(" @>R!F;VYT+69A;6EL>3H at 07)I
M86P[(&9O;G0M<VEZ93H@,31P="!]#0IH,B @(" @(" @(" @("![(&9O;G0M
M9F%M:6QY.B!!<FEA;#L at 9F]N="US:7IE.B Q,G!T('T-"G1D+F)I9W1E>'0@
M(" @('L at 9F]N="UF86UI;'DZ($%R:6%L.R!F;VYT+7-I>F4Z(#$T<'0@?0T*
This is not garbled. It is the uuencoded AgentCommissions.html file. A
uuencoded file is not human readable, but as far as mail agents are
concerned, it is plain text and will be archived in its raw form. Some
MUAs see a uuencoded file in an email and call it an 'attachment' and
render it as such, but this is non-standard. Also some MUAs when
composing will put uuencoded data in a separate MIME part with
Content-Transfer-Encoding: uuencode, x-uuencode, uue or x-uue, but
this to is non-standard.
Anyone know if there is a better way to deal with this?
You should be able to send HTML messages to your list members.
What Mailman version is this? What are your content filtering settings?
Can you provide a sample of a message as sent to Mailman and as
received from Mailman?

The messages appear as HTML code, and do not display an HTML looking
email when it arrives to the subscribers. This was one of the reasons
why I tried using uuencode, but in the list archives, they show in their
uuencoded form which is useless. I wanted it to appear as HTML in
thunderbird, and include it as an attachment also.

I'm using mailman-2.1.5.1-34.rhel4.3 and tried different content
filering options, it didn't seem to help though. Here is a sample of
part of an email:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Mobility Points Report</title>
<style>

</style>
</head>
<body>

Thanks,

reply | permalink

Mark Sapiro

at Sep 19, 2006 at 1:24 am

⇧

Justin Zygmont wrote:
The messages appear as HTML code, and do not display an HTML looking
email when it arrives to the subscribers.

This indicates that the MIME structure of the message has somehow been
corrupted or you are looking at a 'plain' format digest as opposed to
a 'mime' digest or an individual message from the list.

I'm using mailman-2.1.5.1-34.rhel4.3 and tried different content
filering options, it didn't seem to help though. Here is a sample of
part of an email:

The sample doesn't help. What we need to see is the headers of the
message and it's subparts. Ideally, both what you send to the list and
what you get back.

Send a very short, HTML message to the list with a Bcc: to yourself.
Then view the message source of both the Bcc: and the message from the
list. If you want help, you can copy and paste these message sources
into a post to this list.

--
Mark Sapiro <msapiro at value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan

reply | permalink

Justin Zygmont

at Sep 19, 2006 at 10:28 pm

⇧

Mark Sapiro wrote:
Justin Zygmont wrote:

The messages appear as HTML code, and do not display an HTML looking
email when it arrives to the subscribers.

This indicates that the MIME structure of the message has somehow been
corrupted or you are looking at a 'plain' format digest as opposed to
a 'mime' digest or an individual message from the list.

I'm using mailman-2.1.5.1-34.rhel4.3 and tried different content
filering options, it didn't seem to help though. Here is a sample of
part of an email:

The sample doesn't help. What we need to see is the headers of the
message and it's subparts. Ideally, both what you send to the list and
what you get back.

Send a very short, HTML message to the list with a Bcc: to yourself.
Then view the message source of both the Bcc: and the message from the
list. If you want help, you can copy and paste these message sources
into a post to this list.

ok, thanks. Here is a paste of the message sources. These both don't
show up as HTML in thunderbird for some reason, I guess mailman may not
be the cause? Messages below were each shortened to save size.

Emailed directly to me:

From - Tue Sep 19 14:51:50 2006
X-Account-Key: account2
X-UIDL: 1154544464.3663
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <jbill at cityfone.net>
X-Original-To: justin at cityfone.net
Delivered-To: justin at cityfone.net
Received: from citybillingcityfone.net (citybilling.cityfone.local
[192.168.43.30])
by citysupport.cityfone.local (Postfix) with ESMTP id A8CA12543C1
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:05 -0700 (PDT)
Received: from citybilling.cityfone.local (localhost [127.0.0.1])
by citybillingcityfone.net (8.13.4+Sun/8.13.3) with ESMTP id k8JLovlg007676
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Received: (from jan at localhost)
by citybilling.cityfone.local (8.13.4+Sun/8.13.3/Submit) id k8JLovKo007675
for justin at cityfone.net; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Date: Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
From: jbill at cityfone.net
Message-Id: <200609192150.k8JLovKo007675 at citybilling.cityfone.local>
X-Authentication-Warning: citybilling.cityfone.local: jan set sender to
jbill at cityfone.net using -r
To: justin at cityfone.net
Subject: test
X-IMAPbase: 1154544464 3663
Status: O
X-UID: 3663
Content-Length: 3344
X-Keywords:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>MoreMobility Points Report</title>
<style>

</style>
</head>
<body>
<img src="http://citynet/public/CityNetReportsLogo.gif" /><hr>
<h1>Connection Points</h1>
<hr>
<h2>These are clawbacks for phones returned in 30 days.</h2>
<table border="1" width="100%">
<tr>
<td class="littletext">Line</td>
<td class="littletext">Handset Code</td>
<td class="littletext">Activation Date</td>
<td class="littletext">Dealer</td>
<td class="littletext">Points</td>
</tr>
<tr>
<td colspan="4" class="littletext">Total</td>
<td class="littletext" align="right"> 0</td>
</tr>
</table>
<br><br><hr><table width="100%"><tr><td align="left"
class="littletext">© 2004-2006 Test report. All Rights
Reserved.</td><td align="right" class="littletext">Report generated on
September 14, 2006 00:00.</td></tr></table>
</body>
</html>

Same message, sent to me through mailman:

From - Tue Sep 19 14:56:50 2006
X-Account-Key: account2
X-UIDL: 1154544464.3664
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <reportgroup-bounces at cityfone.net>
X-Original-To: justin at cityfone.net
Delivered-To: justin at cityfone.net
Received: from citysupport.cityfone.local (localhost.localdomain
[127.0.0.1])
by citysupport.cityfone.local (Postfix) with ESMTP id 995942543C1;
Tue, 19 Sep 2006 14:55:06 -0700 (PDT)
X-Original-To: reportgroup at cityfone.net
Delivered-To: reportgroup at cityfone.net
Received: from citybillingcityfone.net (citybilling.cityfone.local
[192.168.43.30])
by citysupport.cityfone.local (Postfix) with ESMTP id B424F2543C1
for <reportgroup at cityfone.net>; Tue, 19 Sep 2006 14:55:04 -0700 (PDT)
Received: from citybilling.cityfone.local (localhost [127.0.0.1])
by citybillingcityfone.net (8.13.4+Sun/8.13.3) with ESMTP id
k8JLtuVL007686
for <reportgroup at cityfone.net>; Tue, 19 Sep 2006 14:55:56 -0700 (PDT)
Received: (from jan at localhost)
by citybilling.cityfone.local (8.13.4+Sun/8.13.3/Submit) id
k8JLtu47007685
for reportgroup at cityfone.net; Tue, 19 Sep 2006 14:55:56 -0700 (PDT)
Date: Tue, 19 Sep 2006 14:55:56 -0700 (PDT)
From: jbill at cityfone.net
Message-Id: <200609192155.k8JLtu47007685 at citybilling.cityfone.local>
X-Authentication-Warning: citybilling.cityfone.local: jan set sender to
jbill at cityfone.net using -r
To: reportgroup at cityfone.net
Subject: [Reportgroup] test
X-BeenThere: reportgroup at cityfone.net
X-Mailman-Version: 2.1.5
Precedence: list
Sender: reportgroup-bounces at cityfone.net
Errors-To: reportgroup-bounces at cityfone.net
X-IMAPbase: 1154544464 3664
Status: O
X-UID: 3664
Content-Length: 3475
X-Keywords:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>MoreMobility Points Report</title>
<style>

</style>
</head>
<body>
<img src="http://citynet/public/CityNetReportsLogo.gif" /><hr>
<h1>Connection Points</h1>
<hr>
<h2>These are clawbacks for phones returned in 30 days.</h2>
<table border="1" width="100%">
<tr>
<td class="littletext">Line</td>
<td class="littletext">Handset Code</td>
<td class="littletext">Activation Date</td>
<td class="littletext">Dealer</td>
<td class="littletext">Points</td>
</tr>
<tr>
<td colspan="4" class="littletext">Total</td>
<td class="littletext" align="right"> 0</td>
</tr>
</table>
<br><br><hr><table width="100%"><tr><td align="left"
class="littletext">© 2004-2006 test report. All Rights
Reserved.</td><td align="right" class="littletext">Report generated on
September 14, 2006 00:00.</td></tr></table>
</body>
</html>
_______________________________________________
Reportgroup mailing list
Reportgroup at cityfone.net
http://citysupport/mailman/admin

reply | permalink

Dragon

at Sep 19, 2006 at 11:08 pm

⇧

Justin Zygmont sent the message below at 15:28 9/19/2006:

ok, thanks. Here is a paste of the message sources. These both don't
show up as HTML in thunderbird for some reason, I guess mailman may not
be the cause? Messages below were each shortened to save size.

Emailed directly to me:

From - Tue Sep 19 14:51:50 2006
X-Account-Key: account2
X-UIDL: 1154544464.3663
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <jbill at cityfone.net>
X-Original-To: justin at cityfone.net
Delivered-To: justin at cityfone.net
Received: from citybillingcityfone.net (citybilling.cityfone.local
[192.168.43.30])
by citysupport.cityfone.local (Postfix) with ESMTP id A8CA12543C1
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:05 -0700 (PDT)
Received: from citybilling.cityfone.local (localhost [127.0.0.1])
by citybillingcityfone.net (8.13.4+Sun/8.13.3) with ESMTP
id k8JLovlg007676
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Received: (from jan at localhost)
by citybilling.cityfone.local (8.13.4+Sun/8.13.3/Submit) id
k8JLovKo007675
for justin at cityfone.net; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Date: Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
From: jbill at cityfone.net
Message-Id: <200609192150.k8JLovKo007675 at citybilling.cityfone.local>
X-Authentication-Warning: citybilling.cityfone.local: jan set sender to
jbill at cityfone.net using -r
To: justin at cityfone.net
Subject: test
X-IMAPbase: 1154544464 3663
Status: O
X-UID: 3663
Content-Length: 3344
X-Keywords:

---------------- End original message. ---------------------

The most immediate problem I see is that these messages are missing
any sort of MIME headers (as defined in RFC2045). Without those
headers, there is no definitive way for the mail client to know what
the content is. Some MUAs will then default to plain-text and not try
to even guess what the content type is. Some (like Eudora which I
use) may detect the HTML structure of the message body and render it.

I am curious how you are creating these messages, are you composing
them in an MUA in HTML format or are you cutting and pasting an HTML
document from another editor into the message body? If the latter,
you are not going to get the correct headers to identify the message
content to a MIME compliant MUA. If the former, your MUA should be
inserting these headers correctly.

As stated in section 1 of RFC2046:

The first document in this set,
<http://www.faqs.org/rfcs//rfcs/rfc2045.html>RFC 2045, defines a
number of header
fields, including Content-Type. The Content-Type field is used to
specify the nature of the data in the body of a MIME entity, by
giving media type and subtype identifiers, and by providing auxiliary
information that may be required for certain media types. After the
type and subtype names, the remainder of the header field is simply a
set of parameters, specified in an attribute/value notation. The
ordering of parameters is not significant.

In general, the top-level media type is used to declare the general
type of data, while the subtype specifies a specific format for that
type of data. Thus, a media type of "image/xyz" is enough to tell a
user agent that the data is an image, even if the user agent has no
knowledge of the specific image format "xyz". Such information can
be used, for example, to decide whether or not to show a user the raw
data from an unrecognized subtype -- such an action might be
reasonable for unrecognized subtypes of "text", but not for
unrecognized subtypes of "image" or "audio". For this reason,
registered subtypes of "text", "image", "audio", and "video" should
not contain embedded information that is really of a different type.
Such compound formats should be represented using the "multipart" or
"application" types.

Dragon

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Venimus, Saltavimus, Bibimus (et naribus canium capti sumus)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

reply | permalink

Justin Zygmont

at Sep 21, 2006 at 12:21 am

⇧

Dragon wrote:
Justin Zygmont sent the message below at 15:28 9/19/2006:

ok, thanks. Here is a paste of the message sources. These both don't
show up as HTML in thunderbird for some reason, I guess mailman may not
be the cause? Messages below were each shortened to save size.

Emailed directly to me:

From - Tue Sep 19 14:51:50 2006
X-Account-Key: account2
X-UIDL: 1154544464.3663
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <jbill at cityfone.net>
X-Original-To: justin at cityfone.net
Delivered-To: justin at cityfone.net
Received: from citybillingcityfone.net (citybilling.cityfone.local
[192.168.43.30])
by citysupport.cityfone.local (Postfix) with ESMTP id A8CA12543C1
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:05 -0700 (PDT)
Received: from citybilling.cityfone.local (localhost [127.0.0.1])
by citybillingcityfone.net (8.13.4+Sun/8.13.3) with ESMTP id
k8JLovlg007676
for <justin at cityfone.net>; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Received: (from jan at localhost)
by citybilling.cityfone.local (8.13.4+Sun/8.13.3/Submit) id
k8JLovKo007675
for justin at cityfone.net; Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
Date: Tue, 19 Sep 2006 14:50:57 -0700 (PDT)
From: jbill at cityfone.net
Message-Id: <200609192150.k8JLovKo007675 at citybilling.cityfone.local>
X-Authentication-Warning: citybilling.cityfone.local: jan set sender to
jbill at cityfone.net using -r
To: justin at cityfone.net
Subject: test
X-IMAPbase: 1154544464 3663
Status: O
X-UID: 3663
Content-Length: 3344
X-Keywords:

---------------- End original message. ---------------------

The most immediate problem I see is that these messages are missing any
sort of MIME headers (as defined in RFC2045). Without those headers,
there is no definitive way for the mail client to know what the content
is. Some MUAs will then default to plain-text and not try to even guess
what the content type is. Some (like Eudora which I use) may detect the
HTML structure of the message body and render it.

I am curious how you are creating these messages, are you composing them
in an MUA in HTML format or are you cutting and pasting an HTML document
from another editor into the message body? If the latter, you are not
going to get the correct headers to identify the message content to a
MIME compliant MUA. If the former, your MUA should be inserting these
headers correctly.

As stated in section 1 of RFC2046:

The first document in this set,
<http://www.faqs.org/rfcs//rfcs/rfc2045.html>RFC 2045, defines a number
of header
fields, including Content-Type. The Content-Type field is used to
specify the nature of the data in the body of a MIME entity, by
giving media type and subtype identifiers, and by providing auxiliary
information that may be required for certain media types. After the
type and subtype names, the remainder of the header field is simply a
set of parameters, specified in an attribute/value notation. The
ordering of parameters is not significant.

Thanks, I guess its just a matter of inserting the content-type header
into the message then. I created these messages with a script that
saved its output to a file, then used:
mailx -s "subject" me at mydomain.com <filename.html
If only there is a command line mailer that will allow me to add the
mime type header to the email, then that would allow the message to
display correctly in both my email client, and the list archives.
Anyone know a way to do this?

reply | permalink

Brad Knowles

at Sep 21, 2006 at 4:43 am

⇧

At 5:21 PM -0700 9/20/06, Justin Zygmont wrote:

Thanks, I guess its just a matter of inserting the content-type header
into the message then. I created these messages with a script that
saved its output to a file, then used:
mailx -s "subject" me at mydomain.com <filename.html

If you build the message yourself, as opposed to using a MIME-aware
MUA, then yes -- you will also need to build the necessary MIME
headers and internal infrastructure. However, this can be a complex
subject, and I would recommend that you at least look into
command-line utilities that can help make this process easier.

I can't think of the names of the toolkits off the top of my head,
but you should be able to find them pretty easily with Google.

But do keep in mind that even command-line MUAs like mailx won't let
you control your own headers. If you want to control the headers,
you will need to feed your formatted message directly to sendmail or
some other comparable message submission agent.

When I've done that in the past, it's looked something like:

sendmail -t -fsender at address.he.re.example.com < /path/to/message

If only there is a command line mailer that will allow me to add the
mime type header to the email, then that would allow the message to
display correctly in both my email client, and the list archives.

I'd bet that "mutt" could do that, since it is the MUA that was
originally written by the guy who also wrote the PGP/MIME RFC, so he
clearly knows both his crypto/PGP stuff and his MIME stuff.

Note that mutt does give you some control over what goes in the
headers, but mostly that's for putting in your own "X-" headers and
not doing critical things like mucking about with the MIME structure
of the message.

--
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755

Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.

reply | permalink

Justin Zygmont

at Sep 21, 2006 at 5:03 pm

⇧

Brad Knowles wrote:

At 5:21 PM -0700 9/20/06, Justin Zygmont wrote:

Thanks, I guess its just a matter of inserting the content-type header
into the message then. I created these messages with a script that
saved its output to a file, then used:
mailx -s "subject" me at mydomain.com <filename.html

If you build the message yourself, as opposed to using a MIME-aware MUA,
then yes -- you will also need to build the necessary MIME headers and
internal infrastructure. However, this can be a complex subject, and I
would recommend that you at least look into command-line utilities that
can help make this process easier.

I can't think of the names of the toolkits off the top of my head, but
you should be able to find them pretty easily with Google.

But do keep in mind that even command-line MUAs like mailx won't let you
control your own headers. If you want to control the headers, you will
need to feed your formatted message directly to sendmail or some other
comparable message submission agent.

When I've done that in the past, it's looked something like:

sendmail -t -fsender at address.he.re.example.com < /path/to/message

If only there is a command line mailer that will allow me to add the
mime type header to the email, then that would allow the message to
display correctly in both my email client, and the list archives.

I'd bet that "mutt" could do that, since it is the MUA that was
originally written by the guy who also wrote the PGP/MIME RFC, so he
clearly knows both his crypto/PGP stuff and his MIME stuff.

Note that mutt does give you some control over what goes in the headers,
but mostly that's for putting in your own "X-" headers and not doing
critical things like mucking about with the MIME structure of the message.

thanks, I see it adds proper headers this way, but I still don't see a
way to control what mime type to use. I used the sendmail command with
an html file and it still came through as ascii. I've found that one
workaround is to use mutt -a to include the file as an attachment, but
i'm suprised there's no easy way to do this. Thanks everyone for your
help, I think the lesson is to find something other than uuencode when
sending to mailman so it doesnt spoil the list archives.

reply | permalink

Mark Sapiro

at Sep 21, 2006 at 5:47 pm

⇧

Justin Zygmont wrote:
thanks, I see it adds proper headers this way, but I still don't see a
way to control what mime type to use. I used the sendmail command with
an html file and it still came through as ascii.

When using sendmail to send a message, the input to sendmail needs to
have all headers as well as the message body. At a minimum, you should
have

From: ...
Subject: ...
Date: ...
To: ...
Message-ID: ...
MIME-Version: 1.0
Content-Type: text/html

<html message body>

--
Mark Sapiro <msapiro at value.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan

reply | permalink

group	mailman-users @
categories	python
posted	Sep 15, '06 at 10:52p
active	Sep 21, '06 at 5:47p
posts	10
users	4
website	list.org

responses	expanded	Hotkey: s
font	variable	Hotkey: f
user style	avatars	Hotkey: a

[Mailman-Users] [Fwd: archive emails are garbled]

Search Discussions

Related Discussions

4 users in discussion