FAQ

On Fri, 2006-04-28 at 19:12 -0500, Brad Knowles wrote:

I think we need to gather a lot more information about the likely
outcome from this change, and I think the best way to achieve this is
through giving admins (either site admins or list admins) the ability
to set an option and choose whether or not they want to see what
happens.
I agree that we need a lot more data, but I'm not sure making this an
option is the best way to gather that data. Besides, if we do it that
way, it'll be Mailman 2.3 (or whatever <3.0 wink>) before we make the
change.

I'd like to work up an unofficial diff to Mailman 2.1 for people like
Stephen who are willing to give it a try on a live site. We just have
to agree as to what that change should be!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 309 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/mailman-users/attachments/20060501/fbd11fb6/attachment.pgp

Search Discussions

  • Barry Warsaw at May 1, 2006 at 1:34 pm

    On Sun, 2006-04-30 at 00:00 +0900, Stephen J. Turnbull wrote:

    Sender doesn't instruct *conformant* MTAs at all, does it? AFAIK the
    only thing that a RFC 2821-conforming MTA looks at is the Return-Path
    header, and it's supposed to remove that.

    So this is purely a matter of pragmatic self-defense against broken
    MTAs that do bounce to Sender.
    Correct, and what we're trying to figure out is whether we need that
    self-defense any longer. The change to test this may be as simple as
    commenting out "msg['Sender'] = envsender" in bulkdeliver() inside
    SMTPDirect.py (a little more complicated if you want to do it just for
    one domain though -- you'd want to test for something like "if
    'xemacs.org' in mlist.host_name")
    Agreed. For a number of reasons, I think this information can be
    useful. As I mentioned elsewhere, the Resent-Message-Id field can be
    used to supply a UUID that we can trust (eg, for constructing
    canonical archive URLs). Unlike the Received headers, these are
    readable by humans who aren't wall-eyed, helpful in tracing delays,
    for example.
    It's an intersting idea.

    -Barry

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 309 bytes
    Desc: This is a digitally signed message part
    Url : http://mail.python.org/pipermail/mailman-users/attachments/20060501/5471c4b3/attachment.pgp
  • Neal Groothuis at May 1, 2006 at 6:27 pm

    I'd like to work up an unofficial diff to Mailman 2.1 for people like
    Stephen who are willing to give it a try on a live site.
    I'm not sure this is even necessary.

    Ezmlm doesn't touch the Sender: header at all, Majordomo sets it to the
    owner of the list, and (AFAICT) Listserv sets it to the list itself.
    This would seem to me to indicate that incidences of mail being returned
    inappropriately to the sender are infrequent, at worst.

    The important question would seem to be "what's appropriate?" From RFC
    2822, 3.6.2: "The originator fields indicate the mailbox(es) of the
    source of the message." Given this, the definition of the Sender and
    From headers, and the example given in this section, it seems that
    Outlook's interpretation of the fields ("SENDER on behalf of FROM") is
    reasonable. Mailman is not the originator of the message, so it should
    not be tampering with the From: or Sender: fields at all.

    It might be appropriate for Mailman to add Resent-* headers, depending
    on how one reads RFC 2822, 3.6.6. I personally don't think it's
    necessary or useful, since list servers add their own List-* headers,
    per RFC 2369. The Resent-* headers seem to exist for individuals
    resending messages, not automated systems. This is supported by the
    RFC: "Resent fields are used to identify a message as having been
    reintroduced into the transport system by a user."
  • William D. Tallman at May 1, 2006 at 8:03 pm
    Watching this with interest; a newbie learns...

    On Mon, May 01, 2006 at 01:27:40PM -0500, Neal Groothuis wrote:

    <snip>
    It might be appropriate for Mailman to add Resent-* headers, depending
    on how one reads RFC 2822, 3.6.6. I personally don't think it's
    necessary or useful, since list servers add their own List-* headers,
    per RFC 2369. The Resent-* headers seem to exist for individuals
    resending messages, not automated systems. This is supported by the
    RFC: "Resent fields are used to identify a message as having been
    reintroduced into the transport system by a user."
    How does the RFC, or the writers thereof, define "user"? An automated
    system is the tool of some deliberate intent, implying (necessarily?)
    the will of a "user", I would think.

    Or is this relevant?

    Thanks for reading.

    Bill Tallman
  • Stephen J. Turnbull at May 3, 2006 at 2:11 pm
    "William" == William D Tallman <wtallman at olypen.com> writes:
    William> How does the RFC, or the writers thereof, define "user"?

    They don't. IMHO (there are those more expert than I on this list)
    anything that is normally expected to touch the headers or body of a
    message is a "user" for the purpose of RFC 2822. What is excluded is
    the mail transport system (MTAs) which are specified in RFC 2821.

    There is also a distinction between "originators" and "others".
    Certain headers ("From", "Subject", "Date", etc) are specified as for
    the use of the originator. Other headers are generally specified in
    terms of their semantics alone, and not according to who may use them.

    William> An automated system is the tool of some deliberate
    William> intent, implying (necessarily?) the will of a "user", I
    William> would think.

    I don't think that is the way that RFC writers in general think.

    Although there is a desire for RFC 2822 headers to be intelligible to
    humans, and many are very useful in day-to-day work, RFCs are in the
    end about machine-to-machine communication. Thus the focus is on
    syntax so that machines can parse them without knowing what they mean,
    and of semantics that allow machines to make a good guess at what the
    humans are going to want.

    For example, if there is a Reply-To header, the From header should be
    ignored, and the Reply-To address used in composing the addressee list
    for a reply. However, one of the examples used IIRC is where the
    author of the original message is traveling and uses his own address
    as From (that's the identity the recipient recognizes) but Reply-To to
    direct the response to his host, whose email address he is borrowing.
    Now, a human who replies a week later will know that the boss is back
    home and want to reply to From but the mail client can't know that.

    So a good mail client will initialize the address of a reply to the
    Reply-To, but provide a way for the user to override. The RFC only
    specifies the former, though, and only as a suggestion. Exactly how
    to handle this problem is a user interface issue, and the RFC remains
    silent on such issues.

    William> Or is this relevant?

    Yes. Sometimes such definitions are made explicitly. I don't think
    they exist in this case, but it's a very good idea to ask.

    *****

    Disclaimer: this is the way I think about these things, and I've found
    it useful in understanding what RFCs do and don't say. Others will
    surely have different opinions.

    --
    School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
    University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
    Ask not how you can "do" free software business;
    ask what your business can "do for" free software.
  • William D. Tallman at May 4, 2006 at 3:46 am

    On Wed, May 03, 2006 at 11:11:22PM +0900, Stephen J. Turnbull wrote:
    "William" == William D Tallman <wtallman at olypen.com> writes:
    William> How does the RFC, or the writers thereof, define "user"?

    They don't. IMHO (there are those more expert than I on this list)
    anything that is normally expected to touch the headers or body of a
    message is a "user" for the purpose of RFC 2822. What is excluded is
    the mail transport system (MTAs) which are specified in RFC 2821. Okay.
    There is also a distinction between "originators" and "others".
    Certain headers ("From", "Subject", "Date", etc) are specified as for
    the use of the originator. Other headers are generally specified in
    terms of their semantics alone, and not according to who may use them. Okay.
    William> An automated system is the tool of some deliberate
    William> intent, implying (necessarily?) the will of a "user", I
    William> would think.

    I don't think that is the way that RFC writers in general think.
    Yes, so I gather.
    Although there is a desire for RFC 2822 headers to be intelligible to
    humans, and many are very useful in day-to-day work, RFCs are in the
    end about machine-to-machine communication. Thus the focus is on
    syntax so that machines can parse them without knowing what they mean,
    and of semantics that allow machines to make a good guess at what the
    humans are going to want.
    Okay, that follows.
    For example, if there is a Reply-To header, the From header should be
    ignored, and the Reply-To address used in composing the addressee list
    for a reply. However, one of the examples used IIRC is where the
    author of the original message is traveling and uses his own address
    as From (that's the identity the recipient recognizes) but Reply-To to
    direct the response to his host, whose email address he is borrowing.
    Now, a human who replies a week later will know that the boss is back
    home and want to reply to From but the mail client can't know that.
    Which means that people really should learn to use email systems
    intelligently, with the MUA of choice as the means of control.
    So a good mail client will initialize the address of a reply to the
    Reply-To, but provide a way for the user to override. The RFC only
    specifies the former, though, and only as a suggestion. Exactly how
    to handle this problem is a user interface issue, and the RFC remains
    silent on such issues.
    Implication here is that 'user' is a real human being, not a software
    agent.

    RFC2822, section 3.6.6 discusses re-sending fields as intended for use
    by a re-sending 'user'. It also specifies that these fields are for
    informational purposes only, and MUST NOT be used to actively manipulate
    the message. As a re-sender does not alter the originating fields,
    software presumably cannot automagically use that information to ID the
    source of the message, which remains the purview of the originating
    fields.

    So an email list server cannot act as a re-sender, IIUC.

    The alternative is that the server actually initiates a new message as a
    'forwarding' agent. That means that the server must (MUST?) identify
    itself in the originator fields. The address of the author of the
    message goes in the From: field, and the address of the forwarder (the
    email list's originating mailbox) goes in the Sender: field, with
    information on responses in the Reply-To: field. As the author is not
    the email list server, the address of the server's mailer MUST be by
    itself in the Sender: field. All as per section 3.6.5.

    Additionally, one would think that the server is a 'forwarder' because
    the message it sends out is not identical to the message it receives: it
    adds footers, etc.

    IIUC, that is. Which apparently I do not, having read through the
    headers of a message from this list.

    There is no Sender: field. The first field is apparently an
    unstructured field with no identifier with the canonical following
    colon. It contains the sender (mailman-users-bounces...) and the date,
    presumably of sending. The second field is Return-Path: with an
    'addr-spec'. The originator fields are untouched.

    Which means that the list server is neither a re-sender or a forwarder,
    I gather, and that means I don't understanding any of this at all! Or
    is it that the server really is a re-sender in disguise and my MUA (MDA,
    actually: Procmail) is forced to process this message to its final
    destination in my mail system illegally?

    As I'm (recreationally) in the process of setting up and understanding a
    wee Mailman server on my own system, I'd really like to understand all
    this, but looks like I've got a ways to go.
    William> Or is this relevant?

    Yes. Sometimes such definitions are made explicitly. I don't think
    they exist in this case, but it's a very good idea to ask.
    Okay, thanks for this response!

    And thanks again for reading,

    Bill Tallman
  • Stephen J. Turnbull at May 8, 2006 at 3:33 pm
    "William" == William D Tallman <wtallman at olypen.com> writes:
    William> On Wed, May 03, 2006 at 11:11:22PM +0900, Stephen
    William> J. Turnbull wrote:
    I don't think that is the way that RFC writers in general
    think.
    William> Yes, so I gather.

    :-)

    William> Which means that people really should learn to use email
    William> systems intelligently, with the MUA of choice as the
    William> means of control.

    I firmly believe that, but unfortunately there are lots of MUAs that
    don't really permit intelligent use. Many people "inherit" an MUA
    either from their OS or maybe their brother-in-law, and do not have
    the desire or resources to change MUAs or even learn to use the one
    they've got effectively.

    There are a number of ways to look at this, but what the people who
    write RFCs have come to insist is that you be strict with yourself
    (always follow the rules and best practices), while being lenient with
    others. You can think of this as simply aikido on the Internet (==
    self-defense), or some kind of generosity to those less clued than
    oneself, but the rule works well whyever you follow it. :-)
    So a good mail client will initialize the address of a reply to
    the Reply-To, but provide a way for the user to override. The
    RFC only specifies the former, though, and only as a
    suggestion. Exactly how to handle this problem is a user
    interface issue, and the RFC remains silent on such issues.
    William> Implication here is that 'user' is a real human being,
    William> not a software agent.

    In this particular case, user refers to "user of a good mail client",
    presumably human (but it could easily be an Emacs Lisp program or an
    expect script ... ok, ok, that's not "easy", that's "heroic" ... but
    it could be heroicly!) However, most of RFC 2822 doesn't refer to
    how the headers should be treated by a mail client, just to what they
    mean. That meaning could be useful to a human, or a mailing list
    server, or whatever.

    William> RFC2822, section 3.6.6 discusses re-sending fields as
    William> intended for use by a re-sending 'user'. It also
    William> specifies that these fields are for informational
    William> purposes only, and MUST NOT be used to actively
    William> manipulate the message.

    "Automatically." There's nothing that says that you can't write a
    mail client that has an "bounce followup" feature which looks for
    sender, resent-sender and so on, and adds them to the "To" header, as
    well as formatting the body with a summary of the progress of the
    message by using Date and Resent-Date headers.

    William> So an email list server cannot act as a re-sender, IIUC.

    I don't see why not. I think you're overinterpreting the RFCs.
    Certainly, in this case "human user" is a leading interpretation. But
    if the actions described could be executed by a program, then there's
    no good reason not to interpret "user" as possibly being a program,
    unless the RFC explicitly says so.

    William> The alternative is that the server actually initiates a
    William> new message as a 'forwarding' agent.

    I don't think either of the meanings of "forward" suggested in RFC
    2822 section 3.6.6 apply here. ("New message with old message as
    body" clearly applies to digests, but I think we're more interested in
    non-digest delivery in this thread.)

    [William gives an example]
    William> That means that the server must (MUST?) identify itself
    William> in the originator fields.

    No, I think that's wrong. If the server wants to claim responsibility
    for injecting the message into the mail system, then it should put
    itself in the Sender field. This absolves the original Sender of all
    responsibility for misformatting of the message, misdirection to wrong
    addresses, etc, etc. If the server doesn't change the body at all,
    and only adds new headers, then I think it should not do this. In the
    grey areas like Mailman, it's unclear.

    However, suppose Mailman is configured to leave the headers alone, and
    to leave the body alone, forwarding verbatim to the addresses on the
    mailing list (except for adding its List-XXXX headers, etc). I would
    argue that since Mailman doesn't automatically forward, but instead
    checks for spam, whether you're subscribed or not, whether the
    subscriber is already an addressee in the message, etc., it makes
    decisions about what to send where, and is therefore a "user" in the
    sense of section 3.6.6.

    Mailman SHOULD add Resent-XXXX headers, because if delivery gets
    screwed up, bugs in its logic should be considered a candidate cause.
    Ie, those headers mean that Mailman accepts partial responsibility for
    misdirecting the reinjection of the message into the mail transport
    system.

    For example, suppose Mailman hiccups and reinjects the same post
    twice. It would be useful to check whether the Resent-Message-Ids
    differ. If they do, you know for sure it was Mailman. If they don't,
    it doesn't prove it wasn't Mailman, but you do know that the phase
    where the error occurred was fairly late in Mailman's delivery
    pipeline.

    William> IIUC, that is. Which apparently I do not, having read
    William> through the headers of a message from this list.

    While I respect the Mailman authors for their efforts to conform to
    the RFCs, nobody's perfect. Specifically there is controversy about
    whether Mailman's treatment of the Sender field is conforming to
    RFCs. There may be other errors (I don't know of any, but I don't
    know everything :-).

    William> There is no Sender: field.

    This implies that whoever is in From is Sender.

    William> The first field is apparently an unstructured field with
    William> no identifier with the canonical following colon.

    This was added by *your* MTA when it delivered to your mailbox. It
    was not present at any time before the message reached your host.
    This is a non-standard (but very frequently used) format for mailboxes
    called "Unix mbox".

    Here "non-standard" does not mean "does not conform to the standard"
    for some value of "the". It means "this is neither defined nor
    prohibited by any formal standard."

    William> It contains the sender (mailman-users-bounces...) and the
    William> date, presumably of sending.

    The date is a timestamp added when the message was delivered to your
    mailbox.

    William> The second field is Return-Path: with an 'addr-spec'.

    This was also added by your MTA. This is a standard header, specified
    by RFC 2821 (the RFC for the protocol spoken by one MTA to another).

    William> The originator fields are untouched.

    This is not always true. Mailman will sometimes alter the From header
    to preserve privacy, IIRC.

    William> Which means that the list server is neither a re-sender
    William> or a forwarder,

    Neither of these have a definition in RFC 2822 (the RFC for the
    protocol spoken by MUAs, to each other and to human users). There are
    two suggested interpretations for forwarder, and nothing very specific
    about resender, except that it is not the kind of thing the MTAs do.

    Note that although technically when Mailman adds a footer it has
    created a new message containing a copy of the original, I doubt
    *people* think of it that way. As long as there's a complete,
    verbatim copy of the original, the header and footer are considered
    decoration, not part of the message. Even if HTML parts or images are
    removed, I think *people* still consider the message "the same". So I
    don't think these operations necessarily disqualify Mailman as a
    resender. (If you convince me that people will often consider these
    operations to create a *new* message, then I'd have to rethink.)

    As far as I can tell, a re-sender is any agent that (1) receives a
    message, (2) sends it elsewhere, and (3) adds Resent-XXXX headers to
    the resent message.

    You could argue that there's a condition that Mailman, as the
    resender, be a "bona fide addressee" of the message. Well, (1) it's
    the envelope recipient and probably in the addressee headers as well,
    and (2) Mailman often keeps file copies (AKA mailing list archives).
    You could claim that (1) is just an accident of technology, but I
    don't think you can explain away (2).

    To put all this another way, the only intermediary actually defined by
    these RFCs is the "mail transport agent" (MTA), which accepts a sender
    address, a list of recipients, and message data from another agent,
    and returns to that agent a promise to deliver. It then contacts
    other MTAs that it believes represent some of those recipients, and
    passes on the list of (a subset of) recipients, the sender address and
    the message data, or delivers the message data verbatim to mailboxes
    it has responsibility for. Once it has received promises to deliver
    for all of the recipients on the list it received, it is done, and can
    purge the message data and envelope information from its queues. This
    is not what Mailman does, so it's a "user".

    Note that a quick search of the RFC site for "delivery agent" or "MDA"
    turns up nothing relevant, and a search for "mailing list" turns up
    only RFCs 2369 and 2919 (describing the List-XXXX headers, but not the
    kind of thing we're talking about) and RFC 1429 which describes an
    obsolete service called DISTRIBUTE that is something intermediate
    between a mailing list and a Usenet news server (and hosted on Bitnet
    *shiver*). There are no relevant standards yet.

    William> my MUA (MDA, actually: Procmail) is forced to process
    William> this message to its final destination in my mail system
    William> illegally?

    I don't really understand what you're trying to say. I can offer the
    following random comments:

    "Illegal" of course is a slight overstatement. :-) The preferred
    word is "nonconforming".

    But I don't think it's very useful to talk about "nonconforming
    delivery processing" here. Nothing about the delivery process is
    non-conforming for the simple reason that the *only* things that you
    can see in the message that reflect delivery processing are the
    Received and Return-Path headers, and the (nonstandard) Unix mbox
    "From " headline.

    The other headers are all for the use of the MUAs and the users
    (including Mr. Procmail).

    A general remark: You might find RFC 2369 useful as context (and in
    the future, to understand Mailman's use of those headers, though it's
    not relevant to this thread).


    --
    School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
    University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
    Ask not how you can "do" free software business;
    ask what your business can "do for" free software.
  • William D. Tallman at May 8, 2006 at 7:02 pm

    On Tue, May 09, 2006 at 12:33:29AM +0900, Stephen J. Turnbull wrote:
    "William" == William D Tallman <wtallman at olypen.com> writes:
    <snip>

    Well damn!!! I am genuinely impressed and appreciative of this
    response! Have it saved off in a separate file to study. Mr. Turnbull
    has my sincere thanks for his effort here, and I hope that others may
    have found it as valuable as did I.

    On reflection, I stand instructed in several respects (not just about
    failing to discount what my own MTA added to the headers :D ), but
    specifically in the distinction between illegal and non-conforming. I
    should well have known better than that, having some knowledge of
    programming (C), and being a long-time detractor of message windows
    produced by a popular operating system to the effect that one has
    "performed an ILLEGAL operation" (emphasis mine).

    I may never actually put Mailman into service, but the project is both
    interesting and instructive, in no small part in consequence of the
    traffic on this list. Again, my thanks to Mr. Turnbull.

    Thanks for reading,

    Bill Tallman
  • Mark Sapiro at May 2, 2006 at 1:16 am

    Neal Groothuis wrote:
    Mailman is not the originator of the message, so it should
    not be tampering with the From: or Sender: fields at all.

    This is arguably not true. Mailman may add a list header and/or list
    footer to the body of the message as well as potentially filtering or
    scrubbing various MIME parts. The message sent by Mailman can be
    significantly different from the one originally received.

    --
    Mark Sapiro <msapiro at value.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Barry Warsaw at May 2, 2006 at 2:03 pm

    On Mon, 2006-05-01 at 18:16 -0700, Mark Sapiro wrote:
    Neal Groothuis wrote:
    Mailman is not the originator of the message, so it should
    not be tampering with the From: or Sender: fields at all.

    This is arguably not true. Mailman may add a list header and/or list
    footer to the body of the message as well as potentially filtering or
    scrubbing various MIME parts. The message sent by Mailman can be
    significantly different from the one originally received.
    The copy that Mailman sends is almost always modified in some way, and
    given the ambiguities in the standards, I think it's defensible to say
    that Mailman is the originator of the messages received by list members.

    -Barry

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 309 bytes
    Desc: This is a digitally signed message part
    Url : http://mail.python.org/pipermail/mailman-users/attachments/20060502/58c12a4c/attachment.pgp
  • Barry Warsaw at May 2, 2006 at 2:22 pm

    On Mon, 2006-05-01 at 13:27 -0500, Neal Groothuis wrote:
    I'd like to work up an unofficial diff to Mailman 2.1 for people like
    Stephen who are willing to give it a try on a live site.
    I'm not sure this is even necessary.

    Ezmlm doesn't touch the Sender: header at all, Majordomo sets it to the
    owner of the list, and (AFAICT) Listserv sets it to the list itself.
    This would seem to me to indicate that incidences of mail being returned
    inappropriately to the sender are infrequent, at worst.
    As I said, I think it's defensible to claim Mailman is the originator,
    but practicality beats purity, and I do think a list manager falls into
    a gray area here.

    Having said all that, you might be right, in that we really don't need
    to do much except remove the addition of the Sender: header in
    bulkdeliver().

    -Barry

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 309 bytes
    Desc: This is a digitally signed message part
    Url : http://mail.python.org/pipermail/mailman-users/attachments/20060502/2f587ad8/attachment.pgp

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedMay 1, '06 at 1:12p
activeMay 8, '06 at 7:02p
posts11
users5
websitelist.org

People

Translate

site design / logo © 2022 Grokbase