I have a python script that can process an ascii text file, but I want
to run this script for one of my mailing lists, so that it processes
the e-mail message (the script populates my database). Where can I
find help with details about writing python scripts to be inserted in
a mailing list pipeline.

i.e. I know my code should have all the includes (mm_cfg, etc...) and
def process(mlist, msg, msgdata)
BUT i do not know what kind of objects mlist, msg and msgdata are,
what their properties are and how to simply retreive the e-mail text
in simple ascii format.

Thank you for your help.

Search Discussions

  • Brad Knowles at Sep 7, 2006 at 1:56 am

    At 3:31 PM -0700 2006-09-06, Nerses Ohanyan wrote:

    I have a python script that can process an ascii text file, but I want
    to run this script for one of my mailing lists, so that it processes
    the e-mail message (the script populates my database). Where can I
    find help with details about writing python scripts to be inserted in
    a mailing list pipeline.
    You're looking for a custom handler. Unfortunately, there don't
    appear to be any FAQs addressing this issue, but I do know that this
    subject has been discussed many times on the mailman-users and
    mailman-developers lists, so you should search the archives.

    When you find posts that provide the information you need, please
    feel free to create a suitable entry for the subject in the
    community-supported FAQ Wizard.

    --
    Brad Knowles, <brad at stop.mail-abuse.org>

    "Those who would give up essential Liberty, to purchase a little
    temporary Safety, deserve neither Liberty nor Safety."

    -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
    Assembly to the Governor, November 11, 1755

    Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.
  • Stephen at Sep 7, 2006 at 4:01 am

    Brad Knowles writes:
    At 3:31 PM -0700 2006-09-06, Nerses Ohanyan wrote:
    >
    I have a python script that can process an ascii text file, but I want
    to run this script for one of my mailing lists, so that it processes
    the e-mail message (the script populates my database). Where can I
    find help with details about writing python scripts to be inserted in
    a mailing list pipeline.
    >
    You're looking for a custom handler. Unfortunately, there don't
    appear to be any FAQs addressing this issue,
    The basic thing you need to realize is that the email message is an
    email.Message message. See the documentation on that module in your
    Python docs, or point pydoc at /usr/lib/python/email. It's pretty
    good.

    Note that there is one method of the message object to recover the
    original text, or maybe you can simply put the message object into a
    string context, to process the whole thing. Or, if you're
    specifically looking at the body or headers, there are specific APIs
    for them, but these return "cooked" Unicode strings.

    Note that Mailman gets these messages raw. Meta-data like envelope
    information is stored in a separate object. If you want to know that
    stuff, you need to access that separately from the email.Message
    object.

    You will be running as the mailman user, I believe, so your database
    will need to be writable by mailman.

    One other hint is that Mailman assumes that most messages will go
    through the pipeline to the end. Thus there is liberal use of
    exceptions to handle practically everything else: filtering spam,
    moderation, etc. What this means is that if (1) your code doesn't
    infloop and (2) you don't change the message or meta-data objects in
    any way, you can wrap your whole handler in "try: ...; except: pass"
    and guarantee that it doesn't affect list delivery.

    I remember that it was easy to create a new log simply by copying
    existing logging code and giving it a new file name. That can be
    useful for debugging. (Sorry, the disk that code was on went away a
    week ago, but that should be enough to get you started.)

    Store your code in a file, say PopulateDB, in the Handlers directory
    of your Mailman installation. Then for the list in question, do
    something like

    $ bin/withlist my-list
    import mm_cfg
    new_pipeline = mm_cfg.GLOBAL_PIPELINE
    new_pipeline.insert(10,'PopulateDB')
    m.Lock()
    m.pipeline = new_pipeline
    m.Save()
    m.Unlock()
    ^D
    $

    'm' is the MailList object. Strictly speaking you don't need to
    Unlock() it, withlist will do that, but I prefer to be pedantic.

    'PopulateDB' is a suggested name. I personally use a prefix to
    identify my local handlers, but there are no community conventions for
    this yet as far as I know.

    I suggest 11th position because that's after a bunch of things like
    spam detection that might throw out the message, but before Mailman
    starts munging (AFAIK, you should check). That may not be appropriate
    (eg, if your database is going to be used in spam detection, you
    probably want that Handler to be first!)

    SpamDetect is probably a good Handler to model your local Handler on;
    it also does textual analysis.

    A final hint: bin/config_list does not know about the pipeline
    attribute. You'll need bin/withlist to access it, even just to read
    it.

    HTH

    Steve

    P.S. I'll eventually get around to posting this to the FAQ, but I've
    already spent more time on email today than I should. Feel free to
    beat me to it!
  • Nerses Ohanyan at Sep 7, 2006 at 9:14 pm
    I have set up everything, but it simply doesn't work. How can I debug
    my code? How can I see what the code is doing (i.e. can I print to
    terminal?)

    Thanks
    On 9/7/06, Nerses Ohanyan wrote:
    Thank you. That's exactly what I needed to know. Unfortunately, I'm
    really not an expert with python or mailman, so I hesitate to write
    FAQ documents, fearing that I might actually confuse someone rather
    then help them.

    Thanks.
    On 9/6/06, stephen at xemacs.org wrote:
    Brad Knowles writes:
    At 3:31 PM -0700 2006-09-06, Nerses Ohanyan wrote:

    I have a python script that can process an ascii text file, but I want
    to run this script for one of my mailing lists, so that it processes
    the e-mail message (the script populates my database). Where can I
    find help with details about writing python scripts to be inserted in
    a mailing list pipeline.
    You're looking for a custom handler. Unfortunately, there don't
    appear to be any FAQs addressing this issue,
    The basic thing you need to realize is that the email message is an
    email.Message message. See the documentation on that module in your
    Python docs, or point pydoc at /usr/lib/python/email. It's pretty
    good.

    Note that there is one method of the message object to recover the
    original text, or maybe you can simply put the message object into a
    string context, to process the whole thing. Or, if you're
    specifically looking at the body or headers, there are specific APIs
    for them, but these return "cooked" Unicode strings.

    Note that Mailman gets these messages raw. Meta-data like envelope
    information is stored in a separate object. If you want to know that
    stuff, you need to access that separately from the email.Message
    object.

    You will be running as the mailman user, I believe, so your database
    will need to be writable by mailman.

    One other hint is that Mailman assumes that most messages will go
    through the pipeline to the end. Thus there is liberal use of
    exceptions to handle practically everything else: filtering spam,
    moderation, etc. What this means is that if (1) your code doesn't
    infloop and (2) you don't change the message or meta-data objects in
    any way, you can wrap your whole handler in "try: ...; except: pass"
    and guarantee that it doesn't affect list delivery.

    I remember that it was easy to create a new log simply by copying
    existing logging code and giving it a new file name. That can be
    useful for debugging. (Sorry, the disk that code was on went away a
    week ago, but that should be enough to get you started.)

    Store your code in a file, say PopulateDB, in the Handlers directory
    of your Mailman installation. Then for the list in question, do
    something like

    $ bin/withlist my-list
    import mm_cfg
    new_pipeline = mm_cfg.GLOBAL_PIPELINE
    new_pipeline.insert(10,'PopulateDB')
    m.Lock()
    m.pipeline = new_pipeline
    m.Save()
    m.Unlock()
    ^D
    $

    'm' is the MailList object. Strictly speaking you don't need to
    Unlock() it, withlist will do that, but I prefer to be pedantic.

    'PopulateDB' is a suggested name. I personally use a prefix to
    identify my local handlers, but there are no community conventions for
    this yet as far as I know.

    I suggest 11th position because that's after a bunch of things like
    spam detection that might throw out the message, but before Mailman
    starts munging (AFAIK, you should check). That may not be appropriate
    (eg, if your database is going to be used in spam detection, you
    probably want that Handler to be first!)

    SpamDetect is probably a good Handler to model your local Handler on;
    it also does textual analysis.

    A final hint: bin/config_list does not know about the pipeline
    attribute. You'll need bin/withlist to access it, even just to read
    it.

    HTH

    Steve

    P.S. I'll eventually get around to posting this to the FAQ, but I've
    already spent more time on email today than I should. Feel free to
    beat me to it!
  • Brad Knowles at Sep 7, 2006 at 10:21 pm

    At 2:14 PM -0700 2006-09-07, Nerses Ohanyan wrote:

    I have set up everything, but it simply doesn't work. How can I debug
    my code? How can I see what the code is doing (i.e. can I print to
    terminal?)
    Unfortunately, Python doesn't have a lot of debugging tools. You can
    either run the program interactively, or you can add some print or
    logging statements, and that's about all I know of.

    --
    Brad Knowles, <brad at stop.mail-abuse.org>

    "Those who would give up essential Liberty, to purchase a little
    temporary Safety, deserve neither Liberty nor Safety."

    -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
    Assembly to the Governor, November 11, 1755

    Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.
  • Stephen at Sep 8, 2006 at 5:41 am
    Nerses Ohanyan writes:
    I have set up everything, but it simply doesn't work. How can I debug
    my code? How can I see what the code is doing (i.e. can I print to
    terminal?)
    You can run it under an interactive Python, I'm sure, but you'll need
    to do a bunch of importing by hand and preparation of test input and
    stuff like that. It will depend on what your code does. I've never
    done it, though, so I can't really say.

    As I wrote, you can write to a log, there are a couple of examples in
    the standard handlers. If you want to see that in real time, you can
    tail the log. It's as easy as

    from Mailman.Logging.Syslog import syslog
    def process(mlist, msg, msgdata):
    syslog('mylog', 'Got right HERE --> <-- I mean, LEFT here!')

    and you'll find the "got here ..." message in $MAILMAN_HOME/log/mylog.

    If you want something more interactive than that, it's beyond my
    experience. I'd suggest that you ask on comp.lang.python, but of
    course you'll have to provide most of the Mailman specifics yourself.

    HTH

    Steve
  • Nerses Ohanyan at Sep 7, 2006 at 2:47 am
    Thanks. I looked at the archives, but so far I'm not seeing anything
    particularly useful. Does anyone know what exactly is being passed to
    my handler? does anyone know where msg, mlist and msgdata are defined
    (or the appropriate variables) ? Not even sure I'm making sense.

    I'll keep digging and if I find anything I'll post.

    Thanks again Brad.
    On 9/6/06, Brad Knowles wrote:
    At 3:31 PM -0700 2006-09-06, Nerses Ohanyan wrote:

    I have a python script that can process an ascii text file, but I want
    to run this script for one of my mailing lists, so that it processes
    the e-mail message (the script populates my database). Where can I
    find help with details about writing python scripts to be inserted in
    a mailing list pipeline.
    You're looking for a custom handler. Unfortunately, there don't
    appear to be any FAQs addressing this issue, but I do know that this
    subject has been discussed many times on the mailman-users and
    mailman-developers lists, so you should search the archives.

    When you find posts that provide the information you need, please
    feel free to create a suitable entry for the subject in the
    community-supported FAQ Wizard.

    --
    Brad Knowles, <brad at stop.mail-abuse.org>

    "Those who would give up essential Liberty, to purchase a little
    temporary Safety, deserve neither Liberty nor Safety."

    -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
    Assembly to the Governor, November 11, 1755

    Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.
  • Matthew Thompson at Sep 7, 2006 at 2:16 pm
    When I implemented a custom handler for my server, I followed this FAQ:
    http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.067.htp

    Between that, the python docs that someone else mentioned and looking at the
    existing handlers in Mailman/Handlers I was able to get things going with
    little trouble.

    Good luck,
    -Matt
    On Wednesday 06 September 2006 21:47, Nerses Ohanyan wrote:
    Thanks. I looked at the archives, but so far I'm not seeing anything
    particularly useful. Does anyone know what exactly is being passed to
    my handler? does anyone know where msg, mlist and msgdata are defined
    (or the appropriate variables) ? Not even sure I'm making sense.

    I'll keep digging and if I find anything I'll post.

    Thanks again Brad.
    On 9/6/06, Brad Knowles wrote:
    At 3:31 PM -0700 2006-09-06, Nerses Ohanyan wrote:
    I have a python script that can process an ascii text file, but I want
    to run this script for one of my mailing lists, so that it processes
    the e-mail message (the script populates my database). Where can I
    find help with details about writing python scripts to be inserted in
    a mailing list pipeline.
    You're looking for a custom handler. Unfortunately, there don't
    appear to be any FAQs addressing this issue, but I do know that this
    subject has been discussed many times on the mailman-users and
    mailman-developers lists, so you should search the archives.

    When you find posts that provide the information you need, please
    feel free to create a suitable entry for the subject in the
    community-supported FAQ Wizard.

    --
    Brad Knowles, <brad at stop.mail-abuse.org>

    "Those who would give up essential Liberty, to purchase a little
    temporary Safety, deserve neither Liberty nor Safety."

    -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
    Assembly to the Governor, November 11, 1755

    Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedSep 6, '06 at 10:31p
activeSep 8, '06 at 5:41a
posts8
users4
websitelist.org

People

Translate

site design / logo © 2022 Grokbase