FAQ
Hi all,



I started a new thread for this as it no longer applies to cron, but

rather digests.



I just ran the senddigests cron option (again, for the first time ever)

but it didn't seem to do anything. I can't really understand what it
does anyway

(from the description given). All of our lists allow users to select the
digest

option ("digestable=yes"), and the digests are sent out when the 30k
threshold is

reached ("digest_size_threshold0"). The next option asks if a digest
should be

sent out DAILY when the threshold isn't reached
("digest_send_periodic=yes").



So that is how all of our lists have been set all this time, and the
digest mails

seem to have been working, despite not having run anything in crontab.



The crontab entry for senddigests says:



# Noon, mail digests for lists that do periodic as well as threshhold
delivery.

0 12 * * * /usr/bin/python -S /mail/mailman/cron/senddigests





Not sure what this does differently, other than maybe force the digests
to go out

at noon, rather than the vague "DAILY" option from above. Can anyone
share any

light? I just ran it and nothing seemed to happen.



Thanks!

Search Discussions

  • Mark Sapiro at Mar 3, 2009 at 9:30 pm

    Savoy, Jim wrote:
    All of our lists allow users to select the
    digest

    option ("digestable=yes"), and the digests are sent out when the 30k
    threshold is

    reached ("digest_size_threshold0"). The next option asks if a digest
    should be

    sent out DAILY when the threshold isn't reached
    ("digest_send_periodic=yes").



    So that is how all of our lists have been set all this time, and the
    digest mails

    seem to have been working, despite not having run anything in crontab.

    The digest messages are accumulated for a list in a mbox format file
    lists/LISTNAME/digest.mbox. When a new message arrives and is added to
    digest.mbox and the size of digest.mbox is now greater than the list's
    digest_size_threshold, a digest is sent at that time for that list,
    and the digest.mbox is removed.

    So, digests based on digest_size_threshold will work whether or not
    cron/senddigests is run.

    The crontab entry for senddigests says:



    # Noon, mail digests for lists that do periodic as well as threshhold
    delivery.

    0 12 * * * /usr/bin/python -S /mail/mailman/cron/senddigests





    Not sure what this does differently, other than maybe force the digests
    to go out

    at noon, rather than the vague "DAILY" option from above. Can anyone
    share any

    light? I just ran it and nothing seemed to happen.

    Running cron/senddigests will send a digest for any list which has a
    digest.mbox file and for which digest_send_periodic=yes regardless of
    how big the digest is. So, if you ran it and nothing happened, there
    were no such lists. This seems unlikely. Are you sure no digests were
    sent?

    And yes, running it at noon daily forces the periodic digests at noon,
    but you can chose another time. I prefer mine to go over night so I
    run cron/senddigests daily at 03:30.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 3, 2009 at 10:53 pm

    Mark Sapiro wrote:
    The digest messages are accumulated for a list in a mbox format file
    lists/LISTNAME/digest.mbox. When a new message arrives and is added to
    digest.mbox and the size of digest.mbox is now greater than the list's
    digest_size_threshold, a digest is sent at that time for that list,
    and the digest.mbox is removed.
    So, digests based on digest_size_threshold will work whether or not
    cron/senddigests is run.

    Right. I'm with you so far.
    Running cron/senddigests will send a digest for any list which has a
    digest.mbox file and for which digest_send_periodic=yes regardless of
    how big the digest is.
    This is where we part ways.

    Without the senddigests cron job, isn't the above statement also true
    anyway? Because the digest_sent_periodic=yes setting also doesn't care
    if the digest.mbox is over 30K. It just sends out a message "daily"
    (what
    time, I do not know) whenever a digest.mbox exists.

    So my question was: Is the only reason you run the cron job in the first
    place,
    Is to force that "daily" run to a specific time (eg noon)? I say this
    because I
    have never ran that cron job, but I'm pretty sure everyone gets their
    digests
    anyway.

    So, if you ran it and nothing happened, there were no such lists. This seems
    unlikely. Are you sure no digests were sent?
    I guess it's possible no digest.mboxes exist. There aren't that many
    digest
    users here.
    And yes, running it at noon daily forces the periodic digests at noon,
    but you can chose another time. I prefer mine to go over night so I
    run cron/senddigests daily at 03:30.
    I will probably set it up to run at 3:00 am as well. I just ran it at
    12:48
    today so I could watch it (do nothing (!)).

    I suppose I should change the setting that sends a digest if it exceeds
    30K
    too (change it to 1000K or something, so that people never get more than
    one
    digest-per-day), and just use the cron job to send once a day,
    regardless of size,
    at 3:00. I just am not clear on why I need a cronjob to do that, unless
    its
    only purpose is to force a specific time.

    - jim -
  • Mark Sapiro at Mar 3, 2009 at 11:06 pm

    Savoy, Jim wrote:
    Mark Sapiro wrote:
    Running cron/senddigests will send a digest for any list which has a
    digest.mbox file and for which digest_send_periodic=yes regardless of
    how big the digest is.
    This is where we part ways.

    Without the senddigests cron job, isn't the above statement also true
    anyway?

    No.

    Because the digest_sent_periodic=yes setting also doesn't care
    if the digest.mbox is over 30K. It just sends out a message "daily"
    (what
    time, I do not know) whenever a digest.mbox exists.

    It relies on cron/senddigests being run in order to do this and "daily"
    is whenever cron/senddigests runs and could be "twice daily" or "Mon,
    Wed and Fri" or whatever depending on when cron/senddigests runs.

    So my question was: Is the only reason you run the cron job in the first
    place,
    Is to force that "daily" run to a specific time (eg noon)? I say this
    because I
    have never ran that cron job, but I'm pretty sure everyone gets their
    digests
    anyway.

    Either they were only getting digests when the size threshold was
    reached, or if they were getting them daily at a specific time, then
    something was running cron/senddigests at that time.

    So, if you ran it and nothing happened, there were no such lists. This seems
    unlikely. Are you sure no digests were sent?
    I guess it's possible no digest.mboxes exist. There aren't that many
    digest
    users here.
    And yes, running it at noon daily forces the periodic digests at noon,
    but you can chose another time. I prefer mine to go over night so I
    run cron/senddigests daily at 03:30.
    I will probably set it up to run at 3:00 am as well. I just ran it at
    12:48
    today so I could watch it (do nothing (!)).

    I suppose I should change the setting that sends a digest if it exceeds
    30K
    too (change it to 1000K or something, so that people never get more than
    one
    digest-per-day), and just use the cron job to send once a day,
    regardless of size,
    at 3:00. I just am not clear on why I need a cronjob to do that, unless
    its
    only purpose is to force a specific time.

    You need the cron job because nothing else in Mailman sends digests
    periodically. The only other mechanism sends a digest when the list's
    digest.mbox exceeds a certain size, which may not happen for weeks on
    a low traffic list.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 3, 2009 at 11:25 pm

    Mark Sapiro wrote:
    You need the cron job because nothing else in Mailman sends digests
    periodically. The only other mechanism sends a digest when the list's
    digest.mbox exceeds a certain size, which may not happen for weeks on
    a low traffic list.

    Ok I understand 100% now. The only reason I kept telling you that I am
    "pretty sure" people are getting their digests, is because I went to
    someone's
    office once and saw that they had some digests in their inbox.

    And I personally forced one of my own tests lists to get a 30K
    digest.mbox
    so I could see what a digest delivery looked like.

    In looking over my recursive listing closely, I see that there are 501
    total digest.mboxes out there (!). All of them are 30K or under.

    That still doesn't explain why they are all still there and nothing
    was
    delivered when I ran the cron job at 12:48 today...
    Are you sure digest_send_periodic is set to Yes for those lists that have
    old digest.mbox files?
    Absolutely. That is the default we set up and I just checked a bunch of
    lists to verify it.

    I did check out a couple of the lists that had outstanding digest.mboxes
    and found that they didn't have any subscribers with the digest option
    checked
    (as I said, very few of our users know about and use this feature) so
    perhaps
    nothing was delivered today because there was just nothing to be
    delivered.
    I take it a digest.mobx is created regardless of whether or not a list
    has
    any members using that option. So perhaps everything is fine and
    starting this
    daily senddigests cron job will be a seamless transition.

    Thanks!

    - jim -
  • Mark Sapiro at Mar 4, 2009 at 1:07 am

    Savoy, Jim wrote:
    Mark Sapiro wrote:
    In looking over my recursive listing closely, I see that there are 501
    total digest.mboxes out there (!). All of them are 30K or under.

    That still doesn't explain why they are all still there and nothing
    was
    delivered when I ran the cron job at 12:48 today...
    Are you sure digest_send_periodic is set to Yes for those lists that have
    old digest.mbox files?
    Absolutely. That is the default we set up and I just checked a bunch of
    lists to verify it.

    I did check out a couple of the lists that had outstanding digest.mboxes
    and found that they didn't have any subscribers with the digest option
    checked
    (as I said, very few of our users know about and use this feature) so
    perhaps
    nothing was delivered today because there was just nothing to be
    delivered.

    But, assuming this is a more or less standard Mailman (2.1.5), even if
    no one is subscribed to the digest, when cron/senddigests runs, the
    digests should still be prepared and the digest.mbox unlinked, even if
    there are ultimately no recipients. The only thing that affects this
    is if the list's digestable setting is No, and all this does is keep
    from creating the digest.mbox in the first place.

    Are there any Mailman error log entries from when senddigests ran? Did
    you run it from cron or by hand. If by hand, did you run it as
    'mailman' and did it produce any output? If from cron, and it produced
    any output, the output was probably mailed to 'mailman' which is
    probably the 'mailman' list, and where does it go from there?

    Note: it is a good idea to put a MAILTO= at the beginning of Mailman's
    crontab to direct any mailed output to a good place. There is normally
    only mailed output if something goes wrong.

    I take it a digest.mobx is created regardless of whether or not a list
    has
    any members using that option. So perhaps everything is fine and
    starting this
    daily senddigests cron job will be a seamless transition.

    Correct, but even if there are no digest members, the digest.mbox will
    be unlinked when cron/senddigests successfully runs.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 4, 2009 at 5:59 pm
    Jim Savoy wrote:
    I did check out a couple of the lists that had outstanding
    digest.mboxes and found that they didn't have any subscribers with the
    digest option checked.
    Mark Sapiro wrote:
    But, assuming this is a more or less standard Mailman (2.1.5), even if
    no one is subscribed to the digest, when cron/senddigests runs, the
    digests should still be prepared and the digest.mbox unlinked, even if
    there are ultimately no recipients. The only thing that affects this is
    if the list's digestable setting is No, and all this does is keep from
    creating the digest.mbox in the first place.
    All of our lists are digestable (that's the default). I did run one of
    your
    scripts once-upon-a-time to see how many digest members we had, and
    there
    were a few dozen.

    Unfortunately, I'm still in the state that the only time a digest goes
    out
    is when it exceeds 30K in size.

    Did you run it from cron or by hand. If by hand, did you run it as
    'mailman' and did it produce any output? If from cron, and it produced
    any
    output, the output was probably mailed to 'mailman' which is probably
    the
    'mailman' list, and where does it go from there?

    Ran it by cron. I don't see any activity at all when it runs (when I ran
    the checkdbs program, python lead the way when I did a "top", but
    nothing
    seems to happen when I run senddigests).

    There was also no output sent to the mailman mailing list (I am the
    owner
    and only subscriber on that list, and it accepts mail from anyone).

    Nothing in the Mailman logs either.
    Note: it is a good idea to put a MAILTO= at the beginning of Mailman's
    crontab to direct any mailed output to a good place. There is normally
    only mailed output if something goes wrong.

    Can you give me the exact syntax of that Mark? I am not sure what you
    mean
    or how I would do that. I did a man on "cron" but don't see anything
    there
    about MAILTO. Thanks.

    - jim -
  • Mark Sapiro at Mar 4, 2009 at 6:28 pm

    Savoy, Jim wrote:
    Ran it by cron. I don't see any activity at all when it runs (when I ran
    the checkdbs program, python lead the way when I did a "top", but
    nothing
    seems to happen when I run senddigests).

    Try

    su mailman
    cron/senddigests

    It probably won't be any different, but it's worth a try. If it does
    nothing, what is the contents of the cron/senddigests script. There
    appears to be something wrong with it. You could compare it to
    <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/annotate/419/cron/senddigests>
    to see if you have any modifications.

    There was also no output sent to the mailman mailing list (I am the
    owner
    and only subscriber on that list, and it accepts mail from anyone).

    Nothing in the Mailman logs either.
    Note: it is a good idea to put a MAILTO= at the beginning of Mailman's
    crontab to direct any mailed output to a good place. There is normally
    only mailed output if something goes wrong.

    Can you give me the exact syntax of that Mark? I am not sure what you
    mean
    or how I would do that. I did a man on "cron" but don't see anything
    there
    about MAILTO. Thanks.
    You can set environment variables in a crontab via

    VARIABLE = value

    (whitespace around "=" is optional). See "man 5 crontab". By default,
    cron mails any output to the 'owner' of the crontab, but if you set
    MAILTO, e.g.

    MAILTO=user at example.com

    or

    MAILTO=localuser

    any output will go there.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 4, 2009 at 6:46 pm

    Mark Sapiro wrote:
    Thanks for the MAILTO info.
    Try
    su mailman
    cron/senddigests

    Boom. That worked. All (but 3) of the 501 digest.mboxes are gone
    now and I got a delivery from the list I am a digest member of.
    It ran rather quickly too (1 minute flat).

    If/when we resolve this, I guess I'll have to start a new thread on
    why those 3 didn't go away. :-)

    So obviously, this has something to do with this being run from cron.
    You would think all of it would fail though (the checkdbs stuff runs
    fine from cron).
    It probably won't be any different, but it's worth a try. If it does
    nothing, what is the contents of the cron/senddigests script. There
    appears to be something wrong with it. You could compare it to
    <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/annotate/419/cr
    on/senddigests>
    to see if you have any modifications.

    These were identical.
  • Mark Sapiro at Mar 4, 2009 at 7:06 pm

    Savoy, Jim wrote:
    If/when we resolve this, I guess I'll have to start a new thread on
    why those 3 didn't go away. :-)

    Are they for lists that had a post arrive during or immediately after
    the time the digest was sent? In other words do they just contain
    messages waiting for the next digest?

    So obviously, this has something to do with this being run from cron.
    You would think all of it would fail though (the checkdbs stuff runs
    fine from cron).

    What does "crontab -u mailman -l" show?

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 4, 2009 at 7:24 pm
    Jim Savoy wrote:
    So obviously, this has something to do with this being run from cron.
    You would think all of it would fail though (the checkdbs stuff runs
    fine from cron).
    Hmmm - the above may not be true. Checkdbs did not run this morning.

    Mark Sapiro wrote:
    What does "crontab -u mailman -l" show?
    I think you nailed it. This is different from cron/crontab.in. The
    latest
    changes I made are not reflected. So am I supposed to do the:

    cd mailman/cron
    crontab -u mailman crontab.in

    every time I make a change to the crontab.in file? I believe that was
    the
    very first question I asked when I started this thread. :-)

    - jim -
  • Mark Sapiro at Mar 4, 2009 at 9:31 pm

    Savoy, Jim wrote:
    I think you nailed it. This is different from cron/crontab.in. The
    latest
    changes I made are not reflected. So am I supposed to do the:

    cd mailman/cron
    crontab -u mailman crontab.in

    every time I make a change to the crontab.in file? I believe that was
    the
    very first question I asked when I started this thread. :-)

    Yes, it was one of the first questions you asked, and it went
    unanswered, but now you know.

    Mailman's crontab is /var/spool/cron/mailman.

    crontab -u mailman crontab.in

    copies crontab.in to /var/spool/cron/mailman, but creates no other link
    between them.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 4, 2009 at 7:36 pm
    Savoy, Jim wrote:
    If/when we resolve this, I guess I'll have to start a new thread on
    why those 3 didn't go away. :-)
    Mark Sapiro wrote:
    Are they for lists that had a post arrive during or immediately after
    the time the digest was sent? In other words do they just contain
    messages waiting for the next digest?
    Two of them yes, but the other two no. I will deal with this later and
    Start a new thread if they don't go out tomorrow.
    What does "crontab -u mailman -l" show?
    I re-ran the "crontab -u mailman crontab.in" and my changes have been
    reflected. The old edit-the-source-code-and-forget-to-recompile problem!

    All is well now. Thanks for your patience.
  • Mark Sapiro at Mar 4, 2009 at 9:33 pm

    Savoy, Jim wrote:
    Mark Sapiro wrote:
    Are they for lists that had a post arrive during or immediately after
    the time the digest was sent? In other words do they just contain
    messages waiting for the next digest?
    Two of them yes, but the other two no. I will deal with this later and
    Start a new thread if they don't go out tomorrow.
    For the two old ones, be sure to check the list's digest_send_periodic
    setting.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 4, 2009 at 9:53 pm
    Savoy, Jim wrote:
    Two of them yes, but the other two no.
    Mark Sapiro wrote:
    For the two old ones, be sure to check the list's digest_send_periodic
    setting.

    Right. That was the difference. They are the only two that say NO to
    this query.

    Thanks for everything, Mark.
  • Savoy, Jim at Mar 4, 2009 at 7:06 pm

    Mark Sapiro wrote:
    Try
    su mailman
    cron/senddigests

    Since we are trying to troubleshoot a very mysterious problem, I should
    include all of the information I have. When I ran senddigest by hand (as
    user mailman), I did get a warning:

    [mailman cron]$ ./senddigests
    /mail/mailman/pythonlib/korean/c/euc_kr.py:24: RuntimeWarning: Python C
    API version mismatch for module _koco: This Python has API version 1012,
    module _koco has version 1011.
    import _koco


    It carried on anyway after that, but perhaps that is what causes cron to
    fail.
    As I said, when it is run by cron, I don't see anything when I do a
    "top" and
    there doesn't seem to by any activity at all, so perhaps it's instantly
    failing
    because of this Python koco warning. I didn't get anything mailed to me
    when I
    added the MAILTO though...

    - jim -
  • Tim Bell at Mar 4, 2009 at 7:15 pm

    Savoy, Jim wrote:
    I didn't get anything mailed to me when I added the MAILTO though...
    Divide and conquer. First get a dead simple cron job working, maybe one
    that runs every few minutes and does something like:

    date >> /tmp/foo-cron; echo "done"

    Once that is working and you are receiving cron notification emails and
    cron logs to look at, then work on getting a cron entry for the
    senddigests job running.

    HTH - Tim
  • Mark Sapiro at Mar 4, 2009 at 9:19 pm

    Savoy, Jim wrote:
    Since we are trying to troubleshoot a very mysterious problem, I should
    include all of the information I have. When I ran senddigest by hand (as
    user mailman), I did get a warning:

    [mailman cron]$ ./senddigests
    /mail/mailman/pythonlib/korean/c/euc_kr.py:24: RuntimeWarning: Python C
    API version mismatch for module _koco: This Python has API version 1012,
    module _koco has version 1011.
    import _koco

    That says Python has been updated since Mailman was last installed.

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan
  • Savoy, Jim at Mar 3, 2009 at 11:03 pm

    Mark Sapiro wrote:
    The digest messages are accumulated for a list in a mbox format file
    lists/LISTNAME/digest.mbox. When a new message arrives and is added to
    digest.mbox and the size of digest.mbox is now greater than the list's
    digest_size_threshold, a digest is sent at that time for that list,
    and the digest.mbox is removed.

    Hmmm. Removed huh? Maybe our digesting situation isn't so healthy
    afterall.
    I just did a recursive listing of the /lists directory and there are
    quite
    a few digest.mboxes in there! A lot of them have really old dates on
    them,
    and NONE of them is over 30K. So I think the problem we have here is
    that
    our digests ONLY go out when the digest.mbox exceeds 30K, and not
    "daily,
    regardless of size". Maybe that's where the senddigests cron job kicks
    in.

    That still doesn't explain why they are all still there and nothing was
    delivered when I ran the cron job at 12:48 today...
  • Mark Sapiro at Mar 3, 2009 at 11:09 pm

    Savoy, Jim wrote:
    Mark Sapiro wrote:
    The digest messages are accumulated for a list in a mbox format file
    lists/LISTNAME/digest.mbox. When a new message arrives and is added to
    digest.mbox and the size of digest.mbox is now greater than the list's
    digest_size_threshold, a digest is sent at that time for that list,
    and the digest.mbox is removed.

    Hmmm. Removed huh? Maybe our digesting situation isn't so healthy
    afterall.
    I just did a recursive listing of the /lists directory and there are
    quite
    a few digest.mboxes in there! A lot of them have really old dates on
    them,
    and NONE of them is over 30K. So I think the problem we have here is
    that
    our digests ONLY go out when the digest.mbox exceeds 30K, and not
    "daily,
    regardless of size". Maybe that's where the senddigests cron job kicks
    in.

    Right.

    That still doesn't explain why they are all still there and nothing was
    delivered when I ran the cron job at 12:48 today...

    Are you sure digest_send_periodic is set to Yes for those lists that
    have old digest.mbox files?

    --
    Mark Sapiro <mark at msapiro.net> The highway is for gamblers,
    San Francisco Bay Area, California better use your sense - B. Dylan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedMar 3, '09 at 8:20p
activeMar 4, '09 at 9:53p
posts20
users3
websitelist.org

People

Translate

site design / logo © 2022 Grokbase