FAQ
Hello,

Still no success of fast processing the qfiles dir.

WITH DEFAULT MAILMAN VALUES at Default.py file

I did do the following :
Using SMTPDirect module to 127.0.0.1 port 25 (Normal Sendmail)
Using SMTPDirect module to 127.0.0.1 port 250 (Sendmail no DNS resolving
according the FAQ)
Using SMTPDirect module to 192.168.0.1 port 25 (Another SMTP server running
on windows 2000)
Using Sendmail module

All methods are the same results. qrunner is processing just 1 message at
the time and the next message at the qfiles dir is processed about 10
minutes later.

So qrunner is processing the qfiles dir for about 10 messages in 1 hour.
Because there are more than 10 messages comming in .. the qfiles dir is
growing and growing.

Now you may tell me what i now can try to make qrunner faster processing.

Mailman 2.0.10
Pentium 120, 96Mb ram
2x40gb HDD
Redhat 7.2

Greetings,
Danny Terweij

Search Discussions

  • Jon Carnes at Apr 28, 2002 at 10:33 pm
    Try creating a test list in your /etc/aliases on the box (non-local email
    addresses) and send to the list. Monitor the queue while you are doing it.
    Does this list move out quickly? How does the speed compare with qrunner
    handling a Mailman list?

    I'm guessing that either your sendmail is having some weird timout, or that
    you box is too heavily loaded with other processes...

    What does "top" give you when you look at it? How much processor is
    available? How much memory is being used? Is it swapping alot?

    How big is your Swap on your box? How much of your swap are you using while
    sending to a list?

    Are you running your box in Run Level 3 or in Run Level 5 (graphical user
    interface)? Please run it in 3 - your machine is fairly under powered for RH
    7.2 so it is not going to be able to handle Run Level 5 very quickly.

    Have you updated your kernel from the default install:
    up2date -f -i kernel, kernel-headers

    How much space do you have on your box - are any of the filesystems running
    low on space?
    df -h

    Of key importance for space is the /var area used to handle the mail queue.

    Since Mailman in general runs extremely fast - even on low processor boxes, I
    would focus on your system and its load to see if the bottle neck is there.

    Good Luck
    --- Original Message: Sunday 28 April 2002 04:57 pm ---
    Hello,

    Still no success of fast processing the qfiles dir.

    WITH DEFAULT MAILMAN VALUES at Default.py file

    I did do the following :
    Using SMTPDirect module to 127.0.0.1 port 25 (Normal Sendmail)
    Using SMTPDirect module to 127.0.0.1 port 250 (Sendmail no DNS resolving
    according the FAQ)
    Using SMTPDirect module to 192.168.0.1 port 25 (Another SMTP server running
    on windows 2000)
    Using Sendmail module

    All methods are the same results. qrunner is processing just 1 message at
    the time and the next message at the qfiles dir is processed about 10
    minutes later.

    So qrunner is processing the qfiles dir for about 10 messages in 1 hour.
    Because there are more than 10 messages comming in .. the qfiles dir is
    growing and growing.

    Now you may tell me what i now can try to make qrunner faster processing.

    Mailman 2.0.10
    Pentium 120, 96Mb ram
    2x40gb HDD
    Redhat 7.2

    Greetings,
    Danny Terweij





    ------------------------------------------------------
    Mailman-Users mailing list
    Mailman-Users at python.org
    http://mail.python.org/mailman/listinfo/mailman-users
    Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
  • Danny Terweij at Apr 29, 2002 at 12:39 am
    From: "Jon Carnes" <jonc at nc.rr.com>
    Try creating a test list in your /etc/aliases on the box (non-local email
    addresses) and send to the list. Monitor the queue while you are doing it.
    Does this list move out quickly? How does the speed compare with qrunner
    handling a Mailman list?
    I have a few test lists, local adresses and non-local adresses (separated).
    Both same results.

    When a external message is comming in, it is in just seconds transfered to
    the qfiles dir.
    That part of mailman is working speedy and fine.
    I'm guessing that either your sendmail is having some weird timout, or that
    you box is too heavily loaded with other processes...
    No heavy loads.
    What does "top" give you when you look at it? How much processor is
    available? How much memory is being used? Is it swapping alot?
    top not running qrunner:
    2:19am up 4:21, 1 user, load average: 1.17, 1.54, 1.58
    66 processes: 65 sleeping, 1 running, 0 zombie, 0 stopped
    CPU states: 1.9% user, 2.3% system, 0.0% nice, 95.7% idle
    Mem: 94124K av, 44044K used, 50080K free, 4K shrd, 2760K
    buff
    Swap: 192772K av, 31316K used, 161456K free 23684K
    cached

    top running qrunner:
    2:23am up 4:25, 1 user, load average: 1.46, 1.58, 1.58
    68 processes: 65 sleeping, 3 running, 0 zombie, 0 stopped
    CPU states: 97.5% user, 2.4% system, 0.0% nice, 0.0% idle
    Mem: 94124K av, 63608K used, 30516K free, 4K shrd, 4384K
    buff
    Swap: 192772K av, 31252K used, 161520K free 33332K
    cached

    PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
    8288 mailman 14 0 8388 8388 1388 R 92.7 8.9 1:56 python

    Using standard python 1.5.2 orso from RH 7.2 dist.
    Also python 2 installed here.. invoked as command python2 (for mm2.1 when it
    comes out :-) )
    How big is your Swap on your box? How much of your swap are you using while
    sending to a list?
    See above...
    Are you running your box in Run Level 3 or in Run Level 5 (graphical user
    interface)? Please run it in 3 - your machine is fairly under powered for RH
    7.2 so it is not going to be able to handle Run Level 5 very quickly.
    No GUI, i have an unsupported video card. X wont start. So i use level 3 all
    times and don't start X.
    (even no monitor attached to this box.. doing all remote)
    Have you updated your kernel from the default install:
    up2date -f -i kernel, kernel-headers
    I have 2.4.9-31, latest from up2date.
    Also using apt for latest releases.
    How much space do you have on your box - are any of the filesystems running
    low on space?
    df -h

    Filesystem Size Used Avail Use% Mounted on
    /dev/hdb1 38G 17G 19G 46% /
    /dev/hda1 49M 7.6M 38M 17% /boot
    none 46M 0 45M 0% /dev/shm
    /dev/hda2 455M 98M 334M 23% /prv
    /dev/hdc1 38G 14G 22G 38% /users
    Of key importance for space is the /var area used to handle the mail
    queue.

    /var sitting on / and 19gb free.
    Since Mailman in general runs extremely fast - even on low processor boxes, I
    would focus on your system and its load to see if the bottle neck is
    there.

    The heavy load is when qrunner is running. The linux box is not doing other
    heavy tasks.

    Running :
    mrtg
    squid
    mysql
    httpd
    dhcpd
    named
    webmin
    sendmail
    pop3
    samba

    All above is not used often, except mrtg every 5 mins and sendmail.
    Good Luck
    Maybe you see something wrong...

    Danny Terweij.

    --- Original Message: Sunday 28 April 2002 04:57 pm ---
    Hello,

    Still no success of fast processing the qfiles dir.

    WITH DEFAULT MAILMAN VALUES at Default.py file
    [cutted]
  • Joern Nettingsmeier at Apr 29, 2002 at 8:06 pm
    hi danny.

    i have no idea what's wrong, and i don't know python, but if such
    things happen to me, i try strace.
    my suggestion is disable the mailman cronjobs, invoke qrunner by
    hand under strace (strace <command>) and see if it hangs or loops
    somewhere. it will spew out a lot of garbage, but in some cases,
    there are useful hints for mere mortals in it.

    just a guess,

    j?rn


    Danny Terweij wrote:

    top not running qrunner:
    2:19am up 4:21, 1 user, load average: 1.17, 1.54, 1.58
    66 processes: 65 sleeping, 1 running, 0 zombie, 0 stopped
    CPU states: 1.9% user, 2.3% system, 0.0% nice, 95.7% idle
    Mem: 94124K av, 44044K used, 50080K free, 4K shrd, 2760K
    buff
    Swap: 192772K av, 31316K used, 161456K free 23684K
    cached

    top running qrunner:
    2:23am up 4:25, 1 user, load average: 1.46, 1.58, 1.58
    68 processes: 65 sleeping, 3 running, 0 zombie, 0 stopped
    CPU states: 97.5% user, 2.4% system, 0.0% nice, 0.0% idle
    Mem: 94124K av, 63608K used, 30516K free, 4K shrd, 4384K
    buff
    Swap: 192772K av, 31252K used, 161520K free 33332K
    cached

    PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
    8288 mailman 14 0 8388 8388 1388 R 92.7 8.9 1:56 python
    --
    Watch out where the huskies go and don't you eat
    the yellow snow !
    - Frank Zappa
  • Alan L. Waller at Apr 30, 2002 at 12:54 pm
    Set up a caching DNS server on the Mailman server, this is installed by
    default in RH7.2 just start it by

    /etc/rc.d/init.d/named start

    These are the final value I found via many hours of testing to get the max
    performance out of my server with 550 lists and about 400,000 users

    This is found in /your local dir/mailman/Mailman/Defaults.py

    #####
    # VARIABLES DEPENDING ON THE SIZE OF YOUR LISTS, THE PERFORMANCE OF YOUR
    # HARDWARE, NETWORK AND GENERAL MAIL HANDLING CAPABILITIES, ETC.


    # Set this to true to turn on MailList object lock debugging messages, which
    # will be written to logs/locks. If you think you're having lock problems, or
    # just want to tune the locks for your system, turn on lock debugging.
    LIST_LOCK_DEBUGGING = 1


    # This variable specifies how long the lock will be retained for a specific
    # operation on a mailing list. Watch your logs/lock file and if you see a lot
    # of lock breakages, you might need to bump this up. However if you set this
    # too high, a faulty script (or incorrect use of bin/withlist) can prevent the
    # list from being used until the lifetime expires. This is probably one of
    # the most crucial tuning variables in the system.
    LIST_LOCK_LIFETIME = hours(15)


    # This variable specifies how long an attempt will be made to acquire a list
    # lock by the qrunner process. If the lock acquisition times out, the message
    # will be re-queued for later delivery.
    LIST_LOCK_TIMEOUT = seconds(15)


    # cron/qrunner lock lifetime. This is probably the second most crucial tuning
    # variable in the system. See the notes for LIST_LOCK_LIFETIME above. Watch
    # your logs/smtp file and make sure that QRUNNER_LOCK_LIFETIME is set longer
    # than the longest period you see here. It is a bad thing if multiple
    # qrunners run at the same time.
    QRUNNER_LOCK_LIFETIME = hours(48)


    # Two other qrunner resource management variables. The first controls the
    # maximum lifetime of any single qrunner process, and the second controls the
    # maximum number of messages a single qrunner process will, er, process.
    # Exceeding either limit causes qrunner to exit, reclaiming system resources
    # and deleting the lock. Other qrunners will then process the remaining
    # messages. Set either to None to inhibit this resource check.
    QRUNNER_PROCESS_LIFETIME = hours(4)
    QRUNNER_MAX_MESSAGES = 150000

    Regards,

    Al

    At 10:57 PM 4/28/2002 +0200, Danny Terweij wrote:

    Hello,

    Still no success of fast processing the qfiles dir.

    WITH DEFAULT MAILMAN VALUES at Default.py file

    I did do the following :
    Using SMTPDirect module to 127.0.0.1 port 25 (Normal Sendmail)
    Using SMTPDirect module to 127.0.0.1 port 250 (Sendmail no DNS resolving
    according the FAQ)
    Using SMTPDirect module to 192.168.0.1 port 25 (Another SMTP server running
    on windows 2000)
    Using Sendmail module

    All methods are the same results. qrunner is processing just 1 message at
    the time and the next message at the qfiles dir is processed about 10
    minutes later.

    So qrunner is processing the qfiles dir for about 10 messages in 1 hour.
    Because there are more than 10 messages comming in .. the qfiles dir is
    growing and growing.

    Now you may tell me what i now can try to make qrunner faster processing.

    Mailman 2.0.10
    Pentium 120, 96Mb ram
    2x40gb HDD
    Redhat 7.2

    Greetings,
    Danny Terweij





    ------------------------------------------------------
    Mailman-Users mailing list
    Mailman-Users at python.org
    http://mail.python.org/mailman/listinfo/mailman-users
    Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedApr 28, '02 at 8:57p
activeApr 30, '02 at 12:54p
posts5
users4
websitelist.org

People

Translate

site design / logo © 2022 Grokbase