FAQ
I am not sure whether this mail belongs to the users or developers list so
I am sending to both. Apologies.

I have a large 1.2 GB mbox file which i need to archive, the messages in
mbox typically have jpeg image attachments
whose approximate image sizes is 4 KB.
when i run arch script in /usr/lib/mailman/bin/ on my ubuntu 7.04 I get the
following error

*File "/usr/lib/python2.5/os.py", line 172, in makedirs mkdir(name, mode)
OSError: [Errno 31] Too many links:

*My basic objective is to archive 250GB of mails every month for two years .
The size of which would exceed 7 Tera Bytes approx. Would this be a feasible
option using mailman.(out of curiosity !!)

What has been the largest mail archive volume anyone has ever deployed into
mailman ? (out of curiosity !!)

Regards
-Alpesh

Search Discussions

  • Brad Knowles at Jul 17, 2007 at 9:45 pm

    On 7/18/07, alpesh gajbe wrote:

    I am not sure whether this mail belongs to the users or developers list so
    I am sending to both. Apologies.
    This sounds to me more like a question for -users.
    I have a large 1.2 GB mbox file which i need to archive, the messages in
    mbox typically have jpeg image attachments
    whose approximate image sizes is 4 KB.
    4KB JPEGs? That's pretty small for a JPEG. You have over 300,000 of
    these messages per mailbox file?
    when i run arch script in /usr/lib/mailman/bin/ on my ubuntu 7.04 I get the
    following error

    *File "/usr/lib/python2.5/os.py", line 172, in makedirs mkdir(name, mode)
    OSError: [Errno 31] Too many links:
    That sounds like a directory problem, and not a file problem. With
    as large a mailbox file as you're talking about, and as many messages
    as you're talking about, you probably need to break your archives
    more frequently than once a month. Try breaking them weekly instead.
    Alternatively, try changing your underlying filesystem to one that
    supports large numbers of files in a single directory, and preferably
    does so with an internal hashed directory/inode structure (e.g., XFS).

    Also keep in mind that you'll want to make sure that your OS is built
    to support large files (files over 2GB). Many Linux distributions
    are not built out-of-the-box to support large files.

    Finally, the version of Python that is recommended for use with the
    latest release version of Mailman, is Python 2.4.x for Mailman 2.1.9,
    for whatever the most recent version of Python 2.4.x (currently
    2.4.3, I believe). More recent versions of Python may or may not
    work with Mailman 2.1.9, and almost certainly will not work correctly
    with earlier versions of Mailman.
    *My basic objective is to archive 250GB of mails every month for two years .
    The size of which would exceed 7 Tera Bytes approx. Would this be a feasible
    option using mailman.(out of curiosity !!)
    That should be possible. We have 4GB worth of archives for
    python-list at python.org going back to 1999, and I'm pretty sure those
    messages are text-only or text+code fragments.
    What has been the largest mail archive volume anyone has ever deployed into
    mailman ? (out of curiosity !!)
    We've got some information in the FAQ Wizard about large mailing
    lists servers with regards to numbers of subscribers or numbers of
    messages, but I don't know that anyone has tried to gather any
    specific information with regards to large sizes of archives.

    The largest archives I am personally aware of are the ones we have
    for python-list, but I'd love to hear any information that anyone
    else has about any others.

    --
    Brad Knowles <brad at shub-internet.org>, Consultant & Author
    LinkedIn Profile: <http://tinyurl.com/y8kpxu>
    Slides from Invited Talks: <http://tinyurl.com/tj6q4>

    09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmailman-users @
categoriespython
postedJul 17, '07 at 7:15p
activeJul 17, '07 at 9:45p
posts2
users2
websitelist.org

2 users in discussion

Alpesh gajbe: 1 post Brad Knowles: 1 post

People

Translate

site design / logo © 2022 Grokbase