FAQ
Hello,

Can you recommend a job queue system that works under Windows (and which works with Oracle if it needs a database)?

TheSchwartz can't be installed, TheSchwartz::Moosified doesn't work with Oracle and Gearman::Server can't be installed under Windows because of Danga::Socket.

Thanks.

--Octavian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110121/ac0b3dca/attachment.htm

Search Discussions

  • Jon Schutz at Jan 21, 2011 at 10:36 am

    On 01/21/2011 07:35 PM, Octavian Rasnita wrote:
    Hello,

    Can you recommend a job queue system that works under Windows (and
    which works with Oracle if it needs a database)?

    TheSchwartz can't be installed, TheSchwartz::Moosified doesn't work
    with Oracle and Gearman::Server can't be installed under Windows
    because of Danga::Socket.
    Have you considered Amazon SQS? (Amazon::SQS::Simple)

    Or depending on your requirements, it might be just as easy to build
    your own.

    --
    Jon Schutz
    http://notes.jschutz.net/

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110121/62c39a00/attachment.htm
  • Bill Moseley at Jan 21, 2011 at 2:49 pm
    2011/1/21 Octavian Rasnita <orasnita@gmail.com>
    Hello,

    Can you recommend a job queue system that works under Windows (and which
    works with Oracle if it needs a database)?
    Have you looked at AMQP implementations? RabbitMQ seems to have a Windows
    download. But normally you would run the queue on its on machine (or
    machines) so might as well use Linux for that. I suspect the Windows
    RabbitMQ is not as well used, but that's just a guess.

    http://www.rabbitmq.com/server.html

    http://qpid.apache.org/download.cgi

    It seems like there's some discussion around about if a database makes a
    good queue. Grated, on the RabbitMQ site, so with a bias, but here's one:
    http://www.rabbitmq.com/resources/RabbitMQ_Oxford_Geek_Night.pdf

    A bit dated, but this is often cited when someone is comparing options:

    http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes



    --
    Bill Moseley
    moseley@hank.org
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110121/d03d78e3/attachment.htm
  • Alexander Hartmaier at Jan 21, 2011 at 3:30 pm
    t0m created CatalystX::JobServer which is currently only available on
    github:
    https://github.com/bobtfish/CatalystX-JobServer

    --
    Best regards, Alex

    On Fri, 2011-01-21 at 15:49 +0100, Bill Moseley wrote:


    2011/1/21 Octavian Rasnita <orasnita@gmail.com>
    Hello,

    Can you recommend a job queue system that works under Windows
    (and which works with Oracle if it needs a database)?


    Have you looked at AMQP implementations? RabbitMQ seems to have a
    Windows download. But normally you would run the queue on its on
    machine (or machines) so might as well use Linux for that. I suspect
    the Windows RabbitMQ is not as well used, but that's just a guess.


    http://www.rabbitmq.com/server.html


    http://qpid.apache.org/download.cgi


    It seems like there's some discussion around about if a database makes
    a good queue. Grated, on the RabbitMQ site, so with a bias, but
    here's one:
    http://www.rabbitmq.com/resources/RabbitMQ_Oxford_Geek_Night.pdf


    A bit dated, but this is often cited when someone is comparing
    options:


    http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes






    --
    Bill Moseley
    moseley@hank.org

    *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
    T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
    Handelsgericht Wien, FN 79340b
    *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
    Notice: This e-mail contains information that is confidential and may be privileged.
    If you are not the intended recipient, please notify the sender and then
    delete this e-mail immediately.
    *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
  • Bill Moseley at Jan 21, 2011 at 8:13 pm

    On Fri, Jan 21, 2011 at 7:30 AM, Alexander Hartmaier wrote:

    t0m created CatalystX::JobServer which is currently only available on
    github:
    https://github.com/bobtfish/CatalystX-JobServer


    So, venturing a bit off topic, anyone have suggestions on managing worker
    ("consumer") processes?

    There seems to be two common approaches. One is to just start up a bunch of
    worker processes independently (managed like any other daemon) killing and
    starting more as needed. Another is using a manager process that forks off
    workers and will kill long-running or memory-consuming processes and respawn
    if a process dies. Perhaps monitoring queue sizes and adjusting as
    necessary.


    --
    Bill Moseley
    moseley@hank.org
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110121/06a055d4/attachment.htm
  • Cosimo Streppone at Jan 22, 2011 at 7:41 am
    In data 22 gennaio 2011 alle ore 07:13:49, Bill Moseley <moseley@hank.org>
    ha scritto:
    On Fri, Jan 21, 2011 at 7:30 AM, Alexander Hartmaier <
    alexander.hartmaier@t-systems.at> wrote:
    t0m created CatalystX::JobServer which is currently only available on
    github:
    https://github.com/bobtfish/CatalystX-JobServer
    So, venturing a bit off topic, anyone have suggestions on managing worker
    ("consumer") processes?

    There seems to be two common approaches. One is to just start up a bunch
    of worker processes independently (managed like any other daemon)
    killing and starting more as needed.
    For some time I used this approach, with a "fake" init
    script like '/etc/init.d/worker' that would start one of them,
    so I could run it multiple times to spawn more workers.

    A munin plugin monitored the number of active workers,
    and if it detected less than normal, it would spawn a new
    one.

    I thought that'd be neat, then I changed my mind.
    It worked just fine, but in the end I didn't like
    mixing munin plugins with application-related stuff.

    --
    Cosimo
  • Bill Moseley at Jan 22, 2011 at 4:01 pm

    On Fri, Jan 21, 2011 at 11:41 PM, Cosimo Streppone wrote:

    There seems to be two common approaches. One is to just start up a bunch of
    worker processes independently (managed like any other daemon) killing and
    starting more as needed.
    For some time I used this approach, with a "fake" init
    script like '/etc/init.d/worker' that would start one of them,
    so I could run it multiple times to spawn more workers.

    A munin plugin monitored the number of active workers,
    and if it detected less than normal, it would spawn a new
    one.

    I thought that'd be neat, then I changed my mind.
    It worked just fine, but in the end I didn't like
    mixing munin plugins with application-related stuff.
    What did you change to using?

    IMO, we (at work) have tended to over-engineer the workers. For example,
    for a given task we have servers that run via init.d scripts and they fork
    off a number of threads to handle connections and also fork off a bunch of
    worker threads that another manager thread allocates work to.

    It's a complex beast just to run some code. And as a beast if it breaks
    there's only one or two people that can fix. When the on-call tech gets a
    call at 2am it's unlikely they would know how to restart specifying more
    threads.

    The advantage I see in using separate scripts, like you describe above, is
    simplicity. If a human was alerted that a queue was getting large then it's
    just a matter of finding a machine that has headroom and start a few more
    workers. Any operator can do that. Plus, that's something that could be
    automated pretty easily with a shell script (and maybe even run via cron).

    A disadvantage is Perl is not the fastest in startup time (compared to
    forking, for example), but hopefully starting extra processes would be a
    rare situation when load changed dramatically.


    BTW -- Celery is worth a look: http://celeryq.org/





    --
    Bill Moseley
    moseley@hank.org
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110122/c094969d/attachment.htm
  • Tomas Doran at Jan 24, 2011 at 10:39 am

    On 21 Jan 2011, at 15:30, Alexander Hartmaier wrote:

    t0m created CatalystX::JobServer which is currently only available on
    This entirely requires RabbitMQ (well, any AMQP broker should be fine,
    but I've only tried Rabbit) to do the actual queueing.

    Cheers
    t0m
  • Alejandro Imass at Jan 24, 2011 at 10:49 am

    On Mon, Jan 24, 2011 at 5:40 AM, Tomas Doran wrote:
    On 21 Jan 2011, at 15:30, Alexander Hartmaier wrote:

    t0m created CatalystX::JobServer which is currently only available on
    This entirely requires RabbitMQ (well, any AMQP broker should be fine, but
    I've only tried Rabbit) to do the actual queueing.
    I would also recommend to try:

    http://search.cpan.org/~dsnopek/POE-Component-MessageQueue-0.2.10/lib/POE/Component/MessageQueue.pm
    Cheers
    t0m
  • Lars Dɪᴇᴄᴋᴏᴡ 迪拉斯 at Jan 21, 2011 at 8:41 pm
    http://kr.github.com/beanstalkd/
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 198 bytes
    Desc: This is a digitally signed message part.
    Url : http://lists.scsys.co.uk/pipermail/catalyst/attachments/20110121/28ffb6c2/attachment.pgp

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcatalyst @
categoriescatalyst, perl
postedJan 21, '11 at 9:05a
activeJan 24, '11 at 10:49a
posts10
users8
websitecatalystframework.org
irc#catalyst

People

Translate

site design / logo © 2022 Grokbase