Grokbase › Groups › RabbitMQ › rabbitmq-discuss › February 2010
FAQ
Hello,



I am facing a strange issue with the Rabbitmq environment.

I try to run two simple processes written in Python (with amqplib): One
publisher and one subscriber.

The publisher is publishing a 10Meg data (not exactly 10Meg but
10,000,000 * 'A') and the subscriber is receiving it.

When all these three processes (including rabbitmq) are running under
the same OS (not especially the same computer but the same OS),
everything is working correctly but when one of them is running on a
different OS, it is lagging a lot.

Hereunder is a king of matrix giving you a rough idea of what is
happening:



- Publisher-OS: WIN ; RabbitMQ-OS: WIN ; Subscriber-OS: WIN =>
OK (less than a sec for the transfert)

- Publisher-OS: WIN ; RabbitMQ-OS: WIN ; Subscriber-OS: LIN =>
NOK (Subscriber is lagging)

- Publisher-OS: WIN ; RabbitMQ-OS: LIN ; Subscriber-OS: WIN =>
NOK (At least publisher is lagging - more than 4min to publish)

- Publisher-OS: WIN ; RabbitMQ-OS: LIN ; Subscriber-OS: LIN =>
NOK (Publisher is lagging - more than 4min to publish)

- Publisher-OS: LIN ; RabbitMQ-OS: WIN ; Subscriber-OS: WIN =>
NOK (Publisher is lagging - more than 4min to publish)

- Publisher-OS: LIN ; RabbitMQ-OS: LIN ; Subscriber-OS: LIN =>
OK (less than a sec for the transfert)

- ...



I am using RabbitMQ V1.7.0



Regards

Franck




######################################################################
The information contained in this communication is confidential and
may contain information that is privileged or exempt from disclosure
under applicable law. If you are not a named addressee, please notify
the sender immediately and delete this email from your system.
If you have received this communication, and are not a named
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited.
######################################################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100203/79678cec/attachment.htm

Search Discussions

  • Matthew Sackman at Feb 2, 2010 at 4:26 pm ⇧
    Hi Frank,

    Not really too sure what to suggest, but others on the list have
    reported better throughput when increasing buffers in Rabbit:

    Vladimir Balandin wrote:
    We have fixed the problem by set send/receive buffers on server side.

    -kernel inet_default_listen_options
    [{nodelay,true},{sndbuf,65535},{recbuf,65535}]
    The easiest way to set these is to edit rabbitmq-server(.bat) directly -
    these values are set near the top of rabbitmq-server and near the bottom
    of rabbitmq-server.bat.

    That may or may not solve these issues, but it's certainly worth trying.
    On Wed, Feb 03, 2010 at 12:09:13AM +0800, Boisson, Franck wrote:
    The publisher is publishing a 10Meg data (not exactly 10Meg but
    10,000,000 * 'A') and the subscriber is receiving it.
    If that's all one message, then the message must be fully received by
    the broker before it starts being streamed out to the client.

    Matthew
  • Matthew Sackman at Feb 2, 2010 at 4:48 pm ⇧
    Hi Frank,
    On Tue, Feb 02, 2010 at 04:26:58PM +0000, Matthew Sackman wrote:
    Not really too sure what to suggest, but others on the list have
    reported better throughput when increasing buffers in Rabbit
    Also, you're using the python client, to do performance testing, which
    is a little questionable - in our experience the python clients tend to
    perform badly. I'd suggest you test using our Java client which contains
    an example program called MulticastMain which is pretty flexible for
    testing performance and sustainable throughput.
    From the rabbitmq-java-client directory:
    ant clean dist
    cd build/dist
    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 1000000 -r 25 -a -z 20 -q 1

    where the -s parameter specifies the size, in bytes, the -r parameter
    specifies the sending rate, -a turns on auto-ack, which you may want for
    this, -z says how long to run the test for (in seconds) and -q sets the
    qos on the consumer to 1.

    This will create one producer and one consumer on the same box. You
    should vary the rate (-r) until the sending rates and receiving rates
    are balanced and sustainable (I tend to watch the reported latencies -
    if they keep increasing, then it's not balanced).

    Other if you just provide -? then it'll print out all the options. If
    you want to use different machines for producer and consumer then you
    can use -x 0 to turn off producers, and -y 0 to turn off consumers on
    each box respectively. If you do do that, then you'll also need to
    specify at least -e to name the exchange, and likely set -t to fanout
    otherwise it'll attempt to use a private queue of the default direct
    exchange and thus not route messages to the consumer. Also, start the
    consumer first.

    So, the consumer might look like this:

    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 10000000 -r 1 -a -z 20 -q 1 -x 0 -e my_exchage -t fanout

    whilst the producer looks like:

    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 10000000 -r 1 -a -z 20 -q 1 -y 0 -e my_exchage -t fanout

    These certainly work for me.

    Matthew
  • Pradeep Gatram at Feb 3, 2010 at 5:16 am ⇧
    Franck,

    Have you checked the timestamps of the involved machines? Sounds like a 4min
    time difference between the two clocks.

    Pradeep
    www.masplantiz.com
    On Tue, Feb 2, 2010 at 10:18 PM, Matthew Sackman wrote:

    Hi Frank,
    On Tue, Feb 02, 2010 at 04:26:58PM +0000, Matthew Sackman wrote:
    Not really too sure what to suggest, but others on the list have
    reported better throughput when increasing buffers in Rabbit
    Also, you're using the python client, to do performance testing, which
    is a little questionable - in our experience the python clients tend to
    perform badly. I'd suggest you test using our Java client which contains
    an example program called MulticastMain which is pretty flexible for
    testing performance and sustainable throughput.

    From the rabbitmq-java-client directory:
    ant clean dist
    cd build/dist
    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 1000000 -r 25 -a -z 20
    -q 1

    where the -s parameter specifies the size, in bytes, the -r parameter
    specifies the sending rate, -a turns on auto-ack, which you may want for
    this, -z says how long to run the test for (in seconds) and -q sets the
    qos on the consumer to 1.

    This will create one producer and one consumer on the same box. You
    should vary the rate (-r) until the sending rates and receiving rates
    are balanced and sustainable (I tend to watch the reported latencies -
    if they keep increasing, then it's not balanced).

    Other if you just provide -? then it'll print out all the options. If
    you want to use different machines for producer and consumer then you
    can use -x 0 to turn off producers, and -y 0 to turn off consumers on
    each box respectively. If you do do that, then you'll also need to
    specify at least -e to name the exchange, and likely set -t to fanout
    otherwise it'll attempt to use a private queue of the default direct
    exchange and thus not route messages to the consumer. Also, start the
    consumer first.

    So, the consumer might look like this:

    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 10000000 -r 1 -a -z 20
    -q 1 -x 0 -e my_exchage -t fanout

    whilst the producer looks like:

    sh runjava.sh com/rabbitmq/examples/MulticastMain -s 10000000 -r 1 -a -z 20
    -q 1 -y 0 -e my_exchage -t fanout

    These certainly work for me.

    Matthew

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100203/79ba2a19/attachment.htm

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedFeb 2, '10 at 4:09p
activeFeb 3, '10 at 5:16a
posts4
users3
websiterabbitmq.com
irc#rabbitmq

3 users in discussion

Matthew Sackman: 2 posts Boisson, Franck: 1 post Pradeep Gatram: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase