Hi,

We are testing the new persister (bug21637). It hangs for the following test case:

1) create 5 persistent queues, each queue bind to a different topic.
2) without consumer running, publish 20000 messages to each queue.
each message has size 20KB.
the list_queues command shows each queue has 20000 messages in it.
3) start 5 consumers. each consumer consumes messages from one queue.
after consuming some messages, broker hangs (not responding to list_queues).


this bug happens when publisher does not use tx_commit.
if publisher calls tx_commit for each message, then there is no problem.
if publisher calls tx_select, then basic_publish 100 messages, then tx_commit, the broker may also hang.

Is this a known problem or i am missing something?
thanks.

-alex

Search Discussions

  • Matthew Sackman at Feb 9, 2010 at 3:27 pm
    Hi Alex,
    On Mon, Feb 08, 2010 at 07:30:52PM -0800, alex chen wrote:
    We are testing the new persister (bug21637). It hangs for the following test case:
    1) create 5 persistent queues, each queue bind to a different topic.
    2) without consumer running, publish 20000 messages to each queue.
    each message has size 20KB.
    the list_queues command shows each queue has 20000 messages in it.
    3) start 5 consumers. each consumer consumes messages from one queue.
    after consuming some messages, broker hangs (not responding to list_queues).
    I'm just trying to replicate this here. Are you publishing persistent or
    non-persistent msgs?

    Cheers,

    Matthew
  • Matthew Sackman at Feb 9, 2010 at 3:34 pm

    On Tue, Feb 09, 2010 at 03:27:01PM +0000, Matthew Sackman wrote:
    Hi Alex,
    On Mon, Feb 08, 2010 at 07:30:52PM -0800, alex chen wrote:
    We are testing the new persister (bug21637). It hangs for the following test case:
    1) create 5 persistent queues, each queue bind to a different topic.
    2) without consumer running, publish 20000 messages to each queue.
    each message has size 20KB.
    the list_queues command shows each queue has 20000 messages in it.
    3) start 5 consumers. each consumer consumes messages from one queue.
    after consuming some messages, broker hangs (not responding to list_queues).
    I'm just trying to replicate this here. Are you publishing persistent or
    non-persistent msgs?
    Also, are your consumers acknowledging messages as they're going, or do
    you have noAck/autoAck set, or are you building up lots of unackd msgs
    in the server?

    Matthew
  • Matthew Sackman at Feb 9, 2010 at 3:53 pm

    On Tue, Feb 09, 2010 at 03:34:55PM +0000, Matthew Sackman wrote:
    I'm just trying to replicate this here.
    And indeed I can. Non persistent msgs and ack as we go. Working on it
    now.
  • Alex chen at Feb 9, 2010 at 6:31 pm
    Thanks Matthew for the quick response! It is greatly appreciated.
    We are using persistent msgs and ack. I also tried using basic_qos to set prefetch_count=1, but the broker still hanged. If we call tx_commit for each basic_publish which loading up the queue, then consumers can finish consuming all messages without any problem.
    thanks.

    -alex



    On Tue, Feb 09, 2010 at 03:34:55PM +0000, Matthew Sackman wrote:
    I'm just trying to replicate this here.
    And indeed I can. Non persistent msgs and ack as we go. Working on it
    now.

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Matthew Sackman at Feb 9, 2010 at 7:04 pm
    Hi Alex,
    On Tue, Feb 09, 2010 at 10:31:57AM -0800, alex chen wrote:
    Thanks Matthew for the quick response! It is greatly appreciated.
    We are using persistent msgs and ack. I also tried using basic_qos to set prefetch_count=1, but the broker still hanged. If we call tx_commit for each basic_publish which loading up the queue, then consumers can finish consuming all messages without any problem.
    I've tracked this down and fixed it. Many thanks for reporting this - it
    was nothing suspicious - just my own fault for mixing up atomic and
    non-atomic operations.

    Please pull and hg up -C bug21673 and let me know if it works for you.

    Matthew
  • Alex chen at Feb 9, 2010 at 7:53 pm
    Hi Matthew,

    I just tested it using the latest code and it works! Thanks a lot for fixing it so fast!
    Finally your work has made it possible to remove the memory limit of the previous persister.
    Go Rabbit! :)

    -alex



    ----- Original Message ----
    From: Matthew Sackman <matthew at lshift.net>
    To: alex chen <chen650 at yahoo.com>
    Cc: rabbitmq-discuss at lists.rabbitmq.com
    Sent: Tue, February 9, 2010 11:04:19 AM
    Subject: Re: [rabbitmq-discuss] new persiter hangs while consuming messages from multiple queues

    Hi Alex,
    On Tue, Feb 09, 2010 at 10:31:57AM -0800, alex chen wrote:
    Thanks Matthew for the quick response! It is greatly appreciated.
    We are using persistent msgs and ack. I also tried using basic_qos to set prefetch_count=1, but the broker still hanged. If we call tx_commit for each basic_publish which loading up the queue, then consumers can finish consuming all messages without any problem.
    I've tracked this down and fixed it. Many thanks for reporting this - it
    was nothing suspicious - just my own fault for mixing up atomic and
    non-atomic operations.

    Please pull and hg up -C bug21673 and let me know if it works for you.

    Matthew
  • Matthew Sackman at Feb 9, 2010 at 10:14 pm
    Hi Scott,
    On Tue, Feb 09, 2010 at 01:27:12PM -0800, scott w wrote:
    Is there an easy way to backport bug21673 to 1.7.1 or do I have to go
    against trunk to use it? And if so, is trunk stable right now?
    bug21673 contains everything that you need - every few days I merge from
    default into bug21673 so yes, it keeps up with all the developments and
    fixes going on in default. Default is very stable just at the moment.

    So in short, you shouldn't need to do any merging yourself - you should
    find that running straight off bug21673 is stable, and if you find bugs,
    please do let us know - we're normally pretty quick at tracking them
    down and fixing them.

    Matthew
  • Scott w at Feb 9, 2010 at 10:35 pm
    Great, thanks!
    On Tue, Feb 9, 2010 at 2:14 PM, Matthew Sackman wrote:

    Hi Scott,
    On Tue, Feb 09, 2010 at 01:27:12PM -0800, scott w wrote:
    Is there an easy way to backport bug21673 to 1.7.1 or do I have to go
    against trunk to use it? And if so, is trunk stable right now?
    bug21673 contains everything that you need - every few days I merge from
    default into bug21673 so yes, it keeps up with all the developments and
    fixes going on in default. Default is very stable just at the moment.

    So in short, you shouldn't need to do any merging yourself - you should
    find that running straight off bug21673 is stable, and if you find bugs,
    please do let us know - we're normally pretty quick at tracking them
    down and fixing them.

    Matthew
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100209/2f142d9d/attachment.htm

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedFeb 9, '10 at 3:30a
activeFeb 9, '10 at 10:35p
posts9
users3
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase