Hello,

I am having an issue where shovel seems to be crashing my source rabbitmq
instance intermittently. This is on 2.1.0 version of rabbitmq shoveling to a
2 node cluster using the same version. The publish rate is probably
somewhere around 1 message/sec. The size of the message doesn?t seem to
matter.

Here are links to the shovel config and a log containing the crash:

Config: https://gist.github.com/feebbc2d6c63589ed5b0
Crash: https://gist.github.com/041f757a6e4ae1dd50b1

Thanks for your help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100929/cfdc8ba4/attachment.htm>

Search Discussions

  • Matthew Sackman at Sep 29, 2010 at 8:22 pm
    Hi Chris,
    On Wed, Sep 29, 2010 at 02:37:46PM -0500, Chris Murphy wrote:
    I am having an issue where shovel seems to be crashing my source rabbitmq
    instance intermittently. This is on 2.1.0 version of rabbitmq shoveling to a
    2 node cluster using the same version. The publish rate is probably
    somewhere around 1 message/sec. The size of the message doesn?t seem to
    matter.
    Now that is an excellent crash! It's actually tripping over in a
    function we wrote to assert various invariants within the queue.
    This could well be something we've been trying to track down for a long
    time. Is it happening repeatably for you? If so, is there any chance we
    can get access to your box and see it going on?

    Matthew
  • Chris Murphy at Sep 29, 2010 at 8:36 pm
    Matthew,

    I don't think I'd be able to give you access to these servers but I may be
    able to set it up on another server and see if I can get it to reproduce.

    Right now I am able to reproduce fairly consistently albeit not every single
    time. I've been trying to figure out what combination of events makes it
    crash but so far no luck.

    I'll see what I can do about getting this to a server you can access, in the
    meantime I can try to answer any other questions that might be helpful.

    Chris

    On 9/29/10 3:22 PM, "Matthew Sackman" wrote:

    Hi Chris,
    On Wed, Sep 29, 2010 at 02:37:46PM -0500, Chris Murphy wrote:
    I am having an issue where shovel seems to be crashing my source rabbitmq
    instance intermittently. This is on 2.1.0 version of rabbitmq shoveling to a
    2 node cluster using the same version. The publish rate is probably
    somewhere around 1 message/sec. The size of the message doesn?t seem to
    matter.
    Now that is an excellent crash! It's actually tripping over in a
    function we wrote to assert various invariants within the queue.
    This could well be something we've been trying to track down for a long
    time. Is it happening repeatably for you? If so, is there any chance we
    can get access to your box and see it going on?

    Matthew
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Matthew Sackman at Sep 29, 2010 at 8:47 pm
    Hi Chris,
    On Wed, Sep 29, 2010 at 02:37:46PM -0500, Chris Murphy wrote:
    I am having an issue where shovel seems to be crashing my source rabbitmq
    instance intermittently. This is on 2.1.0 version of rabbitmq shoveling to a
    2 node cluster using the same version. The publish rate is probably
    somewhere around 1 message/sec. The size of the message doesn?t seem to
    matter.
    Now that is an excellent crash!
    Is anyone issuing queue.purge commands to that queue? I think I've
    spotted the bug, and I can trip it, but it's a bug in queue.purge, so
    I'm wondering if you've issued any of those.

    Matthew
  • Chris Murphy at Sep 29, 2010 at 8:49 pm
    Yes, we are issuing purge commands right before we begin inserting to the
    queue!

    On 9/29/10 3:47 PM, "Matthew Sackman" wrote:

    Hi Chris,
    On Wed, Sep 29, 2010 at 02:37:46PM -0500, Chris Murphy wrote:
    I am having an issue where shovel seems to be crashing my source rabbitmq
    instance intermittently. This is on 2.1.0 version of rabbitmq shoveling to a
    2 node cluster using the same version. The publish rate is probably
    somewhere around 1 message/sec. The size of the message doesn?t seem to
    matter.
    Now that is an excellent crash!
    Is anyone issuing queue.purge commands to that queue? I think I've
    spotted the bug, and I can trip it, but it's a bug in queue.purge, so
    I'm wondering if you've issued any of those.

    Matthew
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Matthew Sackman at Sep 29, 2010 at 9:17 pm

    On Wed, Sep 29, 2010 at 03:49:25PM -0500, Chris Murphy wrote:
    Yes, we are issuing purge commands right before we begin inserting to the
    queue!
    Cool, well it's 10:15pm here and I'm not going to fix this right now,
    I'm afraid. There'll most likely be a fix for this tomorrow and it will
    be included in the next release. Many thanks for reporting this - we've
    been searching for the cause of _a_ bug in this area for a while. I'm
    not totally sure that the bug you've reported is exactly the same as the
    other bug we're seeing but they could very well be related.

    Best wishes,

    Matthew
  • Matthew Sackman at Sep 30, 2010 at 10:01 pm

    On Wed, Sep 29, 2010 at 10:17:02PM +0100, Matthew Sackman wrote:
    There'll most likely be a fix for this tomorrow
    There is indeed. It's been through QA and is on default, thus you'll
    need to compile from the head of our default branch to get it. It will
    be in the next release.

    Matthew

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedSep 29, '10 at 7:37p
activeSep 30, '10 at 10:01p
posts7
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Matthew Sackman: 4 posts Chris Murphy: 3 posts

People

Translate

site design / logo © 2022 Grokbase