I hit on a scenario where connection.createChannel() may hang forever. The
scenario: obtain a channel and attempt to publish to a non-existent
exchange. The channel will be shutdown. Attempt to establish a new
(replacement) channel via Connection.createChannel(), which hangs forever.
A simple version of this scenario:


public Channel channelFor(final String tenantId) throws Exception {
   Channel channel = connection.createChannel();
   channel.addShutdownListener(new ShutdownListener() {
     public void shutdownCompleted(ShutdownSignalException e) {
       if (!e.isInitiatedByApplication())
         channelFor(tenantId);
     }
   });


   return channel;
}


Channel channel = channelFor("foo");
channel.basicPublish(someNonExistentExchange, ?

Search Discussions

  • Tim Watson at Jun 7, 2013 at 8:38 am
    Hi Jonathan,

    On 6 Jun 2013, at 23:42, Jonathan Halterman wrote:
    I hit on a scenario where connection.createChannel() may hang forever. The scenario: obtain a channel and attempt to publish to a non-existent exchange. The channel will be shutdown. Attempt to establish a new (replacement) channel via Connection.createChannel(), which hangs forever. A simple version of this scenario:

    This should not be happening. Are you absolutely sure you're using version 3.1.1 of the java client library, since a similar bug (involving accidentally re-used channel ids) was fixed in a previous release: http://hg.rabbitmq.com/rabbitmq-java-client/rev/ea7d68c2154d

    From Tracer:

    1370554962786: <Tracer-0> ch#1 <- {#method<channel.close>(reply-code@4, reply-text=NOT_FOUND - no exchange 'foo' in vhost '/', class-id`, method-id@), null, ""}
    1370554962788: <Tracer-0> ch#1 -> {#method<channel.close-ok>(), null, ""}
    1370554974891: <Tracer-0> ch#1 -> {#method<channel.open>(out-of-band=), null, ""}
    1370554974970: <Tracer-0> ch#1 <- {#method<channel.open-ok>(channel-id=), null, ""}

    That last line seems very odd - I would've expected to a see a channel id in there, though truth be told I'm not overly familiar with the Tracer tool in the java client.

    Any ideas why things are hanging?

    According to the last line of the trace you've supplied, a new channel was opened successfully, though there's no channel id present which is worrying. Please confirm that you are definitely using the right version (i.e., where did you obtain the amqp-client binary from) and we'll take a closer look.


    Cheers,
    Tim
  • Tim Watson at Jun 7, 2013 at 8:57 am
    Hi Jonathan,


    Ok, you can safely ignore what I've written below. What's actually going on (thanks to Matthias for pointing it out) is that the ShutdownListener callbacks run in the connection's thread, so the call to createChannel causes a deadlock since it blocks waiting for a response (whilst the connection's thread is stuck executing the listener). I'm going to add some notes to the API documentation explaining that, so future users don't get caught out in the same way.


    The canonical approach to dealing with this is to have the shutdown listener callback signal your own (application) thread, from which you can manage recovery without risking deadlock in this fashion.


    Cheers,
    Tim


    On 7 Jun 2013, at 09:38, Tim Watson wrote:

    Hi Jonathan,
    On 6 Jun 2013, at 23:42, Jonathan Halterman wrote:
    I hit on a scenario where connection.createChannel() may hang forever. The scenario: obtain a channel and attempt to publish to a non-existent exchange. The channel will be shutdown. Attempt to establish a new (replacement) channel via Connection.createChannel(), which hangs forever. A simple version of this scenario:
    This should not be happening. Are you absolutely sure you're using version 3.1.1 of the java client library, since a similar bug (involving accidentally re-used channel ids) was fixed in a previous release: http://hg.rabbitmq.com/rabbitmq-java-client/rev/ea7d68c2154d
    From Tracer:

    1370554962786: <Tracer-0> ch#1 <- {#method<channel.close>(reply-code@4, reply-text=NOT_FOUND - no exchange 'foo' in vhost '/', class-id`, method-id@), null, ""}
    1370554962788: <Tracer-0> ch#1 -> {#method<channel.close-ok>(), null, ""}
    1370554974891: <Tracer-0> ch#1 -> {#method<channel.open>(out-of-band=), null, ""}
    1370554974970: <Tracer-0> ch#1 <- {#method<channel.open-ok>(channel-id=), null, ""}
    That last line seems very odd - I would've expected to a see a channel id in there, though truth be told I'm not overly familiar with the Tracer tool in the java client.
    Any ideas why things are hanging?
    According to the last line of the trace you've supplied, a new channel was opened successfully, though there's no channel id present which is worrying. Please confirm that you are definitely using the right version (i.e., where did you obtain the amqp-client binary from) and we'll take a closer look.

    Cheers,
    Tim

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Jonathan Halterman at Jun 7, 2013 at 3:47 pm
    Hey Tim,


    Sounds good. Thanks for the info.


    - jonathan


    On Fri, Jun 7, 2013 at 1:57 AM, Tim Watson wrote:

    Hi Jonathan,

    Ok, you can safely ignore what I've written below. What's actually going
    on (thanks to Matthias for pointing it out) is that the ShutdownListener
    callbacks run in the connection's thread, so the call to createChannel
    causes a deadlock since it blocks waiting for a response (whilst the
    connection's thread is stuck executing the listener). I'm going to add some
    notes to the API documentation explaining that, so future users don't get
    caught out in the same way.

    The canonical approach to dealing with this is to have the shutdown
    listener callback signal your own (application) thread, from which you can
    manage recovery without risking deadlock in this fashion.

    Cheers,
    Tim
    On 7 Jun 2013, at 09:38, Tim Watson wrote:

    Hi Jonathan,
    On 6 Jun 2013, at 23:42, Jonathan Halterman wrote:
    I hit on a scenario where connection.createChannel() may hang forever.
    The scenario: obtain a channel and attempt to publish to a non-existent
    exchange. The channel will be shutdown. Attempt to establish a new
    (replacement) channel via Connection.createChannel(), which hangs forever.
    A simple version of this scenario:
    This should not be happening. Are you absolutely sure you're using
    version 3.1.1 of the java client library, since a similar bug (involving
    accidentally re-used channel ids) was fixed in a previous release:
    http://hg.rabbitmq.com/rabbitmq-java-client/rev/ea7d68c2154d
    From Tracer:

    1370554962786: <Tracer-0> ch#1 <-
    {#method<channel.close>(reply-code@4, reply-text=NOT_FOUND - no exchange
    'foo' in vhost '/', class-id`, method-id@), null, ""}
    1370554962788: <Tracer-0> ch#1 -> {#method<channel.close-ok>(), null,
    ""}
    1370554974891: <Tracer-0> ch#1 -> {#method<channel.open>(out-of-band=),
    null, ""}
    1370554974970: <Tracer-0> ch#1 <-
    {#method<channel.open-ok>(channel-id=), null, ""}
    That last line seems very odd - I would've expected to a see a channel
    id in there, though truth be told I'm not overly familiar with the Tracer
    tool in the java client.
    Any ideas why things are hanging?
    According to the last line of the trace you've supplied, a new channel
    was opened successfully, though there's no channel id present which is
    worrying. Please confirm that you are definitely using the right version
    (i.e., where did you obtain the amqp-client binary from) and we'll take a
    closer look.
    Cheers,
    Tim

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130607/3ce45d8b/attachment.htm>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJun 6, '13 at 10:42p
activeJun 7, '13 at 3:47p
posts4
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Jonathan Halterman: 2 posts Tim Watson: 2 posts

People

Translate

site design / logo © 2017 Grokbase