Hi,


One of the Puka users found that it is possible to
crash Puka. It happens as RabbitMQ seems to
keep on sending frames after connection_close_ok.


Puka has trouble behaving in such a case.


Is it a valid AMQP behaviour? Can anything be delivered
after connection_close_ok had been sent?


More comments:
https://github.com/majek/puka/issues/34


To reproduce run two instances of this code on a
SMP machine:
https://gist.github.com/4344132


Tested with RabbitMQ 2.7.1 and probably 2.8.6.


Cheers,
Marek

Search Discussions

  • Tim Watson at Dec 20, 2012 at 10:41 am
    Hi Marek! :)


    Can you actually see the cancel.ok frame arrive on the network after the close.ok frame? I'm not seeing how that can happen on the broker's side, but I'll continue staring at the code a while to make sure.


    Cheers,
    Tim


    On 20 Dec 2012, at 10:35, Marek Majkowski wrote:

    Hi,

    One of the Puka users found that it is possible to
    crash Puka. It happens as RabbitMQ seems to
    keep on sending frames after connection_close_ok.

    Puka has trouble behaving in such a case.

    Is it a valid AMQP behaviour? Can anything be delivered
    after connection_close_ok had been sent?

    More comments:
    https://github.com/majek/puka/issues/34

    To reproduce run two instances of this code on a
    SMP machine:
    https://gist.github.com/4344132

    Tested with RabbitMQ 2.7.1 and probably 2.8.6.

    Cheers,
    Marek
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Simon MacMullen at Dec 20, 2012 at 12:13 pm

    On 20/12/12 10:35, Marek Majkowski wrote:
    One of the Puka users found that it is possible to
    crash Puka. It happens as RabbitMQ seems to
    keep on sending frames after connection_close_ok.

    Puka has trouble behaving in such a case.

    Yes, I've managed to replicate this. However, I would point out that
    Puka's behaviour in this case is rather odd, so I think some fault lies
    with both sides. Certainly RabbitMQ should not let itself be provoked
    into doing this though, so I'll file a bug.


    Looking at the situation with Wireshark, we see the following sequence
    of events. Channel numbers in brackets.


    We have opened the connection and two channels, set confirm mode and
    published a message on (1) and declared a queue on (2). Then things get odd:


    Puka -> RabbitMQ (2) basic.qos
    Puka -> RabbitMQ (3) channel.open
    Puka -> RabbitMQ (0) connection.close


    Up until this point Puka is behaving oddly (not waiting for the qos-ok
    or open-ok before issuing connection.close) but I don't think it's
    violated the spec. But then:


    Puka <- RabbitMQ (2) basic.qos-ok
    Puka -> RabbitMQ (2) basic.consume


    Puka has now sent basic.consume after connection.close!


    Puka <- RabbitMQ (0) connection.close-ok
    Puka <- RabbitMQ (3) channel.open-ok


    And now RabbitMQ sends connection.close-ok followed by channel.open-ok.
    That channel.open-ok is... unhelpful at best.

    Is it a valid AMQP behaviour?

    Good question. The spec is quite vague on this - it only says "This
    method [connection.close-ok] confirms a Connection.Close method and
    tells the recipient that it is safe to release resources
    for the connection and close the socket." It does not say that nothing
    else should be sent!


    So I am not sure we are outside the *letter* of the spec. But this
    behaviour is clearly confusing so we will change it.


    But you should think about whether Puka should be sending normal methods
    after connection.close too :-)


    Cheers, Simon


    --
    Simon MacMullen
    RabbitMQ, VMware
  • Marek Majkowski at Dec 20, 2012 at 2:07 pm

    On Thu, Dec 20, 2012 at 12:13 PM, Simon MacMullen wrote:
    Puka -> RabbitMQ (2) basic.qos
    Puka -> RabbitMQ (3) channel.open
    Puka -> RabbitMQ (0) connection.close

    Up until this point Puka is behaving oddly (not waiting for the qos-ok or
    open-ok before issuing connection.close) but I don't think it's violated the
    spec. But then:

    Puka <- RabbitMQ (2) basic.qos-ok
    Puka -> RabbitMQ (2) basic.consume

    Puka has now sent basic.consume after connection.close!

    Puka <- RabbitMQ (0) connection.close-ok
    Puka <- RabbitMQ (3) channel.open-ok

    Simon, thanks for the great investigation!


    Internally for puka everything is async, everything continues unchanged
    until connection.close.ok is received. Only then puka cleans up.


    Obviously issuing basic.consume after sending connection.close
    makes little sense from protocol perspective, but it's completely
    valid from puka point of view - user just issued connection.close
    and did some further actions before waiting for the result.


    I think my quick workaround will be to just ignore everything rabbit
    sends after receiving connection.close.ok.


    Thanks for fixing that on the server side :)


    Cheers,
    Marek
  • Simon MacMullen at Dec 20, 2012 at 2:12 pm

    On 20/12/12 14:07, Marek Majkowski wrote:
    I think my quick workaround will be to just ignore everything rabbit
    sends after receiving connection.close.ok.

    Matthias has pointed out that the spec also says "After sending this
    method, [connection.close] any received methods except Close and
    Close?OK MUST be discarded. The response to receiving a Close after
    sending Close must be to send Close?Ok."


    So really you should start ignore everything a bit earlier: after
    sending connection.close.


    So strictly we are within the spec (we could just start firing random
    frames at you after that point) - but it's not helpful so we'll still
    fix it.


    Cheers, Simon


    --
    Simon MacMullen
    RabbitMQ, VMware

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedDec 20, '12 at 10:35a
activeDec 20, '12 at 2:12p
posts5
users3
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2017 Grokbase