We're seeing a bunch of failure messages from our Ruby clients to RabbitMQ
3.0.2. From the client side, we see 541 "Internal Error"


On the server side logs, we see a bunch of errors. Normally I can at least
hazard a guess what's wrong, but I have no idea with this:




=ERROR REPORT==== 16-Jul-2013::22:43:15 ===
** Generic server <0.4358.0> terminating
** Last message in was {can_send,<0.4355.0>,true}
** When Server state == {lim,1,<0.4359.0>,false,[],1}
** Reason for termination ==
** {function_clause,[{orddict,update,


[<0.4355.0>,#Fun<rabbit_limiter.3.113433752>,[]],
                               [{file,"orddict.erl"},{line,153}]},
                      {rabbit_limiter,limit_queue,2,[]},
                      {rabbit_limiter,handle_call,3,[]},
                      {gen_server2,handle_msg,2,[]},
                      {proc_lib,init_p_do_apply,3,
                                [{file,"proc_lib.erl"},{line,227}]}]}






Or like this:
--------------------




=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4563.0> terminating
** Last message in was {'$gen_cast',
                            {method,
                                {'basic.consume',0,<<"web-tasks-mpietrek">>,


<<"web-tasks-mpietrek-1374014655000-264079359112">>,
                                    false,false,false,false,[]},
                                none,noflow}}
** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,41,<0.4540.0>,
                          <0.4560.0>,<0.4540.0>,
                          <<"192.168.78.13:57671 -> 192.168.78.50:5672">>,
                          {lstate,<0.4561.0>,true,false},
                          none,2,
                          {[],[]},
                          {user,<<"guest">>,
                           [administrator],
                           rabbit_auth_backend_internal,
                           {internal_user,<<"guest">>,


<<78,56,17,4,226,151,136,14,156,84,34,25,101,29,104,
                              78,125,130,202,207>>,
                            [administrator]}},
                          <<"/">>,<<"web-tasks-mpietrek">>,
                          {dict,1,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                           {{[],[],[],[],[],[],[],
                             [[<0.4558.0>|
                               {resource,<<"/">>,queue,
                                <<"web-tasks-mpietrek">>}]],
                             [],[],[],[],[],[],[],[]}}},
                          {dict,3,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                           {{[],[],[],[],[],[],[],
                             [[<0.4558.0>|#Ref<0.0.0.83887>]],
                             [],[],[],[],
                             [[<2783.2507.0>|#Ref<0.0.0.83888>]],
                             [[<2782.2563.0>|#Ref<0.0.0.83889>]],
                             [],[]}}},
                          {dict,0,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},


{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
                          {set,0,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},


{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
                          {dict,0,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},


{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
                          {set,1,16,16,8,80,48,
                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                           {{[],[],[],[],[],[],[],
                             [<0.4558.0>],
                             [],[],[],[],[],[],[],[]}}},
                          <0.4539.0>,
                          {state,fine,5000,#Ref<0.0.0.86733>},
                          false,1,
                          {{0,nil},{0,nil}},
                          [],
                          [{<<"consumer_cancel_notify">>,bool,true}],
                          none}


Or like this:
--------------------


=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4561.0> terminating
** Last message in was {can_send,<0.4558.0>,true}
** When Server state == {lim,1,<0.4563.0>,false,[],1}
** Reason for termination ==
** {function_clause,[{orddict,update,


[<0.4558.0>,#Fun<rabbit_limiter.3.113433752>,[]],
                               [{file,"orddict.erl"},{line,153}]},
                      {rabbit_limiter,limit_queue,2,[]},
                      {rabbit_limiter,handle_call,3,[]},
                      {gen_server2,handle_msg,2,[]},
                      {proc_lib,init_p_do_apply,3,
                                [{file,"proc_lib.erl"},{line,227}]}]}


=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4558.0> terminating
** Last message in was {basic_consume,false,<0.4563.0>,<0.4561.0>,true,


<<"web-tasks-mpietrek-1374014655000-264079359112">>,
                            false,none,
                            {'basic.consume_ok',


<<"web-tasks-mpietrek-1374014655000-264079359112">>}}
** When Server state == {q,
                          {amqqueue,
                           {resource,<<"/">>,queue,<<"web-tasks-mpietrek">>},
                           true,false,none,[],<0.4558.0>,[],[],
                           [{vhost,<<"/">>},
                            {name,<<"ha-web-tasks-mpietrek">>},
                            {pattern,<<"web-tasks-mpietrek">>},
                            {definition,[{<<"ha-mode">>,<<"all">>}]},
                            {priority,0}],
                           []},
                          none,true,rabbit_mirror_queue_master,
                          {state,
                           {resource,<<"/">>,queue,<<"web-tasks-mpietrek">>},
                           <0.4566.0>,<0.4565.0>,rabbit_variable_queue,
                           {vqstate,
                            {0,{[],[]}},
                            {0,{[],[]}},
                            {delta,undefined,0,undefined},
                            {0,{[],[]}},
                            {13,
                             {[{msg_status,12,


<<114,86,130,79,196,29,163,190,216,213,230,38,
                                  174,139,39,60>>,
                                {basic_message,
                                 {resource,<<"/">>,exchange,<<"skytap">>},
                                 [<<"web-tasks-mpietrek.mpietrek.default">>],
                                 {content,60,
                                  {'P_basic',<<"application/octet-stream">>,


undefined,undefined,2,0,undefined,undefined,
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,undefined},


<<152,0,24,97,112,112,108,105,99,97,116,105,
                                    111,110,47,111,99,116,101,116,45,115,116,
                                    114,101,97,109,2,0>>,
                                  rabbit_framing_amqp_0_9_1,


[<<"{\"payload\":{\"object\":\"BAhvOh1EZWxldGVFeHBpcmVkVXBsb2Fkc1Rhc2sGOhJAZ3JhY2VfcGVyaW9k\\nbzocQWN0aXZlU3VwcG9ydDo6RHVyYXRpb24HOgtAdmFsdWVpAwB1EjoLQHBh\\ncnRzWwZbBzoJZGF5c2kT\\n\"},\"_serialization_type\":\"marshal\",\"_timestamp\":\"2013-07-16T22:43:55Z\",\"_message_type\":\"request\",\"_unique_id\":\"app1:req:wfe-em-init.80690eb0d0940130f02f000c291bd1d8.2.372:336\",\"caller_context\":\"wfe-em-init.80690eb0d0940130f02f000c291bd1d8.2.373\",\"action\":\"async-task\",\"_mq_protocol_version\":1}">>]},


<<114,86,130,79,196,29,163,190,216,213,230,38,
                                   174,139,39,60>>,
                                 true},
                                true,false,true,true,
                                {message_properties,undefined,false}},
                               {msg_status,11,


<<12,29,215,111,220,221,165,115,106,150,132,130,
                                  118,94,193,26>>,
                                {basic_message,
                                 {resource,<<"/">>,exchange,<<"skytap">>},
                                 [<<"web-tasks-mpietrek.mpietrek.default">>],
                                 {content,60,
                                  {'P_basic',<<"application/octet-stream">>,


undefined,undefined,2,0,undefined,undefined,
                                   undefined,undefined,undefined,undefined,
                                   undefined,undefined,undefined},


<<152,0,24,97,112,112,108,105,99,97,116,105,
                                    111,110,47,111,99,116,101,116,45,115,116,
                                    114,101,97,109,2,0>>,
                                  rabbit_framing_amqp_0_9_1,


<rest of error omitted>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130716/7f373738/attachment.htm>

Search Discussions

  • Matthias Radestock at Jul 17, 2013 at 2:42 am
    Matt,

    On 17/07/13 00:26, Matt Pietrek wrote:
    We're seeing a bunch of failure messages from our Ruby clients to
    RabbitMQ 3.0.2. From the client side, we see 541 "Internal Error"

    On the server side logs, we see a bunch of errors. Normally I can at
    least hazard a guess what's wrong, but I have no idea with this:


    =ERROR REPORT==== 16-Jul-2013::22:43:15 ===
    ** Generic server <0.4358.0> terminating
    ** Last message in was {can_send,<0.4355.0>,true}
    ** When Server state == {lim,1,<0.4359.0>,false,[],1}
    ** Reason for termination ==
    ** {function_clause,[{orddict,update,

    [<0.4355.0>,#Fun<rabbit_limiter.3.113433752>,[]],
    [{file,"orddict.erl"},{line,153}]},
    {rabbit_limiter,limit_queue,2,[]},
    {rabbit_limiter,handle_call,3,[]},
    {gen_server2,handle_msg,2,[]},
    {proc_lib,init_p_do_apply,3,
    [{file,"proc_lib.erl"},{line,227}]}]}

    I think this is a duplicate of the problem reported at
    http://rabbitmq.1065348.n5.nabble.com/Rabbit-3-1-1-crash-question-td27540.html,
    the fix for which will appear in the next bugfix release.


    However, we were quite that the bug was only introduced in 3.1.0, so
    it's surprising to see this in 3.0.2. The code has certainly changed a
    lot on that area, so I'd advise upgrading in any case.


    Regards,


    Matthias.
  • Matt Pietrek at Jul 17, 2013 at 5:34 pm
    Thanks for quick response Matthias.


    And just to ease any concern, I misspoke earlier. Almost all of our
    clusters are on 3.0.2. However I personally run a newer version for testing
    and had forgotten that fact.


    Matt




    On Tue, Jul 16, 2013 at 7:42 PM, Matthias Radestock
    wrote:

    Matt,

    On 17/07/13 00:26, Matt Pietrek wrote:

    We're seeing a bunch of failure messages from our Ruby clients to
    RabbitMQ 3.0.2. From the client side, we see 541 "Internal Error"

    On the server side logs, we see a bunch of errors. Normally I can at
    least hazard a guess what's wrong, but I have no idea with this:


    =ERROR REPORT==== 16-Jul-2013::22:43:15 ===
    ** Generic server <0.4358.0> terminating
    ** Last message in was {can_send,<0.4355.0>,true}
    ** When Server state == {lim,1,<0.4359.0>,false,[],1}
    ** Reason for termination ==
    ** {function_clause,[{orddict,**update,

    [<0.4355.0>,#Fun<rabbit_**limiter.3.113433752>,[]],
    [{file,"orddict.erl"},{line,**153}]},
    {rabbit_limiter,limit_queue,2,**[]},
    {rabbit_limiter,handle_call,3,**[]},
    {gen_server2,handle_msg,2,[]},
    {proc_lib,init_p_do_apply,3,
    [{file,"proc_lib.erl"},{line,**227}]}]}
    I think this is a duplicate of the problem reported at
    http://rabbitmq.1065348.n5.**nabble.com/Rabbit-3-1-1-crash-**
    question-td27540.html<http://rabbitmq.1065348.n5.nabble.com/Rabbit-3-1-1-crash-question-td27540.html>,
    the fix for which will appear in the next bugfix release.

    However, we were quite that the bug was only introduced in 3.1.0, so it's
    surprising to see this in 3.0.2. The code has certainly changed a lot on
    that area, so I'd advise upgrading in any case.

    Regards,

    Matthias.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130717/dae817ce/attachment.htm>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJul 16, '13 at 11:26p
activeJul 17, '13 at 5:34p
posts3
users2
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2017 Grokbase