FAQ
I had a vendor's rabbitmq instance just crash on me, wondering if I could
get some help understanding why from the experts...


Windows 2008 with Rabbit 2.8.1 I got a crash dump had:
Slogan: Kernel pid terminated (application_controller)
({application_terminated,rabbit,shutdown})
System version: Erlang R15B01 (erts-5.9.1) [64-bit] [smp:4:4]
[async-threads:30]
...
=proc:<0.3.0>
State: Waiting
Name: erl_prim_loader
Spawned as: erlang:apply/2
Spawned by: <0.2.0>
Started: Sat Apr 06 12:32:28 2013
Message queue length: 0
Number of heap fragments: 0
Heap fragment data: 0
Link list: [<0.0.0>, #Port<0.1>]
Reductions: 1385542
Stack+heap: 6765
OldHeap: 17711
Heap unused: 1985
OldHeap unused: 10676
Program counter: 0x0000000000fd2d20 (erl_prim_loader:loop/3 + 176)
CP: 0x0000000000000000 (invalid)
arity = 0
=proc:<0.6.0>
State: Waiting
Name: error_logger
Spawned as: proc_lib:init_p/5
Spawned by: <0.2.0>
Started: Sat Apr 06 12:32:28 2013
Message queue length: 0
Number of heap fragments: 0
Heap fragment data: 0
Link list: [<0.0.0>]
Reductions: 667825443
Stack+heap: 6419485
OldHeap: 47828850
Heap unused: 2612692
OldHeap unused: 35671810
Program counter: 0x0000000004ddeb00 (gen_event:fetch_msg/5 + 72)
CP: 0x0000000000000000 (invalid)
arity = 0
...
2EFD1C8:lI110|N
7619550:t9:A5:state,A5:efile,N,A4:none,p<0.1>,A8:infinity,A9:undefined,A4:true,H2EF92E0
2EF92E0:t4:AA:prim_state,A5:false,A9:undefined,A9:undefined
=proc_dictionary:<0.6.0>
H52180A50
H51D06CD0
H51D00578
H51D00590
=proc_stack:<0.6.0>
0x00000000123ca118:SReturn addr 0x4DF35D8 (proc_lib:init_p_do_apply/3 + 56)
y0:A5:false
y1:N
y2:H10FDB290
y3:AC:error_logger
y4:P<0.2.0>
0x00000000123ca148:SReturn addr 0xF4814688 (<terminate process normally>)
y0:SCatch 0x4DF35F8 (proc_lib:init_p_do_apply/3 + 88)
=proc_heap:<0.6.0>
10FDB290:lH10FDB138|H10FDB268
10FDB138:t5:A7:handler,A19:rabbit_sasl_report_file_h,A5:false,H10FD8688,A5:false
10FD8688:t3:P<0.36.0>,H51D00A70,A5:error


Any advice?


thanks!
Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130517/5d5d633c/attachment.htm>

Search Discussions

  • Simon MacMullen at May 20, 2013 at 10:08 am
    Unfortunately the erl_crash.dump rarely contains anything very useful.
    Log files from the time of the crash are more likely to give us some
    clue what happened.


    Cheers, Simon

    On 17/05/13 20:29, Dave Gorman wrote:
    I had a vendor's rabbitmq instance just crash on me, wondering if I
    could get some help understanding why from the experts...

    Windows 2008 with Rabbit 2.8.1 I got a crash dump had:
    Slogan: Kernel pid terminated (application_controller)
    ({application_terminated,rabbit,shutdown})
    System version: Erlang R15B01 (erts-5.9.1) [64-bit] [smp:4:4]
    [async-threads:30]
    ...
    =proc:<0.3.0>
    State: Waiting
    Name: erl_prim_loader
    Spawned as: erlang:apply/2
    Spawned by: <0.2.0>
    Started: Sat Apr 06 12:32:28 2013
    Message queue length: 0
    Number of heap fragments: 0
    Heap fragment data: 0
    Link list: [<0.0.0>, #Port<0.1>]
    Reductions: 1385542
    Stack+heap: 6765
    OldHeap: 17711
    Heap unused: 1985
    OldHeap unused: 10676
    Program counter: 0x0000000000fd2d20 (erl_prim_loader:loop/3 + 176)
    CP: 0x0000000000000000 (invalid)
    arity = 0
    =proc:<0.6.0>
    State: Waiting
    Name: error_logger
    Spawned as: proc_lib:init_p/5
    Spawned by: <0.2.0>
    Started: Sat Apr 06 12:32:28 2013
    Message queue length: 0
    Number of heap fragments: 0
    Heap fragment data: 0
    Link list: [<0.0.0>]
    Reductions: 667825443
    Stack+heap: 6419485
    OldHeap: 47828850
    Heap unused: 2612692
    OldHeap unused: 35671810
    Program counter: 0x0000000004ddeb00 (gen_event:fetch_msg/5 + 72)
    CP: 0x0000000000000000 (invalid)
    arity = 0
    ...
    2EFD1C8:lI110|N
    7619550:t9:A5:state,A5:efile,N,A4:none,p<0.1>,A8:infinity,A9:undefined,A4:true,H2EF92E0
    2EF92E0:t4:AA:prim_state,A5:false,A9:undefined,A9:undefined
    =proc_dictionary:<0.6.0>
    H52180A50
    H51D06CD0
    H51D00578
    H51D00590
    =proc_stack:<0.6.0>
    0x00000000123ca118:SReturn addr 0x4DF35D8 (proc_lib:init_p_do_apply/3 + 56)
    y0:A5:false
    y1:N
    y2:H10FDB290
    y3:AC:error_logger
    y4:P<0.2.0>
    0x00000000123ca148:SReturn addr 0xF4814688 (<terminate process normally>)
    y0:SCatch 0x4DF35F8 (proc_lib:init_p_do_apply/3 + 88)
    =proc_heap:<0.6.0>
    10FDB290:lH10FDB138|H10FDB268
    10FDB138:t5:A7:handler,A19:rabbit_sasl_report_file_h,A5:false,H10FD8688,A5:false
    10FD8688:t3:P<0.36.0>,H51D00A70,A5:error

    Any advice?

    thanks!
    Dave


    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



    --
    Simon MacMullen
    RabbitMQ, Pivotal
  • Dave Gorman at May 20, 2013 at 1:09 pm
    Thanks Simon for your response!
    The SASL log has:
    =CRASH REPORT==== 17-May-2013::14:00:06 ===
       crasher:
         initial call: gen:init_it/6
         pid: <0.213.0>
         registered_name: []
         exception exit: {{badmatch,{error,eacces}},
                          [{rabbit_msg_store,safe_file_delete,3,[]},
                           {rabbit_msg_store_gc,'-attempt_action/3-fun-0-',1,[]},
                           {lists,'-filter/2-lc$^0/1-0-',2,
                                  [{file,"lists.erl"},{line,1220}]},
                           {rabbit_msg_store_gc,attempt_action,3,[]},
                           {rabbit_msg_store_gc,handle_cast,2,[]},
                           {gen_server2,handle_msg,2,[]},
                           {proc_lib,wake_up,3,
                                     [{file,"proc_lib.erl"},{line,237}]}]}
           in function gen_server2:terminate/3
         ancestors: [msg_store_persistent,rabbit_sup,<0.159.0>]
         messages: []
         links: [<0.208.0>]
         dictionary: [{fhc_age_tree,{0,nil}}]
         trap_exit: false
         status: running
         heap_size: 17711
         stack_size: 24
         reductions: 1150712551
       neighbours:


    Does badmatch mean anything to you?




    On Mon, May 20, 2013 at 5:08 AM, Simon MacMullen wrote:

    Unfortunately the erl_crash.dump rarely contains anything very useful. Log
    files from the time of the crash are more likely to give us some clue what
    happened.

    Cheers, Simon

    On 17/05/13 20:29, Dave Gorman wrote:

    I had a vendor's rabbitmq instance just crash on me, wondering if I
    could get some help understanding why from the experts...

    Windows 2008 with Rabbit 2.8.1 I got a crash dump had:
    Slogan: Kernel pid terminated (application_controller)
    ({application_terminated,**rabbit,shutdown})
    System version: Erlang R15B01 (erts-5.9.1) [64-bit] [smp:4:4]
    [async-threads:30]
    ...
    =proc:<0.3.0>
    State: Waiting
    Name: erl_prim_loader
    Spawned as: erlang:apply/2
    Spawned by: <0.2.0>
    Started: Sat Apr 06 12:32:28 2013
    Message queue length: 0
    Number of heap fragments: 0
    Heap fragment data: 0
    Link list: [<0.0.0>, #Port<0.1>]
    Reductions: 1385542
    Stack+heap: 6765
    OldHeap: 17711
    Heap unused: 1985
    OldHeap unused: 10676
    Program counter: 0x0000000000fd2d20 (erl_prim_loader:loop/3 + 176)
    CP: 0x0000000000000000 (invalid)
    arity = 0
    =proc:<0.6.0>
    State: Waiting
    Name: error_logger
    Spawned as: proc_lib:init_p/5
    Spawned by: <0.2.0>
    Started: Sat Apr 06 12:32:28 2013
    Message queue length: 0
    Number of heap fragments: 0
    Heap fragment data: 0
    Link list: [<0.0.0>]
    Reductions: 667825443
    Stack+heap: 6419485
    OldHeap: 47828850
    Heap unused: 2612692
    OldHeap unused: 35671810
    Program counter: 0x0000000004ddeb00 (gen_event:fetch_msg/5 + 72)
    CP: 0x0000000000000000 (invalid)
    arity = 0
    ...
    2EFD1C8:lI110|N
    7619550:t9:A5:state,A5:efile,**N,A4:none,p<0.1>,A8:infinity,**
    A9:undefined,A4:true,H2EF92E0
    2EF92E0:t4:AA:prim_state,A5:**false,A9:undefined,A9:**undefined
    =proc_dictionary:<0.6.0>
    H52180A50
    H51D06CD0
    H51D00578
    H51D00590
    =proc_stack:<0.6.0>
    0x00000000123ca118:SReturn addr 0x4DF35D8 (proc_lib:init_p_do_apply/3 +
    56)
    y0:A5:false
    y1:N
    y2:H10FDB290
    y3:AC:error_logger
    y4:P<0.2.0>
    0x00000000123ca148:SReturn addr 0xF4814688 (<terminate process normally>)
    y0:SCatch 0x4DF35F8 (proc_lib:init_p_do_apply/3 + 88)
    =proc_heap:<0.6.0>
    10FDB290:lH10FDB138|H10FDB268
    10FDB138:t5:A7:handler,A19:**rabbit_sasl_report_file_h,A5:**
    false,H10FD8688,A5:false
    10FD8688:t3:P<0.36.0>,**H51D00A70,A5:error

    Any advice?

    thanks!
    Dave


    ______________________________**_________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.**rabbitmq.com<rabbitmq-discuss@lists.rabbitmq.com>
    https://lists.rabbitmq.com/**cgi-bin/mailman/listinfo/**rabbitmq-discuss<https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
    --
    Simon MacMullen
    RabbitMQ, Pivotal
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130520/286e3ca5/attachment.htm>
  • Simon MacMullen at May 20, 2013 at 1:17 pm

    On 20/05/13 14:09, Dave Gorman wrote:
    Thanks Simon for your response!
    The SASL log has:
    exception exit: {{badmatch,{error,eacces}},
    [{rabbit_msg_store,safe_file_delete,3,[]},

    {rabbit_msg_store_gc,'-attempt_action/3-fun-0-',1,[]},
    Does badmatch mean anything to you?

    badmatch just means we saw some result we weren't expecting - but the
    interesting bit is "{error,eacces}".


    This looks like the message store was attempting to delete a file and
    was prevented from doing so by the OS. This is a hard problem to recover
    from - RabbitMQ assumes that it can modify and delete any file it has
    created; it's not clear what to do if it can't.


    On Windows I guess there could be a badly written virus scanner or
    backup program which is taking exclusive access to RabbitMQ's files.
    Could something like that be happening?


    Cheers, Simon


    --
    Simon MacMullen
    RabbitMQ, Pivotal
  • Michael Klishin at May 20, 2013 at 1:25 pm
    2013/5/20 Dave Gorman [{rabbit_msg_store,safe_file_delete,3,[]},




    This suggests RabbitMQ tried to delete a file but could not due to
    insufficient permissions
    (or possibly missing file).

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedMay 17, '13 at 7:29p
activeMay 20, '13 at 1:25p
posts5
users3
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2017 Grokbase