Hello,

I've got a sick rabbit cluster which is acting unusual and things have
gone from bad to worse. Initially, the cluster, consisting of three
nodes/PCs worked ok, i.e. was able to setup a pub/sub test exchange
and worked as expected. Now, not only is the cluster not working (one
machine's rabbitmq log reports Mnesia cannot connect a node on another
machine) but the status command is only working one way!

Basically, I can "rabbitmqctl -n rabbit at A status" while on node B (and
properly see rabbitmq node on A), but "rabbitmqctl -n rabbit at B status"
while on machine A reports the node on B is not running.

I can ping all the machines (nodes A,B,C), all the firewalls have the
proper ports open. All erlang cookies are the same, etc. I've tried
reinstalling from scratch erlang/Rabbitmq on one of the nodes, still
get the same results.



ACTUAL RESULTS:NODE B (rabbit at LAKE-DS140) CAN SEE NODE A (rabbit at LAKE-
HP166)
C:\Documents and Settings\lst>rabbitmqctl -n rabbit at LAKE-HP166 status
Status of node 'rabbit at LAKE-HP166' ...
[{pid,672},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","2.7.1"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","2.7.1"},
{rabbit,"RabbitMQ","2.7.1"},
{os_mon,"CPO CXC 138 46","2.2.8"},
{mnesia,"MNESIA CXC 138 12","4.6"},
{amqp_client,"RabbitMQ AMQP Client","2.7.1"},
{sasl,"SASL CXC 138 11","2.2"},
{rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","2.7.1"},
{webmachine,"webmachine","1.7.0-rmq2.7.1-hg"},
{mochiweb,"MochiMedia Web Server","1.3-rmq2.7.1-git"},
{inets,"INETS CXC 138 49","5.8"},
{stdlib,"ERTS CXC 138 10","1.18"},
{kernel,"ERTS CXC 138 10","2.15"}]},
{os,{win32,nt}},
{erlang_version,"Erlang R15B (erts-5.9) [smp:2:2] [async-threads:
30]\n"},
{memory,
[{total,18332328},
{processes,6829982},
{processes_used,6823567},
{system,11502346},
{atom,428361},
{atom_used,409004},
{binary,216704},
{code,8085351},
{ets,654612}]},
{vm_memory_high_watermark,0.2287319764753309},
{vm_memory_limit,858993459}]
...done.


BUT NODE A (rabbit at LAKE-HP166) CANNOT SEE NODE B (rabbit at LAKE-DS140)
F:\Program Files\erl5.9\erts-5.9\bin>rabbitmqctl -n rabbit at LAKE-DS140
status
Status of node 'rabbit at LAKE-DS140' ...
Error: unable to connect to node 'rabbit at LAKE-DS140': nodedown
diagnostics:
- nodes and their ports on LAKE-DS140: [{rabbit,1052}]
- current node: 'rabbitmqctl4211 at lake-hp166'
- current node home dir: F:\Documents and Settings\prm
- current node cookie hash: OkIu/5IPQIZ171ZMFGQ8nQ==

any help very much appreciated

Thanks
David

Search Discussions

  • Emile Joubert at Feb 9, 2012 at 11:06 am
    Hi,
    On 08/02/12 17:25, davidib wrote:
    Basically, I can "rabbitmqctl -n rabbit at A status" while on node B (and
    properly see rabbitmq node on A), but "rabbitmqctl -n rabbit at B status"
    while on machine A reports the node on B is not running.
    This might be a firewall issue. Check whether you can establish a TCP
    connection to port 1052 on node B from node A. Running a cluster across
    firewalls (outside a datacentre) or over unreliable network links is not
    recommended. For more information see:
    http://www.rabbitmq.com/clustering.html#firewall


    -Emile
  • Davidib at Feb 9, 2012 at 2:07 pm
    Emile,

    Thanks so much for the feedback. I've explicitly opened ports
    2098,4369 and 5672 on all three hosts in the cluster. None of them
    have 1052 currently open, not sure why A can status B and B cannot
    status A but I'll open posts 1052 and try it. Are there any other
    ports that I should make sure are open?

    BTW I can't find any documentation regarding Rabbitmq/erlang using
    port 1052. Did I miss something/is this documented anywhere?

    Thanks again
    David


    On Feb 9, 5:06?am, Emile Joubert wrote:
    Hi,
    On 08/02/12 17:25, davidib wrote:

    Basically, I can "rabbitmqctl -n rabbit at A status" while on node B (and
    properly see rabbitmq node on A), but "rabbitmqctl -n rabbit at B status"
    while on machine A reports the node on B is not running.
    This might be a firewall issue. Check whether you can establish a TCP
    connection to port 1052 on node B from node A. Running a cluster across
    firewalls (outside a datacentre) or over unreliable network links is not
    recommended. For more information see:http://www.rabbitmq.com/clustering.html#firewall

    -Emile
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Emile Joubert at Feb 9, 2012 at 2:25 pm
    David,
    On 09/02/12 14:07, davidib wrote:
    BTW I can't find any documentation regarding Rabbitmq/erlang using
    port 1052. Did I miss something/is this documented anywhere?
    Firewall setup to allow clustering is documented here:
    http://www.rabbitmq.com/clustering.html#firewall

    The cases where it makes sense to run a cluster over a firewall is
    limited and you should carefully consider whether yours qualifies. It is
    more common to use Federation and the Shovel plugins over firewalled
    links. See http://www.rabbitmq.com/distributed.html


    -Emile

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedFeb 8, '12 at 5:25p
activeFeb 9, '12 at 2:25p
posts4
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Emile Joubert: 2 posts Davidib: 2 posts

People

Translate

site design / logo © 2021 Grokbase