Hi,


I've tested out the clustering and HA support (including replication
policies) and was successful in achieving the following:


3 nodes in a cluster with full replication on all exchanges and queues.
Publisher sends messages to node A. I see these messages in replicated
queues on node A, B and C. That's good.


I started 2 consumers one reading from Node B and one from Node C. They
read the messages in a round-robin like fashion (i.e. once one consumer
read the message the replicated message in other node was gone).


That's all very good.


Problem is that this doesn't scale very well. Assume I have a very high
load from publishers. Consumers can't read fast enough. Eventually - I'll
run out of physical resources.


Does rabbitmq support any kind of sharding or message distribution based on
a known key?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130609/7b86c2e5/attachment.htm>

Search Discussions

  • Simon MacMullen at Jun 10, 2013 at 9:31 am

    On 10/06/13 06:59, Yair Ogen wrote:
    Does rabbitmq support any kind of sharding or message distribution based
    on a known key?

    See the Consistent Hash Exchange:
    https://github.com/rabbitmq/rabbitmq-consistent-hash-exchange


    Cheers, Simon


    --
    Simon MacMullen
    RabbitMQ, Pivotal
  • Simon MacMullen at Jun 11, 2013 at 10:57 am
    It already is: http://www.rabbitmq.com/plugins.html


    You just need to enable it with


        rabbitmq-plugins enable rabbitmq_consistent_hash_exchange


    Cheers, Simon

    On 11/06/13 11:54, Yair Ogen wrote:
    this looks great. will it make into rabbitmq as part of the product?


    On Mon, Jun 10, 2013 at 12:31 PM, Simon MacMullen <simon at rabbitmq.com
    wrote:

    On 10/06/13 06:59, Yair Ogen wrote:

    Does rabbitmq support any kind of sharding or message
    distribution based
    on a known key?


    See the Consistent Hash Exchange:
    https://github.com/rabbitmq/__rabbitmq-consistent-hash-__exchange
    <https://github.com/rabbitmq/rabbitmq-consistent-hash-exchange>

    Cheers, Simon

    --
    Simon MacMullen
    RabbitMQ, Pivotal



    --
    Simon MacMullen
    RabbitMQ, Pivotal
  • Nuodb at Jun 18, 2013 at 10:26 am
    Deploy your data where and when its needed. A flexible deployment
    architecture allows you to decouple storage and computational processing
    resources to serve your customer and business needs anywhere in the world.
    NuoDB enables cloud database <http://www.nuodbcloud.com> that will build
    a global data management platform that's highly available with guaranteed
    24/7 up-time, hot upgrades, built-in data redundancy and disaster recovery.






    --
    View this message in context: http://rabbitmq.1065348.n5.nabble.com/RabbitMQ-sharding-supprot-tp27301p27447.html
    Sent from the RabbitMQ mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJun 10, '13 at 5:59a
activeJun 18, '13 at 10:26a
posts4
users3
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2017 Grokbase