[rabbitmq-discuss] Newcomer to Rabbit seeks exchange-type advice

On Tue, Jan 10, 2012 at 10:28 AM, Mark Petrovic wrote:
Hello.

This note is longer than I thought it would be, but I do not believe
my situation is complicated.

I am a RabbitMQ newcomer and am trying to identify which exchange type
to use for my application.

I have a publisher P that needs to send messages about database table
updates. ?A message will bear the name of a single table name,
followed by some opaque application data:

Logically,

message == tableName | somedata

So consider a simplified database with three tables of interest: ?T1,
T2, and T3.

There are two consumers, C1 and C2.

C1 needs to receive all messages concerning tables T1 and T2.

C2 needs to receive all messages concerning tables T2 and T3.

By implication, C1 and C2 must both receive messages about T2, where
their interests overlap.

Ideally, I want to publish to a single exchange, and, therefore, my
consumers also bind to this same single exchange. ?I say this because
in fact there are a lot more than three tables to treat - there are
almost 200. ?A proliferation of exchanges per-table would be not good.

I am considering a topic or header exchange.

If a header exchange, I was thinking of the publisher P putting the
concerned table name in a "table header" and the consumers binding to
the exchange with an interest in receiving messages with a table
header value equal to T1, T2, or T3 (I believe x-match == any would be
appropriate when the consumer binds). ?But some of my readings on
header exhchanges here

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-January/010935.html

suggest that header exchanges may not be as useful as maybe the name
suggests. ?Maybe I'm being overly paranoid.

And I consider topic exchanges because I know they admit messages
about T2 consumed by C1 still being available to C2.

Would someone be kind enough to suggest approaches to discern which
type of exchange to use.

Thank you very kindly.

--
Mark

On Tue, Jan 10, 2012 at 2:20 PM, Mark Petrovic wrote:
Talking to myself here (I have no problem with that :-)), I think I
have at least one solution to this.

- Create a single exchange E
- The Publisher P creates a channel with a single queue and routing
key with the same names as the Table T in question.
- Publisher publishes a message to channel
- The Consumer C1 iterates over the table names T1 and T2 he cares
about and binds his channel to queues and routing keys with the same
name as the table name iterated over
- The Consumer C1 then starts a thread for each of these queues, and
whose run() method calls channel.basicConsume(queueName, autoAck,
rabbitConsumer)

where rabbitConsumer extends com.rabbitmq.client.DefaultConsumer.

While this seems to work, and is a bit easier to understand how to
program than header exchanges (if a header exchange would even work
for me - I wonder), I dont' like the thread-per-queue I need to spawn
to handle messages inbound for each queue.

What would be nice is if I could call channel.basicConsume(autoAck,
rabbitConsumer), which would read messages off all the queues that
were bound to that channel.

While I have something working, I'm not yet real thrilled with my
programming model.

Anybody?

Thanks!


--
Mark

On Wed, Jan 11, 2012 at 1:46 AM, Mark Petrovic wrote:

I think I have a solution, better than one based on a header exchange
or the scheme I outline below.

It's basically this solution:

http://www.rabbitmq.com/tutorials/tutorial-five-java.html

where the routing keys are the table names. This gives me a single
consumer queue bound to mulitple routing keys, which gets my job done.


http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-January/010935.html


--
Mark
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

On Wed, Jan 11, 2012 at 1:28 AM, Alvaro Videla wrote:
Hi Mark,

Yes a topic exchange can be ideal when you need to "listen" based on several
patterns. Also this exchange leaves the possibility open for new table
names,
You just have to think about the binding pattern that you want to use.

Also keep in mind that if you want message X to arrive at consumer A and B
then you need one separate queue per consumer bound to that exchange using a
binding key that matches the routing key used when publishing message X.
What this means is that messages are "fanout'ed" at the exchange, but once
they arrive at a queue then messages are pulled out individually and once
gone from the queue is not seen again.

Cheers,

Alvaro