Hello all,

I recently began reading up on rabbitmq and the amqp paradigm, via rabbitmq's site documentation and miscellaneous tutorials around the web, as I plan to use rabbit as the core for a real-time forum component that will be part of a site I am working on, where new posts will be forwarded out to client machines from a long-polling/websocket server sitting between the users and rabbitmq.(I'm new to message passing systems) I've been reading some posts and even a talk from Alexis Richardson, and I decided to join the mailing list to ask some specific questions; I have three:


1. I've read that basic_consume is the most efficient way to receive messages from rabbitmq. Is it possible for one consumer (my websock/lp server) to register to multiple queues(conceivably several thousand) and have messages continually sent to it?

2. Also, what routing topology/architecture would you recommend for this use case? I was thinking about either one queue per user or, one queue per topic (which would only be created once someone actually visits said topic) where users reading the same topic would be subscribed to the same queue. Either way, all these queues would be forwarding messages to the websocket/comet server which would then push them out to clients. There would also be a work queue, pushing out tasks for crud operations in the database to worker machines.


3. I'm planning on using a non-blocking server (NIO, or maybe twisted) for pushing the messages, however, I'm not sure how I should go about the writes once a message is sent from rabbitmq. Iterating throught the set of ready connections to find the connection to send it through doesn't sound like the best way, and what if the connection is busy and isn't in the ready set of connections? What should be done with the message?


Again I'm new to working with message passing systems as stated above, and a bit of help here would be much appreciated. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120203/44bac04b/attachment.htm>

Search Discussions

  • Steve Powell at Feb 13, 2012 at 4:27 pm
    Hi Jeff,

    Apparently, everyone is non-plussed by your questions; I'll attempt to answer
    one of them.
    1. I've read that basic_consume is the most efficient way to receive messages
    from rabbitmq.
    Correct -- there are two basic ways to get messages from a RabbitMQ queue, PULL
    and PUSH.

    To PULL you (the client application) issue a basicGet() call against a queue,
    and if there is a message waiting you get it. This is inefficient, because it
    requires a round trip to the server (protocol messages flowing both ways), and
    the application needs to poll the queue in some way if it wants to respond to
    messages in a timely manner -- so there is an overhead in two ways.

    PUSH access is where you, just once, register a Consumer class (or similar set
    of callbacks) with RabbitMQ, nominating a particular queue. Thereafter, whenever
    a message appears on the queue, RabbitMQ will deliver it to you (by calling a
    callback method you registered) without you having to ask for it. No polling is
    required by the client application, and no round trip to the server is needed.
    This is highly efficient.

    Furthermore, the Consumer has more than just message delivery callbacks -- the
    server can notify your application about other changes -- for example if the
    queue is deleted, or if the channel to which the Consumer is attached closes for
    some reason. In the PULL case, the application would have to seek that
    information at some point -- making the polling more costly (and more
    interesting to implement).
    Is it possible for one consumer (my websock/lp server) to
    register to multiple queues(conceivably several thousand) and have messages
    continually sent to it?
    Certainly it is. When registering a Consumer you (either nominate, or) get a
    (server-generated) consumer *tag*, essentially an identifier for the Consumer.
    You only have to keep a map of tags so that the Consumer can known (when it is
    called) for which registration this callback is being made.

    If you have a single channel connection to RabbitMQ, then the Consumers (each
    registration looks like a Consumer to the server) get called serially, so there
    is no concurrency. If you have many channels you can register for different
    queues on different channels and the distinct channels' callbacks can run
    concurrently.

    This is easiest for me to describe in the Java Client. I recommend you start by
    looking at the Java tutorials (http://www.rabbitmq.com/getstarted.html) and then
    go on to read the Java API Guide, especially the part about Consumers and the
    threading model.

    It ought to be fairly inexpensive to to basicConsume() on multiple queues, with
    the number of channels determining what sort of concurrency you want to have.

    Steve Powell (a starter bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 3 Feb 2012, at 17:11, Jeff C wrote:
    Hello all,

    I recently began reading up on rabbitmq and the amqp paradigm, via rabbitmq's
    site documentation and miscellaneous tutorials around the web, as I plan to use
    rabbit as the core for a real-time forum component that will be part of a site I
    am working on, where new posts will be forwarded out to client machines from a
    long-polling/websocket server sitting between the users and rabbitmq.(I'm new to
    message passing systems) I've been reading some posts and even a talk from
    Alexis Richardson, and I decided to join the mailing list to ask some specific
    questions; I have three:
    ...

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedFeb 3, '12 at 5:11p
activeFeb 13, '12 at 4:27p
posts2
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Jeff C: 1 post Steve Powell: 1 post

People

Translate

site design / logo © 2021 Grokbase