I have one server application and multiple client applications. The server
can publish different kind of events to the interested clients. Clients
express their interest in a certain event type (together with a predicate
which describes in which situation they should get an event, so they can
filter events). From that moment on the server pushes events to the
subscribed clients when they occur and match the predicates.

For implementing this i thought to have the server listening (using
SimpleRpcServer) to one central queue where the subscription requests get
send to. A new queue matching that subscription is then constructed by the
server. The server publishes messages on the correct queues when certain
events occur. The clients listen on their specific queue(s) (also using a
SimpleRpcServer). The client knows which queue to listen to based on a
convention (name of the event etc).

Now the question: i want to be able to let the server specify (generate) a
unique queue name for each client/event subscription and then send it back
to the client so the client knows which queue to subscribe to. Is something
like this possible? I could not find any support for replying inside the
SimpleRpcServer.

And, more general, is this a valid design using RabbitMQ/AMQP? I'm still new
to RabbitMQ/AMQP so maybe i overlooked a more approriate way to do this..
--
View this message in context: http://old.nabble.com/SimpleRpcServer-send-generated-queue-name-to-client-tp27624063p27624063.html
Sent from the RabbitMQ mailing list archive at Nabble.com.

Search Discussions

  • Robert Raschke at Feb 17, 2010 at 2:15 pm

    On Wed, Feb 17, 2010 at 2:02 PM, glenner wrote:
    I have one server application and multiple client applications. The server
    can publish different kind of events to the interested clients. Clients
    express their interest in a certain event type (together with a predicate
    which describes in which situation they should get an event, so they can
    filter events). From that moment on the server pushes events to the
    subscribed clients when they occur and match the predicates.

    For implementing this i thought to have the server listening (using
    SimpleRpcServer) to one central queue where the subscription requests get
    send to. A new queue matching that subscription is then constructed by the
    server. The server publishes messages on the correct queues when certain
    events occur. The clients listen on their specific queue(s) (also using a
    SimpleRpcServer). The client knows which queue to listen to based on a
    convention (name of the event etc).

    Now the question: i want to be able to let the server specify (generate) a
    unique queue name for each client/event subscription and then send it back
    to the client so the client knows which queue to subscribe to. Is something
    like this possible? I could not find any support for replying inside the
    SimpleRpcServer.

    And, more general, is this a valid design using RabbitMQ/AMQP? I'm still
    new
    to RabbitMQ/AMQP so maybe i overlooked a more approriate way to do this..
    The Erlang amqp client lib has an example RPC server.

    In essence, when your client fires the "new subscription please" request it
    also has to create a reply queue. The initial message from client to server
    has to contain your "subscription request" plus your reply_to queue name
    and, ideally, a correlation_id. The reply_to and correlation_id fields, I
    think, are already part of the AMPQ message properties.

    Your client can then receive the desired "new subscription" queue name as a
    message from the server on the reply_to queue with the appropriate
    correlation_id.

    Hope this gets you into the right direction,
    Robby
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100217/2501a38d/attachment.htm
  • Tony Garnock-Jones at Mar 3, 2010 at 11:20 pm
    Hi,

    glenner wrote:
    And, more general, is this a valid design using RabbitMQ/AMQP? I'm still new
    to RabbitMQ/AMQP so maybe i overlooked a more approriate way to do this..
    It sounds reasonable; but it might be possible to leverage the features
    of the RabbitMQ server even more than you have in mind.

    For example, could it work to design a topic hierarchy for your system's
    events, and then simply use an AMQP topic exchange to broadcast them to
    the correct consumers?

    The application-level server would then blindly send events, with an
    appropriate routing key, to a named exchange it had set up, and the
    clients would create private queues (using queueDeclare()) and bind them
    (using queueBind()) to the exchange with appropriate binding key for
    selecting the subset of events they are interested in. There need then
    be no direct communication between the clients and the application-level
    server.

    You mentioned there were custom filter predicates. These are a potential
    missing piece. If the event types don't fit nicely onto the routing keys
    usable with topic exchanges, then you will be relying on client-side
    filtering, which may or may not be suitable for your application. On the
    other hand, if routing keys and a topic exchange get you 99% of the way
    there, then the remaining small amount of filtering can easily be done
    on the client side. A final option would be to implement a custom
    exchange type which is capable of understanding your filter predicates
    within the AMQP server itself.

    Regards,
    Tony
  • Ergeerts Glenn at Mar 5, 2010 at 9:27 am
    Hi,

    I thought about that option, but I think it is not flexible enough for our application.
    Like you said the custom filer predicates are very flexible. It could be possible to map them
    to routing keys and use client side filtering as you said, but for security reasons I want control exactly which client receives which messages.
    So this rules out client side filtering I guess?

    I did not know about the option of implementing a custom exchange type.. Is it hard to do?
    I don't know if this would be an ideal solution.. it feels strange pulling this application specific logic into the communication infrastructure (so tying our application to RabbitMQ) to me.. ?

    What would be the disadvantages of doing it with the multiple queues as first suggested?
    Is it performance related? Would it make a big difference using a topic exchange compared to for example 20 queues + subscriptions?
    Are there other disadvantages?

    Thanks,
    Glenn

    -----Oorspronkelijk bericht-----
    Van: Tony Garnock-Jones [mailto:tonyg at lshift.net]
    Verzonden: donderdag 4 maart 2010 0:20
    Aan: Ergeerts Glenn
    CC: rabbitmq-discuss at lists.rabbitmq.com
    Onderwerp: Re: [rabbitmq-discuss] SimpleRpcServer send generated queue name to client

    Hi,

    glenner wrote:
    And, more general, is this a valid design using RabbitMQ/AMQP? I'm still new
    to RabbitMQ/AMQP so maybe i overlooked a more approriate way to do this..
    It sounds reasonable; but it might be possible to leverage the features
    of the RabbitMQ server even more than you have in mind.

    For example, could it work to design a topic hierarchy for your system's
    events, and then simply use an AMQP topic exchange to broadcast them to
    the correct consumers?

    The application-level server would then blindly send events, with an
    appropriate routing key, to a named exchange it had set up, and the
    clients would create private queues (using queueDeclare()) and bind them
    (using queueBind()) to the exchange with appropriate binding key for
    selecting the subset of events they are interested in. There need then
    be no direct communication between the clients and the application-level
    server.

    You mentioned there were custom filter predicates. These are a potential
    missing piece. If the event types don't fit nicely onto the routing keys
    usable with topic exchanges, then you will be relying on client-side
    filtering, which may or may not be suitable for your application. On the
    other hand, if routing keys and a topic exchange get you 99% of the way
    there, then the remaining small amount of filtering can easily be done
    on the client side. A final option would be to implement a custom
    exchange type which is capable of understanding your filter predicates
    within the AMQP server itself.

    Regards,
    Tony
  • Matthias Radestock at Mar 23, 2010 at 1:57 pm
    Glenn,

    Ergeerts Glenn wrote:
    Like you said the custom filer predicates are very flexible. It could
    be possible to map them to routing keys and use client side
    filtering as you said, but for security reasons I want control
    exactly which client receives which messages. So this rules out
    client side filtering I guess?
    Do these security considerations only apply to event types or the event
    filtering predicates too?

    If the former, you could go for a hybrid solution where the server
    publishes events to a direct exchange, with the event type as the
    routing key. Each client's queue would then be bound to that exchange
    with binding keys for each of the event types the client is interested
    in. And client-side filtering would take care of the more complex
    filtering predicates.

    The creation/deletion of the client queues could be left up to the
    clients, but the access to the exchange could be restricted to the
    server, with all bindings of client queues to the exchange being
    established by that server.
    I did not know about the option of implementing a custom exchange
    type.. Is it hard to do?
    It's fairly straightforward - if you know a bit about Erlang and the
    rabbit internals ;) See
    http://www.lshift.net/blog/2010/01/22/plugin-exchange-types-for-rabbitmq
    for an example.
    it feels strange pulling this application specific logic
    into the communication infrastructure (so tying our application to
    RabbitMQ) to me.. ?
    The standard, but less efficient, approach to implementing arbitrary
    complex routing logic is to introduce a routing client. That is an
    ordinary AMQP client which

    a) consumes messages from, say, a single queue bound to a fanout
    exchange to which producer send all publishes,

    b) applies the necessary routing logic to compute a set of queues to
    send the message to,

    c) publishes copies of the message to the default exchange - one copy
    per queue it needs routing too.

    That's the basic setup, which can be suitably embellished by, say,
    consuming from multiple exchanges/queues, and publishing to exchanges
    other than the default exchange. Plus of course you need logic to deal
    with subscription management.
    What would be the disadvantages of doing it with the multiple queues
    as first suggested? Is it performance related? Would it make a big
    difference using a topic exchange compared to for example 20 queues +
    subscriptions? Are there other disadvantages?
    You end up with one queue per client in all solutions proposed so far.
    But there is still a performance advantage of getting rabbit to do as
    much of the routing as possible. That's what it's designed to do, and it
    (mostly) does it very efficiently. For example, if the same message
    needs to go to 10 different queues, doing the routing in application
    code results in ten separate copies of the message, all getting routed
    separately, whereas if rabbit does the routing there will be just one
    copy of the message and the distribution of that message to individual
    queues is done very efficiently.


    Regards,

    Matthias.
  • Tony Garnock-Jones at Mar 23, 2010 at 3:24 pm
    Hi Glenn,

    Ergeerts Glenn wrote:
    So this rules out client side filtering I guess?
    Sounds like it, yeah.
    I did not know about the option of implementing a custom exchange type.. Is it hard to do?
    No, it's now quite straightforward, though a bit bleeding-edge. The
    required features have landed on default, meaning they're release-ready,
    and will be in the next major release of rabbit.
    I don't know if this would be an ideal solution.. it feels strange pulling this application specific logic into the communication infrastructure (so tying our application to RabbitMQ) to me.. ?
    I understand completely :-) It's a new kind of thing, I think, and
    certainly a new way of thinking about networks---to have a programmable
    network, how odd!---to me.

    The question is *how much code* would you be embedding in the broker. If
    it's not very much, then the amount you save by not having to
    reimplement subscription management, expiry etc at the application level
    perhaps outweighs the potential cost of migrating to another broker later.
    What would be the disadvantages of doing it with the multiple queues as first suggested?
    Is it performance related? Would it make a big difference using a topic exchange compared to for example 20 queues + subscriptions?
    Are there other disadvantages?
    It's not a performance issue: it's that you end up reimplementing
    subscription and message distribution logic at the application level,
    even though it is already available at the AMQP level. That's why it
    makes sense to put the application logic *into* the AMQP level.

    Tony

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedFeb 17, '10 at 2:02p
activeMar 23, '10 at 3:24p
posts6
users4
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase