This is also somewhat related to the "One producer multiple subscribers"
thread.

My potential use case for AMQP is for distributed "event" handling. A
simple example is where one process detects that a device is down.
Prior to queueing it would do something like:

if is_down(device):
send_email(device)
send_page(device)
play_sound(device)

In my testing with amqp, I wrote a wrapper(attached) which lets me do
the following in different programs on different machines.

#poller.py
e = Events("device_down")
if is_down(device):
e.add_event(device)

#send_email.py
e = Events("device_down")
e.register_callback('send_email', send_email)

#play_sound.py
e = Events("device_down")
e.register_callback('play_sound', play_sound)
...

The wrapper automatically creates a fanout exchange called 'device_down'
and creates a durable queue for each registered handler.

So my first question is, am I correctly using rabbitmq/AMQP for this, or
did I completely miss some built in feature that would make all of this
much easier? I wanted each handler to be able to be restarted and never
miss a message, and I wanted a decoupling between the process emitting
the events and each handler. I think durable queues and fanout
exchanges are the proper solution for this, but I may be wrong.

My second question involves a timeout for the acknowledgment of
messages. I noticed that if a process crashed, a message would
automatically be put back into the queue, but if instead the process
went into an infinite loop, the message would stay checked out
indefinitely. Is there a way to tell the server that if a message has
been received, but not acked in a period of time, that it should be put
back into the queue?

I know there is also one potential issue where it may be difficult to
manage the different exchanges and queues on the server. It would be
nice to be able to specify that a queue is durable, but if messages are
not consumed for a long period of time then it should be deleted.


--
-- Justin Azoff
-- Network Performance Analyst
-------------- next part --------------
A non-text attachment was scrubbed...
Name: amqpevent.py
Type: text/x-python
Size: 3287 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20080710/041df608/attachment.py

Search Discussions

  • Ben Hood at Jul 10, 2008 at 7:35 pm
    Justin,
    On Thu, Jul 10, 2008 at 12:04 PM, Justin Azoff wrote:
    The wrapper automatically creates a fanout exchange called 'device_down'
    and creates a durable queue for each registered handler.

    So my first question is, am I correctly using rabbitmq/AMQP for this, or
    did I completely miss some built in feature that would make all of this
    much easier?
    On face value I don't see any problem with this approach, but I
    haven't had the time to look at your code in too much detail yet, so
    this is just a preliminary response.
    I wanted each handler to be able to be restarted and never
    miss a message, and I wanted a decoupling between the process emitting
    the events and each handler. I think durable queues and fanout
    exchanges are the proper solution for this, but I may be wrong.

    My second question involves a timeout for the acknowledgment of
    messages. I noticed that if a process crashed, a message would
    automatically be put back into the queue, but if instead the process
    went into an infinite loop, the message would stay checked out
    indefinitely. Is there a way to tell the server that if a message has
    been received, but not acked in a period of time, that it should be put
    back into the queue?
    This is handled by volume and not by time. After a certain amount of
    unroutable messages have queued up (because their consumption is not
    ack'ed) they will be returned to the sender.
    I know there is also one potential issue where it may be difficult to
    manage the different exchanges and queues on the server. It would be
    nice to be able to specify that a queue is durable, but if messages are
    not consumed for a long period of time then it should be deleted.
    This issue touches on the message expiration property in the message
    header, which unfortunately is not clearly defined and hence is not
    yet implemented in Rabbit.

    HTH,

    Ben
  • Ben Hood at Jul 11, 2008 at 12:02 pm

    On Thu, Jul 10, 2008 at 2:35 PM, Ben Hood wrote:
    My second question involves a timeout for the acknowledgment of
    messages. I noticed that if a process crashed, a message would
    automatically be put back into the queue, but if instead the process
    went into an infinite loop, the message would stay checked out
    indefinitely. Is there a way to tell the server that if a message has
    been received, but not acked in a period of time, that it should be put
    back into the queue?
    This is handled by volume and not by time. After a certain amount of
    unroutable messages have queued up (because their consumption is not
    ack'ed) they will be returned to the sender.
    Apologies for the misleading statement, this is actually incorrect.
    The messages are not returned to the sender.

    If there are outstanding unack'ed messages that have been delivered to
    a consumer, when the channel is closed that the messages were sent in,
    all of those outstanding messages will be requeued.

    Ben

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJul 10, '08 at 5:04p
activeJul 11, '08 at 12:02p
posts3
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Ben Hood: 2 posts Justin Azoff: 1 post

People

Translate

site design / logo © 2022 Grokbase