Hi,

Does anybody know if it's possible to update messages on a queue without
actually removing them. My use case requires a message to be removed from a
queue, only if the consumer is able to successfully process the message and
perform an action contained therein. If the action fails then the message
remains on the queue ready for a scheduled retry. I was thinking a message
time value could be changed so consumers only process messages that are
within a certain time range. If there's a better way to do this, any advice
would be welcome.

If there any examples or demos that can show this using Ruby or Python, this
would be even better.

Best regards,

Carl ?

--
View this message in context: http://old.nabble.com/Updating-messages-on-queues-tp29203202p29203202.html
Sent from the RabbitMQ mailing list archive at Nabble.com.

Search Discussions

  • Matthew Sackman at Jul 19, 2010 at 10:46 am

    On Mon, Jul 19, 2010 at 03:21:54AM -0700, cbourne wrote:
    Does anybody know if it's possible to update messages on a queue without
    actually removing them.
    Yes, I know. And no, there are no side-effect-free operations on
    messages in a queue.
    My use case requires a message to be removed from a
    queue, only if the consumer is able to successfully process the message and
    perform an action contained therein. If the action fails then the message
    remains on the queue ready for a scheduled retry.
    You simply want to have the consumers delay acknowledging the message
    until after they've successfully processed the message. If they can't
    process the message then presumably some error has occurred, which
    you should make cause the client to close the channel on which it
    received the message. That will cause the message to be requeued.
    I was thinking a message
    time value could be changed so consumers only process messages that are
    within a certain time range. If there's a better way to do this, any advice
    would be welcome.
    I'm afraid I don't really understand this. Can you explain this some
    more?

    Matthew
  • Cbourne at Jul 19, 2010 at 11:06 am

    Matthew Sackman-3 wrote:
    On Mon, Jul 19, 2010 at 03:21:54AM -0700, cbourne wrote:
    Does anybody know if it's possible to update messages on a queue without
    actually removing them.
    Yes, I know. And no, there are no side-effect-free operations on
    messages in a queue.O
    My use case requires a message to be removed from a
    queue, only if the consumer is able to successfully process the message
    and
    perform an action contained therein. If the action fails then the message
    remains on the queue ready for a scheduled retry.
    You simply want to have the consumers delay acknowledging the message
    until after they've successfully processed the message. If they can't
    process the message then presumably some error has occurred, which
    you should make cause the client to close the channel on which it
    received the message. That will cause the message to be requeued.
    I was thinking a message
    time value could be changed so consumers only process messages that are
    within a certain time range. If there's a better way to do this, any
    advice
    would be welcome.
    I'm afraid I don't really understand this. Can you explain this some
    more?
    So by this I mean adding some kind of a time field to the header so the
    consumer can apply a time based filter to the messages. Basically I need a
    retry queue that allows messages to be re-processed after a pre-defined
    time if the first attempt failed. Ideally this would also include the
    number of times to try before failing permanently.

    Carl


    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    --
    View this message in context: http://old.nabble.com/Updating-messages-on-queues-tp29203202p29203456.html
    Sent from the RabbitMQ mailing list archive at Nabble.com.
  • Matthew Sackman at Jul 19, 2010 at 11:12 am

    On Mon, Jul 19, 2010 at 04:06:38AM -0700, cbourne wrote:
    So by this I mean adding some kind of a time field to the header so the
    consumer can apply a time based filter to the messages. Basically I need a
    retry queue that allows messages to be re-processed after a pre-defined
    time if the first attempt failed. Ideally this would also include the
    number of times to try before failing permanently.
    Well, you could always just handle it completely in the consumer - if
    the consumer can't process it, republish the message with an update try
    count and then ack the original message. Because the publish will go to
    the end of the queue, that'll ensure that new messages do get seen and
    that a repeatedly failing message doesn't ensure that other messages get
    starved of attention.

    Matthew
  • Cbourne at Jul 19, 2010 at 11:22 am

    Matthew Sackman-3 wrote:
    On Mon, Jul 19, 2010 at 04:06:38AM -0700, cbourne wrote:
    So by this I mean adding some kind of a time field to the header so the
    consumer can apply a time based filter to the messages. Basically I need a
    retry queue that allows messages to be re-processed after a pre-defined
    time if the first attempt failed. Ideally this would also include the
    number of times to try before failing permanently.
    Well, you could always just handle it completely in the consumer - if
    the consumer can't process it, republish the message with an update try
    count and then ack the original message. Because the publish will go to
    the end of the queue, that'll ensure that new messages do get seen and
    that a repeatedly failing message doesn't ensure that other messages get
    starved of attention.

    Matthew
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

    Thanks Matt - guess the only problem with getting the consumer to do this is
    that if it's the consumer that fails, then the message could be lost. Would
    be better to leave the message on the queue so it could at least be
    processed by a different consumer if that were to happen. Hence, the
    requirement to update a field in the message header.

    Carl
    --
    View this message in context: http://old.nabble.com/Updating-messages-on-queues-tp29203202p29203617.html
    Sent from the RabbitMQ mailing list archive at Nabble.com.
  • Matthew Sackman at Jul 19, 2010 at 11:26 am

    On Mon, Jul 19, 2010 at 04:22:12AM -0700, cbourne wrote:
    Thanks Matt - guess the only problem with getting the consumer to do this is
    that if it's the consumer that fails, then the message could be lost.
    Not true. If the consumer fails before acking the message then Rabbit
    reinserts the message into the queue.

    Matthew
  • Aaron Westendorf at Jul 19, 2010 at 12:17 pm
    What are the rules for this? We're using no-ack messages and I've
    considered switching to explicit acks, but am concerned about the case
    where a message takes "a long time" to process.

    -Aaron
    On Mon, Jul 19, 2010 at 7:26 AM, Matthew Sackman wrote:
    On Mon, Jul 19, 2010 at 04:22:12AM -0700, cbourne wrote:
    Thanks Matt - guess the only problem with getting the consumer to do this is
    that if it's the consumer that fails, then the message could be lost.
    Not true. If the consumer fails before acking the message then Rabbit
    reinserts the message into the queue.

    Matthew

    --
    Aaron Westendorf
    Senior Software Engineer
    Agora Games
    359 Broadway
    Troy, NY 12180
    Phone: 518.268.1000
    aaron at agoragames.com
    www.agoragames.com
  • Matthew Sackman at Jul 19, 2010 at 12:29 pm

    On Mon, Jul 19, 2010 at 08:17:55AM -0400, Aaron Westendorf wrote:
    What are the rules for this? We're using no-ack messages and I've
    considered switching to explicit acks, but am concerned about the case
    where a message takes "a long time" to process.
    There is no timeout on acks. Once a message has been sent to the client,
    the client has as long as it needs before it acks the message. Until it
    acks the message, Rabbit hangs onto the message so that in the event of
    failure, Rabbit can reinsert the message into the queue from whence it
    came.

    Matthew
  • Cbourne at Jul 19, 2010 at 12:44 pm

    Matthew Sackman-3 wrote:
    On Mon, Jul 19, 2010 at 08:17:55AM -0400, Aaron Westendorf wrote:
    What are the rules for this? We're using no-ack messages and I've
    considered switching to explicit acks, but am concerned about the case
    where a message takes "a long time" to process.
    There is no timeout on acks. Once a message has been sent to the client,
    the client has as long as it needs before it acks the message. Until it
    acks the message, Rabbit hangs onto the message so that in the event of
    failure, Rabbit can reinsert the message into the queue from whence it
    came.

    Matthew
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    Thanks for all you help on this. Think this should give me something to go
    on.

    Carl
    --
    View this message in context: http://old.nabble.com/Updating-messages-on-queues-tp29203202p29204231.html
    Sent from the RabbitMQ mailing list archive at Nabble.com.
  • Cbourne at Jul 19, 2010 at 11:07 am

    Matthew Sackman-3 wrote:
    On Mon, Jul 19, 2010 at 03:21:54AM -0700, cbourne wrote:
    Does anybody know if it's possible to update messages on a queue without
    actually removing them.
    Yes, I know. And no, there are no side-effect-free operations on
    messages in a queue.O
    My use case requires a message to be removed from a
    queue, only if the consumer is able to successfully process the message
    and
    perform an action contained therein. If the action fails then the message
    remains on the queue ready for a scheduled retry.
    You simply want to have the consumers delay acknowledging the message
    until after they've successfully processed the message. If they can't
    process the message then presumably some error has occurred, which
    you should make cause the client to close the channel on which it
    received the message. That will cause the message to be requeued.
    I was thinking a message
    time value could be changed so consumers only process messages that are
    within a certain time range. If there's a better way to do this, any
    advice
    would be welcome.
    I'm afraid I don't really understand this. Can you explain this some
    more?
    So by this I mean adding some kind of a time field to the header so the
    consumer can apply a time based filter to the messages. Basically I need a
    retry queue that allows messages to be re-processed after a pre-defined
    time if the first attempt failed. Ideally this would also include the
    number of times to try before failing permanently.

    Carl


    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    --
    View this message in context: http://old.nabble.com/Updating-messages-on-queues-tp29203202p29203464.html
    Sent from the RabbitMQ mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJul 19, '10 at 10:21a
activeJul 19, '10 at 12:44p
posts10
users3
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase