Running the code below, gives
channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope
envelope, BasicProperties properties, byte[] body) {
System.out.println(Thread.currentThread().getName());
}
}

o/p like
pool-1-thread-1
pool-1-thread-2
pool-1-thread-3
etc

There are multiple consumer threads consuming the messages.
Though multiple threads are consuming the messages, I observed that
messages still get processed sequentially.
Is that the case and if NOT, what is the way to guarantee sequential
processing of messages on one queue?

regards, Yogesh

Search Discussions

  • Yogesh Ketkar at Jan 22, 2012 at 6:31 pm
    Is basicGet only way to guarantee sequential processing of the
    messages on a queue?

    while(true) {
    GetResponse res = channel.basicGet(QUEUE_NAME, false);
    if(res != null) {
    // process message
    }
    }

    regards, Yogesh
  • Alexandru Scvorţov at Jan 23, 2012 at 10:51 am
    Hi Yogesh,

    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.

    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).

    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.

    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.

    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).

    Does this answer your question?

    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    new DefaultConsumer(channel) {
    @Override
    public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    System.out.println(Thread.currentThread().getName());
    }
    }

    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc

    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?

    regards, Yogesh

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Yogesh Ketkar at Jan 23, 2012 at 2:38 pm
    Thanks Alex.

    I think, for sequential message processing, rather than this pattern
    while(true) {
    GetResponse res = channel.basicGet(QUEUE_NAME, false);
    if(res != null) {
    // process and ack message
    }
    }

    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();

    is probably a better approach.
    Any comments on this?

    regards, Yogesh

    2012/1/23 Alexandru Scvor?ov <alexandru at rabbitmq.com>
    Hi Yogesh,

    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.

    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).

    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.

    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.

    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).

    Does this answer your question?

    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    new DefaultConsumer(channel) {
    @Override
    public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    System.out.println(Thread.currentThread().getName());
    }
    }

    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc

    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?

    regards, Yogesh

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120123/70745d4d/attachment.htm>
  • Alexandru Scvorţov at Jan 23, 2012 at 2:48 pm
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();

    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.

    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.

    Cheers,
    Alex
    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.

    I think, for sequential message processing, rather than this pattern
    while(true) {
    GetResponse res = channel.basicGet(QUEUE_NAME, false);
    if(res != null) {
    // process and ack message
    }
    }

    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();

    is probably a better approach.
    Any comments on this?

    regards, Yogesh

    2012/1/23 Alexandru Scvor?ov <alexandru at rabbitmq.com>
    Hi Yogesh,

    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.

    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).

    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.

    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.

    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).

    Does this answer your question?

    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    new DefaultConsumer(channel) {
    @Override
    public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    System.out.println(Thread.currentThread().getName());
    }
    }

    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc

    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?

    regards, Yogesh

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Yogesh Ketkar at Jan 23, 2012 at 2:54 pm
    Hello Alex,

    I thought, using default Connection con = factory.newConnection(),
    ThreadPool which gets created will have more than one thread and
    calling basicConsume will result in processing of messages on multiple
    threads.

    But if we explicitly provide Executors.newSingleThreadExecutor() to
    newConnection method, only single thread will come into play with
    basicConsume.

    Is my understanding correct?

    regards, Yogesh

    On Jan 23, 7:48?pm, Alexandru Scvor?ov wrote:
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.

    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.

    Cheers,
    Alex








    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.
    I think, for sequential message processing, rather than this pattern
    while(true) {
    ? ? GetResponse res = channel.basicGet(QUEUE_NAME, false);
    ? ? if(res != null) {
    ? ? ? ? // process and ack message
    ? ?}
    }
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    regards, Yogesh
    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>
    Hi Yogesh,
    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.
    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).
    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. ?So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.
    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).
    Does this answer your question?
    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    ? ? ? new DefaultConsumer(channel) ?{
    ? ? ? ? @Override
    ? ? ? ? public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    ? ? ? ? ? ? ? ? System.out.println(Thread.currentThread().getName());
    ? ? ? ? ?}
    }
    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Alexandru Scvorţov at Jan 23, 2012 at 3:22 pm
    Is my understanding correct?
    Almost. Deliveries will happen serially on a channel, so, as long as you
    only have one consumer, everything should be fine.
    On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
    Hello Alex,

    I thought, using default Connection con = factory.newConnection(),
    ThreadPool which gets created will have more than one thread and
    calling basicConsume will result in processing of messages on multiple
    threads.

    But if we explicitly provide Executors.newSingleThreadExecutor() to
    newConnection method, only single thread will come into play with
    basicConsume.

    Is my understanding correct?

    regards, Yogesh

    On Jan 23, 7:48?pm, Alexandru Scvor?ov wrote:
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.

    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.

    Cheers,
    Alex








    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.
    I think, for sequential message processing, rather than this pattern
    while(true) {
    ? ? GetResponse res = channel.basicGet(QUEUE_NAME, false);
    ? ? if(res != null) {
    ? ? ? ? // process and ack message
    ? ?}
    }
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    regards, Yogesh
    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>
    Hi Yogesh,
    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.
    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).
    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. ?So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.
    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).
    Does this answer your question?
    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    ? ? ? new DefaultConsumer(channel) ?{
    ? ? ? ? @Override
    ? ? ? ? public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    ? ? ? ? ? ? ? ? System.out.println(Thread.currentThread().getName());
    ? ? ? ? ?}
    }
    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Steve Powell at Jan 23, 2012 at 4:54 pm
    Hi Yogesh,

    What Alex says is perfectly correct.

    The executor service is set on the connection, and is there to allow multiple
    channels to execute consumers simultaneously, and without blocking the channel
    thread. By default you get an executor service which has five threads.

    From the point of view of a single channel, the consumers are driven serially,
    one delivery at a time, upon one of the executor service threads. The same
    thread is not guaranteed to be used each time for each consumer call for a
    channel (so don't try to use ThreadLocal objects).

    The reason we provide this, is so that clients with many, many channels are not
    forced to have a Consumer thread per channel -- something that is quite
    expensive -- but have the option of allocating as many threads as they wish if
    they need it. At the same time we guarantee that no Consumer calls can
    'overtake' each other on any one channel, and that Consumer processing runs on a
    distinct thread from the channel and the app, so that channel calls in the
    Consumer code will not deadlock.

    I hope this clarifies rather than obscures the situation,

    Regards,
    Steve Powell (a hoppy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 23 Jan 2012, at 15:22, Alexandru Scvor?ov wrote:

    Is my understanding correct?
    Almost. Deliveries will happen serially on a channel, so, as long as you
    only have one consumer, everything should be fine.
    On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
    Hello Alex,

    I thought, using default Connection con = factory.newConnection(),
    ThreadPool which gets created will have more than one thread and
    calling basicConsume will result in processing of messages on multiple
    threads.

    But if we explicitly provide Executors.newSingleThreadExecutor() to
    newConnection method, only single thread will come into play with
    basicConsume.

    Is my understanding correct?

    regards, Yogesh

    On Jan 23, 7:48 pm, Alexandru Scvor?ov wrote:
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.

    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.

    Cheers,
    Alex








    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.
    I think, for sequential message processing, rather than this pattern
    while(true) {
    GetResponse res = channel.basicGet(QUEUE_NAME, false);
    if(res != null) {
    // process and ack message
    }
    }
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    regards, Yogesh
    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>
    Hi Yogesh,
    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.
    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).
    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.
    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).
    Does this answer your question?
    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    new DefaultConsumer(channel) {
    @Override
    public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    System.out.println(Thread.currentThread().getName());
    }
    }
    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120123/60749c13/attachment.htm>
  • Yogesh Ketkar at Jan 24, 2012 at 5:29 am
    Hello Steve/Alex,

    Thanks a lot for the help. It's pretty clear now.
    So basically for a queue, as long as there is going to be only one
    consumer, it hardly matters if ThreadPool with
    com.rabbitmq.client.Connection underneath the channel has 1 thread or many
    threads.
    Well, having more threads will help if same channel is going to be used to
    consume messages from other queues.

    regards, Yogesh

    2012/1/23 Steve Powell <steve at rabbitmq.com>
    Hi Yogesh,

    What Alex says is perfectly correct.

    The executor service is set on the connection, and is there to allow
    multiple
    channels to execute consumers simultaneously, and without blocking the
    channel
    thread. By default you get an executor service which has five threads.

    From the point of view of a single channel, the consumers are driven
    serially,
    one delivery at a time, upon one of the executor service threads. The same
    thread is not guaranteed to be used each time for each consumer call for a
    channel (so don't try to use ThreadLocal objects).

    The reason we provide this, is so that clients with many, many channels
    are not
    forced to have a Consumer thread per channel -- something that is quite
    expensive -- but have the option of allocating as many threads as they
    wish if
    they need it. At the same time we guarantee that no Consumer calls can
    'overtake' each other on any one channel, and that Consumer processing
    runs on a
    distinct thread from the channel and the app, so that channel calls in the
    Consumer code will not deadlock.

    I hope this clarifies rather than obscures the situation,

    Regards,
    Steve Powell (*a hoppy bunny*)
    *----------some more definitions from the SPD----------*
    *vermin* (*v.*) Treating the dachshund for roundworm.
    *chinchilla* (*n.*) Cooling device for the lower jaw.
    *socialcast* (*n.*) Someone to whom everyone is speaking but nobody likes.

    On 23 Jan 2012, at 15:22, Alexandru Scvor?ov wrote:

    Is my understanding correct?


    Almost. Deliveries will happen serially on a channel, so, as long as you
    only have one consumer, everything should be fine.

    On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:

    Hello Alex,


    I thought, using default Connection con = factory.newConnection(),

    ThreadPool which gets created will have more than one thread and

    calling basicConsume will result in processing of messages on multiple

    threads.


    But if we explicitly provide Executors.newSingleThreadExecutor() to

    newConnection method, only single thread will come into play with

    basicConsume.


    Is my understanding correct?


    regards, Yogesh



    On Jan 23, 7:48 pm, Alexandru Scvor?ov wrote:

    Hi Yogesh,


    using channel.basicConsume on the channel which is created like this

    ExecutorService es = Executors.newSingleThreadExecutor();

    Connection connection = factory.newConnection(es);

    final Channel channel = connection.createChannel();


    is probably a better approach.

    Any comments on this?


    I don't quite see the point of the ExecutorService.


    Just doing this:


    Connection connection = factory.newConnection(es);

    final Channel channel = connection.createChannel();


    Then, calling basicConsume with your own subclass of DefaultConsumer

    should be fine.


    Cheers,

    Alex










    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:

    Thanks Alex.


    I think, for sequential message processing, rather than this pattern

    while(true) {

    GetResponse res = channel.basicGet(QUEUE_NAME, false);

    if(res != null) {

    // process and ack message

    }

    }


    using channel.basicConsume on the channel which is created like this

    ExecutorService es = Executors.newSingleThreadExecutor();

    Connection connection = factory.newConnection(es);

    final Channel channel = connection.createChannel();


    is probably a better approach.

    Any comments on this?


    regards, Yogesh


    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>


    Hi Yogesh,


    In the absence of consumer failures, RabbitMQ will deliver messages from

    a queue in order.


    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver

    them to consumers in the same order (1, 2, 3).


    But, if consumers fail before acknowledging the messages (or if they

    reject the messages), those messages will requeued at the end of the

    queue. So, if the consumer that got message 1 rejects it, the new

    order of messages will be 2, 3, 1.


    This all deals with message *delivery*.


    There are multiple consumer threads consuming the messages.

    Though multiple threads are consuming the messages, I observed that

    messages still get processed sequentially.

    Is that the case and if NOT, what is the way to guarantee sequential

    processing of messages on one queue?


    The library makes no guarantees about the order in which you process

    messages, only about the order in which they're delivered.


    If you want to process all the messages on a queue in order, only

    consume from one thread (doing basic.get from one thread like you

    suggested in the other email would work, but would also be highly

    inefficient).


    Does this answer your question?


    Cheers,

    Alex


    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:

    Running the code below, gives

    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,

    new DefaultConsumer(channel) {

    @Override

    public void handleDelivery(String consumerTag, Envelope

    envelope, BasicProperties properties, byte[] body) {

    System.out.println(Thread.currentThread().getName());

    }

    }


    o/p like

    pool-1-thread-1

    pool-1-thread-2

    pool-1-thread-3

    etc


    There are multiple consumer threads consuming the messages.

    Though multiple threads are consuming the messages, I observed that

    messages still get processed sequentially.

    Is that the case and if NOT, what is the way to guarantee sequential

    processing of messages on one queue?


    regards, Yogesh


    _______________________________________________

    rabbitmq-discuss mailing list

    rabbitmq-disc... at lists.rabbitmq.com

    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


    _______________________________________________

    rabbitmq-discuss mailing list

    rabbitmq-disc... at lists.rabbitmq.comhttps://
    lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

    _______________________________________________

    rabbitmq-discuss mailing list

    rabbitmq-discuss at lists.rabbitmq.com

    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120124/59f58372/attachment.htm>
  • Yogesh Ketkar at Jan 24, 2012 at 5:32 am

    Well, having more threads will help if same channel is going to be used to
    consume messages from other queues.
    Actually more appropriate would be, having more threads will help if
    same Connection is going to be used
    with different channels.
    On Jan 24, 10:29?am, Yogesh Ketkar wrote:
    Hello Steve/Alex,

    Thanks a lot for the help. It's pretty clear now.
    So basically for a queue, as long as there is going to be only one
    consumer, it hardly matters if ThreadPool with
    com.rabbitmq.client.Connection underneath the channel has 1 thread or many
    threads.
    Well, having more threads will help if same channel is going to be used to
    consume messages from other queues.

    regards, Yogesh

    2012/1/23 Steve Powell <st... at rabbitmq.com>






    Hi Yogesh,
    What Alex says is perfectly correct.
    The executor service is set on the connection, and is there to allow
    multiple
    channels to execute consumers simultaneously, and without blocking the
    channel
    thread. By default you get an executor service which has five threads.
    From the point of view of a single channel, the consumers are driven
    serially,
    one delivery at a time, upon one of the executor service threads. The same
    thread is not guaranteed to be used each time for each consumer call for a
    channel (so don't try to use ThreadLocal objects).
    The reason we provide this, is so that clients with many, many channels
    are not
    forced to have a Consumer thread per channel -- something that is quite
    expensive -- but have the option of allocating as many threads as they
    wish if
    they need it. At the same time we guarantee that no Consumer calls can
    'overtake' each other on any one channel, and that Consumer processing
    runs on a
    distinct thread from the channel and the app, so that channel calls in the
    Consumer code will not deadlock.
    I hope this clarifies rather than obscures the situation,
    Regards,
    Steve Powell ?(*a hoppy bunny*)
    *----------some more definitions from the SPD----------*
    *vermin* (*v.*) Treating the dachshund for roundworm.
    *chinchilla* (*n.*) Cooling device for the lower jaw.
    *socialcast* (*n.*) Someone to whom everyone is speaking but nobody likes.
    On 23 Jan 2012, at 15:22, Alexandru Scvor?ov wrote:
    Is my understanding correct?
    Almost. ?Deliveries will happen serially on a channel, so, as long as you
    only have one consumer, everything should be fine.
    On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
    Hello Alex,
    I thought, using default Connection con = factory.newConnection(),
    ThreadPool which gets created will have more than one thread and
    calling basicConsume will result in processing of messages on multiple
    threads.
    But if we explicitly provide Executors.newSingleThreadExecutor() to
    newConnection method, only single thread will come into play with
    basicConsume.
    Is my understanding correct?
    regards, Yogesh
    On Jan 23, 7:48 pm, Alexandru Scvor?ov wrote:
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.
    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.
    Cheers,
    Alex
    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.
    I think, for sequential message processing, rather than this pattern
    while(true) {
    ? ? GetResponse res = channel.basicGet(QUEUE_NAME, false);
    ? ? if(res != null) {
    ? ? ? ? // process and ack message
    ? ?}
    }
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    regards, Yogesh
    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>
    Hi Yogesh,
    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.
    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).
    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. ?So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.
    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).
    Does this answer your question?
    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    ? ? ? new DefaultConsumer(channel) ?{
    ? ? ? ? @Override
    ? ? ? ? public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    ? ? ? ? ? ? ? ? System.out.println(Thread.currentThread().getName());
    ? ? ? ? ?}
    }
    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://
    lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Steve Powell at Jan 25, 2012 at 2:42 pm
    Not only more appropriate, but also more true :-)

    Steve Powell (a happy bunny)
    ----------some more definitions from the SPD----------
    vermin (v.) Treating the dachshund for roundworm.
    chinchilla (n.) Cooling device for the lower jaw.
    socialcast (n.) Someone to whom everyone is speaking but nobody likes.
    On 24 Jan 2012, at 05:32, Yogesh Ketkar wrote:

    Well, having more threads will help if same channel is going to be used to
    consume messages from other queues.
    Actually more appropriate would be, having more threads will help if
    same Connection is going to be used
    with different channels.
    On Jan 24, 10:29 am, Yogesh Ketkar wrote:
    Hello Steve/Alex,

    Thanks a lot for the help. It's pretty clear now.
    So basically for a queue, as long as there is going to be only one
    consumer, it hardly matters if ThreadPool with
    com.rabbitmq.client.Connection underneath the channel has 1 thread or many
    threads.
    Well, having more threads will help if same channel is going to be used to
    consume messages from other queues.

    regards, Yogesh

    2012/1/23 Steve Powell <st... at rabbitmq.com>






    Hi Yogesh,
    What Alex says is perfectly correct.
    The executor service is set on the connection, and is there to allow
    multiple
    channels to execute consumers simultaneously, and without blocking the
    channel
    thread. By default you get an executor service which has five threads.
    From the point of view of a single channel, the consumers are driven
    serially,
    one delivery at a time, upon one of the executor service threads. The same
    thread is not guaranteed to be used each time for each consumer call for a
    channel (so don't try to use ThreadLocal objects).
    The reason we provide this, is so that clients with many, many channels
    are not
    forced to have a Consumer thread per channel -- something that is quite
    expensive -- but have the option of allocating as many threads as they
    wish if
    they need it. At the same time we guarantee that no Consumer calls can
    'overtake' each other on any one channel, and that Consumer processing
    runs on a
    distinct thread from the channel and the app, so that channel calls in the
    Consumer code will not deadlock.
    I hope this clarifies rather than obscures the situation,
    Regards,
    Steve Powell (*a hoppy bunny*)
    *----------some more definitions from the SPD----------*
    *vermin* (*v.*) Treating the dachshund for roundworm.
    *chinchilla* (*n.*) Cooling device for the lower jaw.
    *socialcast* (*n.*) Someone to whom everyone is speaking but nobody likes.
    On 23 Jan 2012, at 15:22, Alexandru Scvor?ov wrote:
    Is my understanding correct?
    Almost. Deliveries will happen serially on a channel, so, as long as you
    only have one consumer, everything should be fine.
    On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
    Hello Alex,
    I thought, using default Connection con = factory.newConnection(),
    ThreadPool which gets created will have more than one thread and
    calling basicConsume will result in processing of messages on multiple
    threads.
    But if we explicitly provide Executors.newSingleThreadExecutor() to
    newConnection method, only single thread will come into play with
    basicConsume.
    Is my understanding correct?
    regards, Yogesh
    On Jan 23, 7:48 pm, Alexandru Scvor?ov wrote:
    Hi Yogesh,
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    I don't quite see the point of the ExecutorService.
    Just doing this:
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    Then, calling basicConsume with your own subclass of DefaultConsumer
    should be fine.
    Cheers,
    Alex
    On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
    Thanks Alex.
    I think, for sequential message processing, rather than this pattern
    while(true) {
    GetResponse res = channel.basicGet(QUEUE_NAME, false);
    if(res != null) {
    // process and ack message
    }
    }
    using channel.basicConsume on the channel which is created like this
    ExecutorService es = Executors.newSingleThreadExecutor();
    Connection connection = factory.newConnection(es);
    final Channel channel = connection.createChannel();
    is probably a better approach.
    Any comments on this?
    regards, Yogesh
    2012/1/23 Alexandru Scvor?ov <alexan... at rabbitmq.com>
    Hi Yogesh,
    In the absence of consumer failures, RabbitMQ will deliver messages from
    a queue in order.
    So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
    them to consumers in the same order (1, 2, 3).
    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    This all deals with message *delivery*.
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    The library makes no guarantees about the order in which you process
    messages, only about the order in which they're delivered.
    If you want to process all the messages on a queue in order, only
    consume from one thread (doing basic.get from one thread like you
    suggested in the other email would work, but would also be highly
    inefficient).
    Does this answer your question?
    Cheers,
    Alex
    On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
    Running the code below, gives
    channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
    new DefaultConsumer(channel) {
    @Override
    public void handleDelivery(String consumerTag, Envelope
    envelope, BasicProperties properties, byte[] body) {
    System.out.println(Thread.currentThread().getName());
    }
    }
    o/p like
    pool-1-thread-1
    pool-1-thread-2
    pool-1-thread-3
    etc
    There are multiple consumer threads consuming the messages.
    Though multiple threads are consuming the messages, I observed that
    messages still get processed sequentially.
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    regards, Yogesh
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://
    lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120125/cec3dc74/attachment.htm>
  • Simone Busoli at Jan 23, 2012 at 8:40 pm
    Alex, I'm quite confident that since 2.7.0 messages are kept in queues in
    the same order in which they reached the queue in the first place.
    On Mon, Jan 23, 2012 at 11:51, Alexandru Scvor?ov wrote:

    But, if consumers fail before acknowledging the messages (or if they
    reject the messages), those messages will requeued at the end of the
    queue. So, if the consumer that got message 1 rejects it, the new
    order of messages will be 2, 3, 1.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120123/d0c1a5fa/attachment.htm>
  • Emile Joubert at Jan 23, 2012 at 10:55 am
    Hi Yogesh,
    On 22/01/12 17:38, Yogesh Ketkar wrote:
    Is that the case and if NOT, what is the way to guarantee sequential
    processing of messages on one queue?
    If I understand your question correctly, you want to know whether
    messages will always be delivered in order. This page has relevant info:
    http://www.rabbitmq.com/semantics.html#ordering :

    """
    messages published in one channel, passing through one exchange and one
    queue and one outgoing channel will be received in the same order that
    they were sent
    """

    This remains true whether you use basic.get or basic.consume. A consumer
    may observe messages out of order if there are multiple consumers
    consuming from the same queue and some of them perform requeues.


    -Emile

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJan 22, '12 at 5:38p
activeJan 25, '12 at 2:42p
posts13
users5
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase