Hi,

I'm having a little trouble with the RabbitMQ-C library. I'm able to declare exchanges, queues, setup bindings, and start the basic.consume (amqp_simple_rpc function) method as I would expect. But when a message is received by the consumer, I am able to observe (via the debugger) the complete message (all the frames) being processed. However, loop where multiple frames are linked together in the amqp_simple_rpc function (labeled "retry:") never terminates, and thus, at the end of what I think is a correct processing of all the frames in the message, control goes back into the wait_frame_inner function, where the recv function is called, anticipating more raw data on the socket that isn't going to arrive.

Do I have a correct (basic) understanding of how the amqp_simple_rpc is supposed to be working?

I am using the current tip (ef4df46cc0db) of the repository, default branch in the repo. RabbitMQ 2.4.1.

Thanks,

Clint Miller

Search Discussions

  • David Wragg at Jul 21, 2011 at 10:50 am

    Clint Miller <clint at rtcreativegroup.com> writes:
    I'm having a little trouble with the RabbitMQ-C library. I'm able to
    declare exchanges, queues, setup bindings, and start the basic.consume
    (amqp_simple_rpc function) method as I would expect.
    You are calling the amqp_basic_*, amqp_queue_* and amqp_exchange_*
    functions, right? amqp_simple_rpc is best thought of as an internal
    function, and calling it directly should never be necessary these days.
    None of the sample code in the tools/ and examples/ directories uses it.
    But when a message is received by the consumer, I am able to observe
    (via the debugger) the complete message (all the frames) being
    processed. However, loop where multiple frames are linked together in
    the amqp_simple_rpc function (labeled "retry:") never terminates, and
    thus, at the end of what I think is a correct processing of all the
    frames in the message, control goes back into the wait_frame_inner
    function, where the recv function is called, anticipating more raw
    data on the socket that isn't going to arrive.
    amqp_simple_rpc doesn't process incoming messages. If it gets an
    incoming frame that doesn't relate to the method, it stashes that frame
    away for later handling, and waits for the next frame (as the reply
    method frame should arrive sooner or later).

    To consume messages via basic.get, use amqp_basic_get and then
    amqp_simple_wait_frame to receive the body (see tools/get.c).

    To consume messages via basic.consume, use amqp_basic_consume and then
    call amqp_simple_wait_frame in a loop (see tools/consume.c and
    examples/amqp_consumer.c).

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • Clint Miller at Jul 21, 2011 at 2:09 pm
    Well,

    I'm actually using the community supported C++ client available here: https://github.com/akalend/amqpcpp

    As best as I can tell, that's the most current/up-to-date version. However, when I'm reading the source, I do see that it DOES NOT call the amqp_basic_* functions. Both the implementations of basic.get and basic.consume use the amqp_simple_rpc function. Is it possible there's a newer version of the C++ client that uses RabbitMQ-C's amqp_basic_* functions? Or do I need to build that myself?

    Clint
    On Jul 21, 2011, at 5:50 AM, David Wragg wrote:

    Clint Miller <clint at rtcreativegroup.com> writes:
    I'm having a little trouble with the RabbitMQ-C library. I'm able to
    declare exchanges, queues, setup bindings, and start the basic.consume
    (amqp_simple_rpc function) method as I would expect.
    You are calling the amqp_basic_*, amqp_queue_* and amqp_exchange_*
    functions, right? amqp_simple_rpc is best thought of as an internal
    function, and calling it directly should never be necessary these days.
    None of the sample code in the tools/ and examples/ directories uses it.
    But when a message is received by the consumer, I am able to observe
    (via the debugger) the complete message (all the frames) being
    processed. However, loop where multiple frames are linked together in
    the amqp_simple_rpc function (labeled "retry:") never terminates, and
    thus, at the end of what I think is a correct processing of all the
    frames in the message, control goes back into the wait_frame_inner
    function, where the recv function is called, anticipating more raw
    data on the socket that isn't going to arrive.
    amqp_simple_rpc doesn't process incoming messages. If it gets an
    incoming frame that doesn't relate to the method, it stashes that frame
    away for later handling, and waits for the next frame (as the reply
    method frame should arrive sooner or later).

    To consume messages via basic.get, use amqp_basic_get and then
    amqp_simple_wait_frame to receive the body (see tools/get.c).

    To consume messages via basic.consume, use amqp_basic_consume and then
    call amqp_simple_wait_frame in a loop (see tools/consume.c and
    examples/amqp_consumer.c).

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • David Wragg at Jul 21, 2011 at 2:22 pm

    Clint Miller <clint at rtcreativegroup.com> writes:
    I'm actually using the community supported C++ client available here:
    https://github.com/akalend/amqpcpp
    I'm afraid I have never looked closely at that client.
    As best as I can tell, that's the most current/up-to-date
    version. However, when I'm reading the source, I do see that it DOES
    NOT call the amqp_basic_* functions. Both the implementations of
    basic.get and basic.consume use the amqp_simple_rpc function. Is it
    possible there's a newer version of the C++ client that uses
    RabbitMQ-C's amqp_basic_* functions? Or do I need to build that
    myself?
    The C++ library should certainly be updated to avoid using
    amqp_simple_rpc, and use the method-specific API functions instead. But
    that is probably not the source of the problem in itself (those
    functions are generated wrappers onto amqp_simple_rpc).

    basic.get and basic.consume do involve an amqp_simple_rpc call. But
    then further code is required to actually receive the message or message
    deliveries.

    Could you construct a minimal program that demonstrates the problem?

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • Clint Miller at Jul 21, 2011 at 2:25 pm
    Would you want the minimal program to demonstrate the problem via use of the C++ client (as it exists today)?
    On Jul 21, 2011, at 9:22 AM, David Wragg wrote:

    Clint Miller <clint at rtcreativegroup.com> writes:
    I'm actually using the community supported C++ client available here:
    https://github.com/akalend/amqpcpp
    I'm afraid I have never looked closely at that client.
    As best as I can tell, that's the most current/up-to-date
    version. However, when I'm reading the source, I do see that it DOES
    NOT call the amqp_basic_* functions. Both the implementations of
    basic.get and basic.consume use the amqp_simple_rpc function. Is it
    possible there's a newer version of the C++ client that uses
    RabbitMQ-C's amqp_basic_* functions? Or do I need to build that
    myself?
    The C++ library should certainly be updated to avoid using
    amqp_simple_rpc, and use the method-specific API functions instead. But
    that is probably not the source of the problem in itself (those
    functions are generated wrappers onto amqp_simple_rpc).

    basic.get and basic.consume do involve an amqp_simple_rpc call. But
    then further code is required to actually receive the message or message
    deliveries.

    Could you construct a minimal program that demonstrates the problem?

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • David Wragg at Jul 21, 2011 at 2:30 pm

    Clint Miller <clint at rtcreativegroup.com> writes:
    Would you want the minimal program to demonstrate the problem via use
    of the C++ client (as it exists today)?
    Yes, that will be fine. I'm not sure where the problem lies, but it
    will be much easier to work that out from a complete example.

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • Clint Miller at Jul 21, 2011 at 10:08 pm
    OK, here's the basic example. It shows how the basic.consume method of the C++ library is not functioning correctly, but the basic.get works correctly.

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: amqpcpp_example.tar.gz
    Type: application/x-gzip
    Size: 1022 bytes
    Desc: not available
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110721/2e2fb15d/attachment.bin>
    -------------- next part --------------


    There's a basic Makefile that will probably require a little tweaking to get it working. It also depends on the 2 following libraries:

    - RabbitMQ-C (from the official repository, I built it at rev/changeset ef4df46cc0db)
    - AMQPcpp (from https://github.com/akalend/amqpcpp.git I built it at rev/changeset 80b17375f5bed6965acc07477499285b6851c92f)

    Thanks,

    Clint
    On Jul 21, 2011, at 9:30 AM, David Wragg wrote:

    Clint Miller <clint at rtcreativegroup.com> writes:
    Would you want the minimal program to demonstrate the problem via use
    of the C++ client (as it exists today)?
    Yes, that will be fine. I'm not sure where the problem lies, but it
    will be much easier to work that out from a complete example.

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • David Wragg at Jul 26, 2011 at 3:13 pm
    Hi Clint,

    Clint Miller <clint at rtcreativegroup.com> writes:
    OK, here's the basic example. It shows how the basic.consume method of
    the C++ library is not functioning correctly, but the basic.get works
    correctly.

    There's a basic Makefile that will probably require a little tweaking to get it working. It also depends on the 2 following libraries:

    - RabbitMQ-C (from the official repository, I built it at rev/changeset ef4df46cc0db)
    - AMQPcpp (from https://github.com/akalend/amqpcpp.git I built it at
    rev/changeset 80b17375f5bed6965acc07477499285b6851c92f)
    Thanks, I got it building quite easily. I'm not sure if I'm able to
    reproduce the problem that you described in your original mail, but I
    can confirm that it doesn't work for me.

    This isn't primarily due to the use of librabbitmq in amqpcpp. That
    looks basically sound (there are some calls to amqp_release_buffers in
    AMQPQueue.cpp that should be amqp_maybe_release_buffers, but otherwise I
    haven't found anything specifically wrong).

    But when I run the program, it aborts. Running it inside gdb, I get
    double free errors reported. So I ran it under valgrind. And valgrind
    finds many problems in amqpcpp. (In fairness, it also complains about
    some things in librabbitmq, but they are about the use of uninitialized
    data, and may be harmless.)

    The critical issue is a double free of the message buffer in
    AMQPQueue::sendConsumeCommand: At AMQPQueue.cpp#L512, buf is handed to
    the AMQPMessage object, and then freed. But AMQPMessage holds on to
    this pointer, and frees it again in its destructor (called via the
    message auto_ptr).

    valgrind also revealed a lot of cases of amqpcpp trying to use freed
    data (e.g. by passing a char* to AMQPQueue::setConsumerTag, you invoke
    the implicit conversion to string, and the string data is freed after
    the call; but setConsumerTag then uses c_str to get a pointer to the
    string data, which is saved in consumer_tag and only used later in
    sendConsumeCommand).

    I think amqpcpp needs cleaning up with valgrind in order to work
    reliably.

    David

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • Clint Miller at Jul 26, 2011 at 3:17 pm
    Hmm... I think I follow you. I'm correct thinking that AMQPcpp is maintained by someone not associated with VMWare/RabbitMQ?

    Regardless, thanks for your work looking into this issue.

    Clint
    On Jul 26, 2011, at 10:13 AM, David Wragg wrote:

    Hi Clint,

    Clint Miller <clint at rtcreativegroup.com> writes:
    OK, here's the basic example. It shows how the basic.consume method of
    the C++ library is not functioning correctly, but the basic.get works
    correctly.

    There's a basic Makefile that will probably require a little tweaking to get it working. It also depends on the 2 following libraries:

    - RabbitMQ-C (from the official repository, I built it at rev/changeset ef4df46cc0db)
    - AMQPcpp (from https://github.com/akalend/amqpcpp.git I built it at
    rev/changeset 80b17375f5bed6965acc07477499285b6851c92f)
    Thanks, I got it building quite easily. I'm not sure if I'm able to
    reproduce the problem that you described in your original mail, but I
    can confirm that it doesn't work for me.

    This isn't primarily due to the use of librabbitmq in amqpcpp. That
    looks basically sound (there are some calls to amqp_release_buffers in
    AMQPQueue.cpp that should be amqp_maybe_release_buffers, but otherwise I
    haven't found anything specifically wrong).

    But when I run the program, it aborts. Running it inside gdb, I get
    double free errors reported. So I ran it under valgrind. And valgrind
    finds many problems in amqpcpp. (In fairness, it also complains about
    some things in librabbitmq, but they are about the use of uninitialized
    data, and may be harmless.)

    The critical issue is a double free of the message buffer in
    AMQPQueue::sendConsumeCommand: At AMQPQueue.cpp#L512, buf is handed to
    the AMQPMessage object, and then freed. But AMQPMessage holds on to
    this pointer, and frees it again in its destructor (called via the
    message auto_ptr).

    valgrind also revealed a lot of cases of amqpcpp trying to use freed
    data (e.g. by passing a char* to AMQPQueue::setConsumerTag, you invoke
    the implicit conversion to string, and the string data is freed after
    the call; but setConsumerTag then uses c_str to get a pointer to the
    string data, which is saved in consumer_tag and only used later in
    sendConsumeCommand).

    I think amqpcpp needs cleaning up with valgrind in order to work
    reliably.

    David

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.
  • David Wragg at Jul 26, 2011 at 3:32 pm

    Clint Miller <clint at rtcreativegroup.com> writes:
    Hmm... I think I follow you. I'm correct thinking that AMQPcpp is
    maintained by someone not associated with VMWare/RabbitMQ?
    That's correct. I have CC'ed Alexandre Kalenderev, who is the author.

    --
    David Wragg
    Staff Engineer, RabbitMQ
    VMware, Inc.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJul 20, '11 at 8:45p
activeJul 26, '11 at 3:32p
posts10
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

David Wragg: 5 posts Clint Miller: 5 posts

People

Translate

site design / logo © 2022 Grokbase