Hi there,
I'm about to use RabbitMQ in my first amqp project and am having a bit
of difficulty identifying the necessary queues and exchanges.
In our use case, there will be only one producer who publishes a
message to the broker. This message represents configuration data.
Then multiple consumers read the published configuration from the
broker during their startup process, so it shouldn't happen very
often. The message should remain in the queue after consumption so
that the following consumers are able to read it. When some
configuration data changes, the producer republishes it and alerts
(possibly through some other queue) the consumers, who then read the
configuration message again.
I know this question isn't exactly related to RabbitMQ, but reading
the AMQP spec didn't help much. Could anyone help me identify which
exchange types, queue and message parameters I need for this to work?
Thanks in advance.

Fabio.

Search Discussions

  • Alexis Richardson at Mar 2, 2011 at 2:20 pm
    Fabio

    The simplest way is probably as follows:

    http://www.rabbitmq.com/tutorial-three-python.html

    Note that each consumer has its own queue, which acts as a buffer on
    behalf of that consumer. The downside of this is a slow consumer may
    get several configuration updates at once, so it will need to discard
    the earlier ones but should act on the latest.

    alexis


    On Wed, Mar 2, 2011 at 2:14 PM, Fabio Margarido
    wrote:
    Hi there,
    I'm about to use RabbitMQ in my first amqp project and am having a bit
    of difficulty identifying the necessary queues and exchanges.
    In our use case, there will be only one producer who publishes a
    message to the broker. This message represents configuration data.
    Then multiple consumers read the published configuration from the
    broker during their startup process, so it shouldn't happen very
    often. The message should remain in the queue after consumption so
    that the following consumers are able to read it. When some
    configuration data changes, the producer republishes it and alerts
    (possibly through some other queue) the consumers, who then read the
    configuration message again.
    I know this question isn't exactly related to RabbitMQ, but reading
    the AMQP spec didn't help much. Could anyone help me identify which
    exchange types, queue and message parameters I need for this to work?
    Thanks in advance.

    Fabio.
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Fabio Margarido at Mar 2, 2011 at 4:30 pm
    Forgot to copy the list.

    Hi Alexis, thank you for the link.
    It answers some of my doubts, but there are still some bits I don't
    understand. For example, the page says: "The messages will be lost if
    no queue is bound to the exchange yet, but that's okay for us; if no
    consumer is listening yet (i.e., the exchange hasn't been created) we
    can safely discard the message.". That's not the case for me; what I
    wanted to do is publish once and consume as many times as necessary.
    The scenarios I can't quite grasp yet are:

    * The producer publishes the message before any consumers are running;
    * The consumer receives the message, dies and reboots, consumes the
    same message once more.

    Are these even possible?
    Thanks again.
    On Wed, Mar 2, 2011 at 11:20, Alexis Richardson wrote:
    Fabio

    The simplest way is probably as follows:

    http://www.rabbitmq.com/tutorial-three-python.html

    Note that each consumer has its own queue, which acts as a buffer on
    behalf of that consumer. ?The downside of this is a slow consumer may
    get several configuration updates at once, so it will need to discard
    the earlier ones but should act on the latest.

    alexis


    On Wed, Mar 2, 2011 at 2:14 PM, Fabio Margarido
    wrote:
    Hi there,
    I'm about to use RabbitMQ in my first amqp project and am having a bit
    of difficulty identifying the necessary queues and exchanges.
    In our use case, there will be only one producer who publishes a
    message to the broker. This message represents configuration data.
    Then multiple consumers read the published configuration from the
    broker during their startup process, so it shouldn't happen very
    often. The message should remain in the queue after consumption so
    that the following consumers are able to read it. When some
    configuration data changes, the producer republishes it and alerts
    (possibly through some other queue) the consumers, who then read the
    configuration message again.
    I know this question isn't exactly related to RabbitMQ, but reading
    the AMQP spec didn't help much. Could anyone help me identify which
    exchange types, queue and message parameters I need for this to work?
    Thanks in advance.

    Fabio.
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Alexis Richardson at Mar 2, 2011 at 4:36 pm
    Fabio

    On Wed, Mar 2, 2011 at 4:30 PM, Fabio Margarido
    wrote:
    Forgot to copy the list.

    Hi Alexis, thank you for the link.
    It answers some of my doubts, but there are still some bits I don't
    understand. For example, the page says: "The messages will be lost if
    no queue is bound to the exchange yet, but that's okay for us; if no
    consumer is listening yet (i.e., the exchange hasn't been created) we
    can safely discard the message.". That's not the case for me; what I
    wanted to do is publish once and consume as many times as necessary.
    In the link I sent, you publish once per update, independent of the
    number of consumers.

    The scenarios I can't quite grasp yet are:

    * The producer publishes the message before any consumers are running;
    Do you want a new consumer to see all old updates, or just the latest
    update that occurred before the new consumer connects to the broker?
    * The consumer receives the message, dies and reboots, consumes the
    same message once more.
    You can set things up so that the message gets requeued, but not 'in order'.

    I'd tentatively suggest that you may be talking more about a cache
    than message queue.

    alexis


    Are these even possible?
    Thanks again.
    On Wed, Mar 2, 2011 at 11:20, Alexis Richardson wrote:
    Fabio

    The simplest way is probably as follows:

    http://www.rabbitmq.com/tutorial-three-python.html

    Note that each consumer has its own queue, which acts as a buffer on
    behalf of that consumer. ?The downside of this is a slow consumer may
    get several configuration updates at once, so it will need to discard
    the earlier ones but should act on the latest.

    alexis


    On Wed, Mar 2, 2011 at 2:14 PM, Fabio Margarido
    wrote:
    Hi there,
    I'm about to use RabbitMQ in my first amqp project and am having a bit
    of difficulty identifying the necessary queues and exchanges.
    In our use case, there will be only one producer who publishes a
    message to the broker. This message represents configuration data.
    Then multiple consumers read the published configuration from the
    broker during their startup process, so it shouldn't happen very
    often. The message should remain in the queue after consumption so
    that the following consumers are able to read it. When some
    configuration data changes, the producer republishes it and alerts
    (possibly through some other queue) the consumers, who then read the
    configuration message again.
    I know this question isn't exactly related to RabbitMQ, but reading
    the AMQP spec didn't help much. Could anyone help me identify which
    exchange types, queue and message parameters I need for this to work?
    Thanks in advance.

    Fabio.
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
  • Fabio Margarido at Mar 2, 2011 at 4:57 pm
    Hi Alexis.
    On Wed, Mar 2, 2011 at 13:36, Alexis Richardson wrote:
    In the link I sent, you publish once per update, independent of the
    number of consumers.
    OK.
    The scenarios I can't quite grasp yet are:

    * The producer publishes the message before any consumers are running;
    Do you want a new consumer to see all old updates, or just the latest
    update that occurred before the new consumer connects to the broker?
    I want the consumers to see just the latest update. But I figured that
    as the queue is exclusive and so gets created only the moment it
    starts running, it wouldn't get the message that was published prior
    to that. If that's not the case, then I guess it should work.
    * The consumer receives the message, dies and reboots, consumes the
    same message once more.
    You can set things up so that the message gets requeued, but not 'in order'.
    Order shouldn't matter, as I'm only interested in that latest update.
    I'd tentatively suggest that you may be talking more about a cache
    than message queue.
    It is a kind of cache. I guess it could be done a number of different
    ways, like setting up an NFS share and storing the config in a file or
    maybe set something up in the application servers. But I figured that
    since I already have the RabbitMQ infrastructure up and running for my
    other (more orthodox) messaging needs, I might as well take advantage
    of it for this case rather than having to maintain another service.
    Thanks.
  • Alexis Richardson at Mar 2, 2011 at 5:11 pm

    On Wed, Mar 2, 2011 at 4:57 PM, Fabio Margarido wrote:
    * The producer publishes the message before any consumers are running;
    Do you want a new consumer to see all old updates, or just the latest
    update that occurred before the new consumer connects to the broker?
    I want the consumers to see just the latest update. But I figured that
    as the queue is exclusive and so gets created only the moment it
    starts running, it wouldn't get the message that was published prior
    to that.
    Correct.

    If you want the broker to retain messages published before any
    consumers appear, then get the publisher to create a queue for them.
    But what happens to those messages, is an application level decision
    you will have to make...


    * The consumer receives the message, dies and reboots, consumes the
    same message once more.
    You can set things up so that the message gets requeued, but not 'in order'.
    Order shouldn't matter, as I'm only interested in that latest update.
    Yes but if a message gets requeued, it will go to the back of the
    (FIFO) queue. You can work around this by attaching an application
    level stamp.

    I'd tentatively suggest that you may be talking more about a cache
    than message queue.
    It is a kind of cache. I guess it could be done a number of different
    ways, like setting up an NFS share and storing the config in a file or
    maybe set something up in the application servers. But I figured that
    since I already have the RabbitMQ infrastructure up and running for my
    other (more orthodox) messaging needs, I might as well take advantage
    of it for this case rather than having to maintain another service.
    Thanks.
    No problem.

    alexis
  • Alister Morton at Mar 2, 2011 at 4:58 pm

    I'd tentatively suggest that you may be talking more about a cache
    than message queue.
    Or failing that, perhaps a bit like the "real time record" model, such as used for financial data, where a consumer gets the last published update for a topic. IIRC the guys at Rabbit HQ did some work on an exchange which persists the last update to a topic - what became of that? This would fit the problem definition quite well, I think.

    The information herein may have been obtained from various sources. Any opinion expressed may be that of the sender only, is subject to change without notice and should be independently evaluated. Nothing herein constitutes investment advice or an offer, or solicitation of an offer, to buy or sell any financial product. Any data consists of purely indicative prices and should not be relied upon to revalue any commercial positions held by any recipient. Tradition makes no warranty that the data represent or indicates prices at which transactions may be or have been made by any Tradition Group company. To the maximum extent of the law, Tradition accepts no responsibility for, and cannot and does not warrant the integrity, accuracy, quality, completeness, merchantability or suitability for a particular purpose or requirement of the information or data, even if arising out of the negligence of Tradition or otherwise. Tradition accepts no liability for any direct, indirect or other consequential loss arising out of any use of the information contained in this document or any omission from it. This communication is directed at Eligible Counterparties and Professional Clients as defined by the FSA. It is not for distribution to nor should it be relied upon by Private Clients. It is not intended for distribution to, or use by any person or entity in any jurisdiction or country where such distribution or use would be contrary to any applicable law or regulation. Please note that, for business or compliance reasons, we may monitor and read emails sent or received using our servers or equipment. Tradition (UK) Ltd (937647; FSA 139200), Tradition Financial Services Ltd (1046064; FSA 147543), TFS Derivatives Ltd (4051930; FSA 197244), Tradition London Clearing Ltd (3633863; FSA 190632) and TFS-ICAP Ltd (4025995; FSA 206018) registered in England at Beaufort House, 15 St Botolph Street, London EC3A 7QX; authorised and regulated by the Financial Services Authority. VAT No: GB 365 4639 27 except TFS-ICAP GB 766 0854 05.
  • Alexis Richardson at Mar 2, 2011 at 5:12 pm

    On Wed, Mar 2, 2011 at 4:58 PM, Alister Morton wrote:
    I'd tentatively suggest that you may be talking more about a cache
    than message queue.
    Or failing that, perhaps a bit like the "real time record" model, such as used for financial data, where a consumer gets the last published update for a topic. IIRC the guys at Rabbit HQ did some work on an exchange which persists the last update to a topic - what became of that? This would fit the problem definition quite well, I think.
    Yes, though I think that plugin won't work with RabbitMQ 2.3.1. Maybe
    it could be revived...

    alexis

    The information herein may have been obtained from various sources. Any opinion expressed may be that of the sender only, is subject to change without notice and should be independently evaluated. Nothing herein constitutes investment advice or an offer, or solicitation of an offer, to buy or sell any financial product. Any data consists of purely indicative prices and should not be relied upon to revalue any commercial positions held by any recipient. Tradition makes no warranty that the data represent or indicates prices at which transactions may be or have been made by any Tradition Group company. To the maximum extent of the law, Tradition accepts no responsibility for, and cannot and does not warrant the integrity, accuracy, quality, completeness, merchantability or suitability for a particular purpose or requirement of the information or data, even if arising out of the negligence of Tradition or otherwise. Tradition accepts no liability for any direct, indirect or other consequential loss arising out of any use of the information contained in this document or any omission from it. This communication is directed at Eligible Counterparties and Professional Clients as defined by the FSA. It is not for distribution to nor should it be relied upon by Private Clients. It is not intended for distribution to, or use by any person or entity in any jurisdiction or country where such distribution or use would be contrary to any applicable law or regulation. Please note that, for business or compliance reasons, we may monitor and read emails sent or received using our servers or equipment. Tradition (UK) Ltd (937647; FSA 139200), Tradition Financial Services Ltd (1046064; FSA 147543), TFS Derivatives Ltd (4051930; FSA 197244), Tradition London Clearing Ltd (3633863; FSA 190632) and TFS-ICAP Ltd (4025995; FSA 206018) registered in England at Beaufort House, 15 St Botolph Street, London EC3A 7QX; authorised and regulated by the Financial Services Authority. VAT No: GB 365 4639 27 except TFS-ICAP GB 766 0854 05.
  • Fabio Margarido at Mar 2, 2011 at 5:24 pm

    On Wed, Mar 2, 2011 at 14:12, Alexis Richardson wrote:
    Or failing that, perhaps a bit like the "real time record" model, such as used for financial data, where a consumer gets the last published update for a topic. IIRC the guys at Rabbit HQ did some work on an exchange which persists the last update to a topic - what became of that? This would fit the problem definition quite well, I think.
    That sounds about right.
    Yes, though I think that plugin won't work with RabbitMQ 2.3.1. ?Maybe
    it could be revived...
    If you guys can provide more information on this, I'd be very interested.
    Thank you.
  • Yoav glazner at Mar 2, 2011 at 9:35 pm

    On Wed, Mar 2, 2011 at 7:24 PM, Fabio Margarido wrote:
    On Wed, Mar 2, 2011 at 14:12, Alexis Richardson wrote:
    Or failing that, perhaps a bit like the "real time record" model, such
    as used for financial data, where a consumer gets the last published update
    for a topic. IIRC the guys at Rabbit HQ did some work on an exchange which
    persists the last update to a topic - what became of that? This would fit
    the problem definition quite well, I think.
    you can make one queue named - conifg_data
    and always call basic_reject() on new config messages (with newer versions
    or such...)
    on old config messages call basic_ack and they will be gone...

    don't forget to make the queue durable and the messages persistent if that
    is needed.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110302/e38f1b9d/attachment.htm>
  • Fabio Margarido at Mar 3, 2011 at 11:06 am

    On Wed, Mar 2, 2011 at 18:35, yoav glazner wrote:
    you can make one queue named - conifg_data
    and always call basic_reject() on new config messages (with newer versions
    or such...)
    on old config messages call basic_ack and they will be gone...
    don't forget to make the queue durable and the messages?persistent?if that
    is needed.
    Thanks for the suggestion, Yoav. I'd rather have a more elegant
    solution, but I'll keep this in mind and give it a try if nothing else
    works as desired.
  • Michael Bridgen at Mar 3, 2011 at 1:02 am

    [..] IIRC the guys at Rabbit HQ did
    some work on an exchange which persists the last update to a
    topic - what became of that? This would fit the problem
    definition quite well, I think.
    The rabbitmq-lvc-plugin gives you an exchange type that acts like a
    direct exchange; but when you bind a queue, the queue gets the most
    recent message for the binding key given before being routed new messages.

    http://github.com/squaremo/rabbitmq-lvc-plugin


    Michael
  • Fabio Margarido at Mar 3, 2011 at 11:20 am

    On Wed, Mar 2, 2011 at 22:02, Michael Bridgen wrote:
    The rabbitmq-lvc-plugin gives you an exchange type that acts like a direct
    exchange; but when you bind a queue, the queue gets the most recent message
    for the binding key given before being routed new messages.

    http://github.com/squaremo/rabbitmq-lvc-plugin
    This sounds really interesting. Is this the plugin Alister and Alexis
    were talking about? Is it actively developed (or considered
    feature-complete) and considered stable?
    I've tried to get it up and running to give it a try but I'm stuck.
    Following the build instructions in the github page led to the
    following error when I ran (cd ../rabbitmq-erlang-client ; make clean
    && make):

    escript ../rabbitmq-server/generate_deps deps.mk ebin
    escript: exception error: no function clause matching
    generate_deps__escript__1299__149734__266135:main(["deps.mk",
    "ebin"])
    in function escript:run/2
    in call from escript:start/1
    in call from init:start_it/1
    in call from init:start_em/1
    sed -e 's:%%VSN%%:0.0.0:g' < rabbit_common.app.in > rabbit_common.app
    mkdir -p dist
    rm -f dist/rabbit_common.ez
    make -C ../rabbitmq-server
    make: *** [dist/rabbit_common.ez] Error 2

    If I skip these steps:

    (cd rabbitmq-server ; hg up -C bug22169 ; make -j)
    (cd ../rabbitmq-erlang-client ; make clean && make)

    This is the error I get when trying to compile the plugin:

    ERL_LIBS=build/deps:deps erlc -I include -o ebin -Wall +debug_info -pa
    ebin src/rabbit_exchange_type_lvc.erl
    src/rabbit_exchange_type_lvc.erl:21: field routing_key undefined in
    record basic_message
    src/rabbit_exchange_type_lvc.erl:27: variable 'RK' is unbound
    make: *** [ebin/rabbit_exchange_type_lvc.beam] Error 1

    Any thoughts?
    Thanks.
  • Michael Bridgen at Mar 3, 2011 at 12:58 pm

    On 03/03/2011 11:20 AM, Fabio Margarido wrote:
    On Wed, Mar 2, 2011 at 22:02, Michael Bridgenwrote:
    The rabbitmq-lvc-plugin gives you an exchange type that acts like a direct
    exchange; but when you bind a queue, the queue gets the most recent message
    for the binding key given before being routed new messages.

    http://github.com/squaremo/rabbitmq-lvc-plugin
    This sounds really interesting. Is this the plugin Alister and Alexis
    were talking about?
    I think so, although I'm surprised Alexis didn't know its name ..
    Is it actively developed (or considered
    feature-complete) and considered stable?
    Nope; have a look at the README --
    https://github.com/squaremo/rabbitmq-lvc-plugin/blob/master/README.md

    It's labeled "experimental", though it's dead simple and there are few
    moving parts. The biggest limitation you may face, *may* face, is that
    it's not written to be hugely scalable -- it just does the simplest
    thing possible.

    I've uploaded an ez on github, which you can drop into
    rabbitmq-server/plugins/; it should work with v2.3.1 and default branch.
    It'll be easier than compiling it yourself.
    I've tried to get it up and running to give it a try but I'm stuck.
    Following the build instructions in the github page led to the
    following error when I ran (cd ../rabbitmq-erlang-client ; make clean
    && make):

    escript ../rabbitmq-server/generate_deps deps.mk ebin
    escript: exception error: no function clause matching
    generate_deps__escript__1299__149734__266135:main(["deps.mk",
    "ebin"])
    in function escript:run/2
    in call from escript:start/1
    in call from init:start_it/1
    in call from init:start_em/1
    sed -e 's:%%VSN%%:0.0.0:g'< rabbit_common.app.in> rabbit_common.app
    mkdir -p dist
    rm -f dist/rabbit_common.ez
    make -C ../rabbitmq-server
    make: *** [dist/rabbit_common.ez] Error 2
    Hmm. I thought that build problem had been eradicated. Maybe it's
    because of the old branch.
    If I skip these steps:

    (cd rabbitmq-server ; hg up -C bug22169 ; make -j)
    (cd ../rabbitmq-erlang-client ; make clean&& make)
    Oh -- you don't need to compile to a particular branch any more. I've
    fixed the README.
    This is the error I get when trying to compile the plugin:

    ERL_LIBS=build/deps:deps erlc -I include -o ebin -Wall +debug_info -pa
    ebin src/rabbit_exchange_type_lvc.erl
    src/rabbit_exchange_type_lvc.erl:21: field routing_key undefined in
    record basic_message
    src/rabbit_exchange_type_lvc.erl:27: variable 'RK' is unbound
    make: *** [ebin/rabbit_exchange_type_lvc.beam] Error 1
    That's a consequence of the problem with compiling
    rabbitmq-erlang-client, above.


    Michael
  • Matthew Sackman at Mar 3, 2011 at 1:11 pm

    On Thu, Mar 03, 2011 at 12:58:54PM +0000, Michael Bridgen wrote:
    This is the error I get when trying to compile the plugin:

    ERL_LIBS=build/deps:deps erlc -I include -o ebin -Wall +debug_info -pa
    ebin src/rabbit_exchange_type_lvc.erl
    src/rabbit_exchange_type_lvc.erl:21: field routing_key undefined in
    record basic_message
    src/rabbit_exchange_type_lvc.erl:27: variable 'RK' is unbound
    make: *** [ebin/rabbit_exchange_type_lvc.beam] Error 1
    I doubt that. #basic_message's routing_key became routing_keys when
    bug23483 landed.

    Matthew
  • Matthew Sackman at Mar 3, 2011 at 1:23 pm

    On Thu, Mar 03, 2011 at 01:11:55PM +0000, Matthew Sackman wrote:
    I doubt that. #basic_message's routing_key became routing_keys when
    bug23483 landed.
    Ahh but of course that'll only affect things if you're compiling the
    latest default of the broker too. If you use the 2.3.1 broker with the
    ez Mike uploaded, that should be fine.

    Matthew
  • Fabio Margarido at Mar 3, 2011 at 2:41 pm

    On Thu, Mar 3, 2011 at 09:58, Michael Bridgen wrote:
    This sounds really interesting. Is this the plugin Alister and Alexis
    were talking about?
    I think so, although I'm surprised Alexis didn't know its name .. Great!
    It's labeled "experimental", though it's dead simple and there are few
    moving parts. ?The biggest limitation you may face, *may* face, is that it's
    not written to be hugely scalable -- it just does the simplest thing
    possible.
    I don't think scalability will be a big issue. There shouldn't be many
    messages delivered through this exchange.
    I've uploaded an ez on github, which you can drop into
    rabbitmq-server/plugins/; it should work with v2.3.1 and default branch.
    ?It'll be easier than compiling it yourself.
    That should make things easier, thank you very much. I'm not able to
    download it though. Clicking the link brings this not very helpful
    message: AccessDeniedAccess
    Denied3E08616583BF12BFHF2krVMH53i4Ea8SXPND7pCE4VC3lGdOcDM0dhg96BFcax9OJ7BOBLaRHAUhU53i.
    Thanks again.
  • Michael Bridgen at Mar 3, 2011 at 3:55 pm

    I've uploaded an ez on github, which you can drop into
    rabbitmq-server/plugins/; it should work with v2.3.1 and default branch.
    It'll be easier than compiling it yourself.
    That should make things easier, thank you very much. I'm not able to
    download it though. Clicking the link brings this not very helpful
    message: AccessDeniedAccess
    Denied3E08616583BF12BFHF2krVMH53i4Ea8SXPND7pCE4VC3lGdOcDM0dhg96BFcax9OJ7BOBLaRHAUhU53i.
    Oh. Looks like github's upload and download system is totally broken.
    (I've now tried to upload it a few times).

    I've attached the ez so it can be stopped by mail filters instead.

    mkb
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: rabbit_lvc_plugin.ez
    Type: application/andrew-inset
    Size: 7526 bytes
    Desc: not available
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110303/4f34b9fc/attachment-0001.ez>
  • Fabio Margarido at Mar 3, 2011 at 6:36 pm

    On Thu, Mar 3, 2011 at 12:55, Michael Bridgen wrote:
    Oh. ?Looks like github's upload and download system is totally broken. (I've
    now tried to upload it a few times).

    I've attached the ez so it can be stopped by mail filters instead.
    Hi Michael.
    Sorry for the extra trouble.

    I've now rebooted my RabbitMQ server and run a few basic tests and
    everything seems to be working as expected. Thank you again for all
    your help and your work on the plugin. I'll keep building my test
    scenarios and I'll let you know if something else comes up.

    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code, but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers) plans?
    Best regards.
  • Michael Bridgen at Mar 4, 2011 at 1:07 am
    Fabio,

    Aces, glad to hear it's working for you thus far.

    On this:
    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code, but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers) plans?
    Best regards.
    -- I was referring to the exchange plugin mechanism overall, on which
    I was finishing (largely other people's!) work at the time. That
    mechanism has been in mainstream rabbitmq since then, and has changed a
    bit, but is pretty stable.

    The Last Value Caching exchange will likely stay a plugin for the
    foreseeable future, rather than being baked into rabbitmq-server. But
    if you follow this list you'll know this is not the last word ..

    Michael
    Oh. Looks like github's upload and download system is totally broken. (I've
    now tried to upload it a few times).

    I've attached the ez so it can be stopped by mail filters instead.
    Hi Michael.
    Sorry for the extra trouble.

    I've now rebooted my RabbitMQ server and run a few basic tests and
    everything seems to be working as expected. Thank you again for all
    your help and your work on the plugin. I'll keep building my test
    scenarios and I'll let you know if something else comes up.

    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code, but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers) plans?
    Best regards.
  • Fabio Margarido at Mar 10, 2011 at 1:36 pm
    Hi Michael.

    I've worked a bit more on my tests by now and another doubt has come
    up. Is there any way to make messages exchanged through x-lvc
    persistent? The scenario I'm exercising is: the publisher sends a
    message to the broker, it is consumed a few times, the broker dies and
    starts up again, then a few new consumers try to receive the message.
    I can't get this to work yet.
  • Fabio Margarido at Mar 17, 2011 at 7:55 pm
    Hi Michael,

    sorry to keep bothering you, but do you have a word on this last question?
    Thank you.
    On Thu, Mar 10, 2011 at 10:36, Fabio Margarido wrote:

    Hi Michael.

    I've worked a bit more on my tests by now and another doubt has come
    up. Is there any way to make messages exchanged through x-lvc
    persistent? The scenario I'm exercising is: the publisher sends a
    message to the broker, it is consumed a few times, the broker dies and
    starts up again, then a few new consumers try to receive the message.
    I can't get this to work yet.

    From what I've seen in my tests, the exchange only works with
    exclusive queues, is that correct? I've tried creating a named,
    durable queue for all consumers to read from, but in this case the
    message can only be consumed once.

    Is there something I'm missing?

    Thank you.
    On Thu, Mar 3, 2011 at 22:07, Michael Bridgen wrote:
    Fabio,

    Aces, glad to hear it's working for you thus far.

    On this:
    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code, but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers) plans?
    Best regards.
    -- I was referring to the exchange plugin mechanism overall, on which I was
    finishing (largely other people's!) work at the time. That mechanism has
    been in mainstream rabbitmq since then, and has changed a bit, but is pretty
    stable.

    The Last Value Caching exchange will likely stay a plugin for the
    foreseeable future, rather than being baked into rabbitmq-server. But if
    you follow this list you'll know this is not the last word ..

    Michael
    Oh. Looks like github's upload and download system is totally broken.
    (I've
    now tried to upload it a few times).

    I've attached the ez so it can be stopped by mail filters instead.
    Hi Michael.
    Sorry for the extra trouble.

    I've now rebooted my RabbitMQ server and run a few basic tests and
    everything seems to be working as expected. Thank you again for all
    your help and your work on the plugin. I'll keep building my test
    scenarios and I'll let you know if something else comes up.

    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code, but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers) plans?
    Best regards.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110317/ad98902d/attachment-0001.htm>
  • Michael Bridgen at Mar 18, 2011 at 4:31 pm
    Fabio,
    I've worked a bit more on my tests by now and another doubt has come
    up. Is there any way to make messages exchanged through x-lvc
    persistent? The scenario I'm exercising is: the publisher sends a
    message to the broker, it is consumed a few times, the broker dies
    and starts up again, then a few new consumers try to receive the
    message. I can't get this to work yet.
    Right, at the minute the last values are transient. It probably wouldn't
    take much coding to make it save things to disk for durable exchanges.
    From what I've seen in my tests, the exchange only works with
    exclusive queues, is that correct? I've tried creating a named,
    durable queue for all consumers to read from, but in this case the
    message can only be consumed once. On Thu, Mar 10, 2011 at 10:36,
    Fabio Margarido <fabiomargarido at gmail.com
    wrote:
    It will work with any queue. However, if you have many consumers to a
    queue, each message will be delivered to one consumer only -- that's the
    semantics of AMQP queues. So, in your case, I think you want a queue
    for each consumer anyway.

    Michael
    I've worked a bit more on my tests by now and another doubt has come
    up. Is there any way to make messages exchanged through x-lvc
    persistent? The scenario I'm exercising is: the publisher sends a
    message to the broker, it is consumed a few times, the broker dies
    and starts up again, then a few new consumers try to receive the
    message. I can't get this to work yet.

    From what I've seen in my tests, the exchange only works with
    exclusive queues, is that correct? I've tried creating a named,
    durable queue for all consumers to read from, but in this case the
    message can only be consumed once.

    Is there something I'm missing?

    Thank you.

    On Thu, Mar 3, 2011 at 22:07, Michael Bridgen <mikeb at rabbitmq.com
    wrote:
    Fabio,

    Aces, glad to hear it's working for you thus far.

    On this:
    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code,
    but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers)
    plans?
    Best regards.
    -- I was referring to the exchange plugin mechanism overall, on
    which I was
    finishing (largely other people's!) work at the time. That
    mechanism has
    been in mainstream rabbitmq since then, and has changed a bit,
    but is pretty
    stable.

    The Last Value Caching exchange will likely stay a plugin for the
    foreseeable future, rather than being baked into rabbitmq-server. But if
    you follow this list you'll know this is not the last word ..

    Michael
    Oh. Looks like github's upload and download system is totally
    broken.
    (I've now tried to upload it a few times).

    I've attached the ez so it can be stopped by mail filters
    instead.
    Hi Michael. Sorry for the extra trouble.

    I've now rebooted my RabbitMQ server and run a few basic tests
    and everything seems to be working as expected. Thank you again
    for all your help and your work on the plugin. I'll keep building
    my test scenarios and I'll let you know if something else comes
    up.

    Just a final question: the previous README file on github said
    something about including the plugin on the main RabbitMQ code,
    but I
    figured this had been written some time ago because it mentioned
    version 1.7.1. Is this still in your (and RabbitMQ developers)
    plans?
    Best regards.
  • Fabio Margarido at Mar 25, 2011 at 2:25 pm
    Hi Michael.
    On Fri, Mar 18, 2011 at 13:31, Michael Bridgen wrote:

    Right, at the minute the last values are transient. It probably wouldn't
    take much coding to make it save things to disk for durable exchanges.
    OK, that figures... :) I know nothing of Erlang, but I'll take an overall
    look and try to come up with an acceptable solution for my case. Would that
    be a feature useful enough that you'd consider including in your code?

    It will work with any queue. However, if you have many consumers to a queue,
    each message will be delivered to one consumer only -- that's the semantics
    of AMQP queues. So, in your case, I think you want a queue for each
    consumer anyway.

    Sorry for not being clearer, that's what I meant by 'exclusive' queues, not
    the exclusive on AMQP terminology.
    Thank you.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110325/fcb943fd/attachment-0001.htm>
  • Michael Bridgen at Mar 29, 2011 at 11:35 am
    Fabio,
    Right, at the minute the last values are transient. It probably
    wouldn't take much coding to make it save things to disk for durable
    exchanges.


    OK, that figures... :) I know nothing of Erlang, but I'll take an
    overall look and try to come up with an acceptable solution for my case.
    Would that be a feature useful enough that you'd consider including in
    your code?
    Sure! Here's a sketch: use two mnesia tables, one ram only, for
    non-durable exchanges, and one on disk, for durable exchanges. In a
    single node this will be relatively easy; if you use a cluster, you'll
    need to find out which nodes have on-disk state.


    Michael
  • Fabio Margarido at Apr 19, 2011 at 3:04 pm
    Hi Michael,

    it's been a long time since our last email exchange, and I've still not been
    able to dedicate some time to this issue. Things are a little too hectic
    right now for me, and I suspect the learning curve involved would make it
    event more complicated.

    If you could consider adding support for this feature whenever you have
    spare time to dedicate to this project, I'd be really grateful. Otherwise I
    guess I'll just have to put it indefinitely on hold... :/

    Thank you.
    On Tue, Mar 29, 2011 at 08:35, Michael Bridgen wrote:

    Fabio,


    Right, at the minute the last values are transient. It probably
    wouldn't take much coding to make it save things to disk for durable
    exchanges.


    OK, that figures... :) I know nothing of Erlang, but I'll take an
    overall look and try to come up with an acceptable solution for my case.
    Would that be a feature useful enough that you'd consider including in
    your code?
    Sure! Here's a sketch: use two mnesia tables, one ram only, for
    non-durable exchanges, and one on disk, for durable exchanges. In a single
    node this will be relatively easy; if you use a cluster, you'll need to find
    out which nodes have on-disk state.



    Michael
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110419/6c7d92f5/attachment-0001.htm>
  • Michael Bridgen at Apr 20, 2011 at 3:03 pm
    Hi Fabio,
    it's been a long time since our last email exchange, and I've still not
    been able to dedicate some time to this issue. Things are a little too
    hectic right now for me, and I suspect the learning curve involved would
    make it event more complicated.

    If you could consider adding support for this feature whenever you have
    spare time to dedicate to this project, I'd be really grateful.
    Otherwise I guess I'll just have to put it indefinitely on hold... :/
    Hectic for me too .. and LVC exchange is not currently on the rabbit
    roadmap, such as it is. So it may be some time, I'm afraid.

    You may wish to look at Jon Brisbin's experiments with Riak that have
    been mentioned here, in the meantime.

    Kind regards,
    Michael
    On Tue, Mar 29, 2011 at 08:35, Michael Bridgen <mikeb at rabbitmq.com
    wrote:

    Fabio,


    Right, at the minute the last values are transient. It probably
    wouldn't take much coding to make it save things to disk for
    durable
    exchanges.


    OK, that figures... :) I know nothing of Erlang, but I'll take an
    overall look and try to come up with an acceptable solution for
    my case.
    Would that be a feature useful enough that you'd consider
    including in
    your code?


    Sure! Here's a sketch: use two mnesia tables, one ram only, for
    non-durable exchanges, and one on disk, for durable exchanges. In a
    single node this will be relatively easy; if you use a cluster,
    you'll need to find out which nodes have on-disk state.



    Michael
    _______________________________________________
    rabbitmq-discuss mailing list
    rabbitmq-discuss at lists.rabbitmq.com
    <mailto:rabbitmq-discuss at lists.rabbitmq.com>
    https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedMar 2, '11 at 2:14p
activeApr 20, '11 at 3:03p
posts27
users6
websiterabbitmq.com
irc#rabbitmq

People

Translate

site design / logo © 2022 Grokbase