Reading through
http://rubydoc.info/github/ruby-amqp/amqp/master/file/docs/ConnectingToTheBroker.textile
and I noticed when using the AMQP gem with Unicorn you must:

/"start the EventMachine reactor and AMQP connection after the master
process forks workers"/

We've been using Unicorn for some time now so I'm aware of
reestablishing connections after Unicorn forks, however I have some
questions in regards to AMQP. Do I simply need to start up AMQP in a
separate thread as in the example:

Thread.new { AMQP.start }

Then in my controllers I would need to do everything inside a next_tick
block?

EventMachine.next_tick do
...
end


Is this all that is required? How come in the example we are connecting
to a channel/queue in the after_fork hook? Is this for debugging purposes?

I also have one other question regarding EventMachine#next_tick. I've
read that:

/"Everything scheduled using EM#next_tick will happen synchronously in
the main thread. Any long running tasks inside an EM#next_tick block
will cause the entire program to block until complete. Typically a bad
thing."

/Does this mean we don't gain anything by using EM in this fashion.. ie
our attempt to use AMQP asynchronously is actually synchronous?

Thanks for the clarification!

- M




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120117/5e6f7e48/attachment.htm>

Search Discussions

  • Michael Klishin at Jan 17, 2012 at 6:10 pm

    Mark:

    Is this all that is required?
    Yes. In 0.9.0 (possibly in 0.8.4 as well) there is a helper that reads config/amqp.yml similarly to config/database.yml:

    require "amqp/utilities/event_loop_helper"
    require "amqp/integration/rails"

    # this helper detects what app server (if any) is being used and starts EventMachine reactor in a separate
    # thread or just uses .next_tick (for example, with Thin).
    AMQP::Utilities::EventLoopHelper.run

    # reads config/amqp.yml, takes environment-specific key out of it (e.g. :development) and
    # merges whatever you pass to .start with it.
    AMQP::Integration::Rails.start do |connection|
    puts "Connected to AMQP broker"
    # this is just an accessor for 1 channel. It has no special properties, just a placeholder for
    # applications that only need one channel
    AMQP.channel = AMQP::Channel.new(connection)
    end

    I would like to try to make it detect Unicorn forking, that's why it is not not yet in the documentation: it may slightly change
    for 1.0. Your suggestions about what you want it to do are very welcome.

    How come in the example we are connecting to a channel/queue in the after_fork hook? Is this for debugging purposes?
    Channels are typically opened when applications boot. Messages are published there just so that people can see messages flow.
    More on what role channels play: http://rubyamqp.info/articles/amqp_9_1_model_explained/#amqp_channels

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin
  • Mark at Jan 17, 2012 at 7:33 pm
    Thanks for clearing this up. Ill have to check out those integration
    classes you mentioned.

    Question... typically only 1 channel is needed per application right?
    On 1/17/12 10:10 AM, Michael Klishin wrote:
    Mark:
    Is this all that is required?
    Yes. In 0.9.0 (possibly in 0.8.4 as well) there is a helper that reads config/amqp.yml similarly to config/database.yml:

    require "amqp/utilities/event_loop_helper"
    require "amqp/integration/rails"

    # this helper detects what app server (if any) is being used and starts EventMachine reactor in a separate
    # thread or just uses .next_tick (for example, with Thin).
    AMQP::Utilities::EventLoopHelper.run

    # reads config/amqp.yml, takes environment-specific key out of it (e.g. :development) and
    # merges whatever you pass to .start with it.
    AMQP::Integration::Rails.start do |connection|
    puts "Connected to AMQP broker"
    # this is just an accessor for 1 channel. It has no special properties, just a placeholder for
    # applications that only need one channel
    AMQP.channel = AMQP::Channel.new(connection)
    end

    I would like to try to make it detect Unicorn forking, that's why it is not not yet in the documentation: it may slightly change
    for 1.0. Your suggestions about what you want it to do are very welcome.

    How come in the example we are connecting to a channel/queue in the after_fork hook? Is this for debugging purposes?
    Channels are typically opened when applications boot. Messages are published there just so that people can see messages flow.
    More on what role channels play: http://rubyamqp.info/articles/amqp_9_1_model_explained/#amqp_channels

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin
  • Michael Klishin at Jan 17, 2012 at 7:42 pm

    Mark:

    Question... typically only 1 channel is needed per application right?
    For Web apps that just publish messages and never consume them, yes.

    For consumer applications, you almost always end up with multiple channels because
    channels should not be shared between threads and error handling in AMQP 0.9.1 is per-channel (connection-level exceptions usually
    indicate misconfiguration or serious client bugs). Another common reason for separation is to be able to see separate message flows
    in the RabbitMQ Web UI.

    You can always start with 1 channel (this is why AMQP.channel accessor is there, so you don't have to decide where to assign it) and add more,
    often moving to this technique:

    http://rubyamqp.info/articles/getting_started/#integration_with_objects

    but It is not uncommon to see publishing being done as

    AMQP.channel.default_exchange.publish(payload, ?)

    initially. This is not discouraged.

    Channels on the wire are just integers every message carries so they are very cheap to create and maintain in terms of resources,
    both for clients and RabbitMQ itself.

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin
  • Mark at Jan 18, 2012 at 1:03 am
    Do I need both statements?

    AMQP::Utilities::EventLoopHelper.run

    AMQP::Integration::Rails.start do |connection|
    AMQP.channel = AMQP::Channel.new(connection)
    end
    On 1/17/12 10:10 AM, Michael Klishin wrote:
    Mark:
    Is this all that is required?
    Yes. In 0.9.0 (possibly in 0.8.4 as well) there is a helper that reads config/amqp.yml similarly to config/database.yml:

    require "amqp/utilities/event_loop_helper"
    require "amqp/integration/rails"

    # this helper detects what app server (if any) is being used and starts EventMachine reactor in a separate
    # thread or just uses .next_tick (for example, with Thin).
    AMQP::Utilities::EventLoopHelper.run

    # reads config/amqp.yml, takes environment-specific key out of it (e.g. :development) and
    # merges whatever you pass to .start with it.
    AMQP::Integration::Rails.start do |connection|
    puts "Connected to AMQP broker"
    # this is just an accessor for 1 channel. It has no special properties, just a placeholder for
    # applications that only need one channel
    AMQP.channel = AMQP::Channel.new(connection)
    end

    I would like to try to make it detect Unicorn forking, that's why it is not not yet in the documentation: it may slightly change
    for 1.0. Your suggestions about what you want it to do are very welcome.

    How come in the example we are connecting to a channel/queue in the after_fork hook? Is this for debugging purposes?
    Channels are typically opened when applications boot. Messages are published there just so that people can see messages flow.
    More on what role channels play: http://rubyamqp.info/articles/amqp_9_1_model_explained/#amqp_channels

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin
  • Michael Klishin at Jan 18, 2012 at 6:26 am

    Mark:

    Do I need both statements?

    AMQP::Utilities::EventLoopHelper.run

    AMQP::Integration::Rails.start do |connection|
    AMQP.channel = AMQP::Channel.new(connection)
    end
    Yes. You need to both start EventMachine (this part depends on application server, hence the helper) and
    connect to the broker, then open channel(s) you need.

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin
  • Michael Klishin at Jan 17, 2012 at 6:16 pm

    Mark:

    I also have one other question regarding EventMachine#next_tick. I've read that:

    "Everything scheduled using EM#next_tick will happen synchronously in the main thread. Any long running tasks inside an EM#next_tick block will cause the entire program to block until complete. Typically a bad thing."

    Does this mean we don't gain anything by using EM in this fashion.. ie our attempt to use AMQP asynchronously is actually synchronous?
    That piece of (EventMachine) documentation tries to say that EventMachine.next_tick always runs callbacks on the event loop thread (it does not necessary
    have to be main thread. With Unicorn, Puma or any Web server that is not Thin, it won't be main). So you need to be careful to not block it. Web apps rarely
    do anything that may do that.

    EventMachine.next_tick used to delay any operations that use EventMachine until after event loop is ready. That's is all. It does not make amqp gem
    synchronous. It just says "hey, EventMachine. I am not sure if you are running already, but when you are, run this code. Kthxbai".

    So, to reiterate: with Unicorn, main Ruby VM thread is used by Unicorn itself. EventMachine is started in another thread. All network activity caused by amqp gem
    always happens on that thread. In the context of Web apps that usually only publish events and commands, you typically never have to worry about it.

    MK

    http://github.com/michaelklishin
    http://twitter.com/michaelklishin

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprabbitmq-discuss @
categoriesrabbitmq
postedJan 17, '12 at 5:40p
activeJan 18, '12 at 6:26a
posts7
users2
websiterabbitmq.com
irc#rabbitmq

2 users in discussion

Michael Klishin: 4 posts Mark: 3 posts

People

Translate

site design / logo © 2022 Grokbase