[Catalyst] Push services with catalyst

If it's AJAX then you are really polling as opposed to pushing. In
this case Lighttpd or Nginx are your friends. They can handle all of
the connections (with a long KeepAlive) and only talk to the backend
FastCGI when a poll request comes through from the AJAX client.

If you're really pushing with a constantly connected client, you'll
need a FastCGI process per client. The difference is who initiates
communication over the channel. If the server can send to the client
at any moment, that's true push. Most people get along with a rapidly
polling client that says "Anything for me?" every so often.

In general polling is wasteful of bandwidth, while push is wasteful of
server-side resources.

Best,
Brian

I'm combining ajax with push. So it's a real push service, the website
is not pulling the data on a constant time rate.

A process per client is really bad. I'll need to serve hundreds of
client at the same time. Seems like push is not an option...

Am 05.03.2008 um 19:39 schrieb Brian Kirkbride:

Take a look at cometd: http://cometd.com/

Thanks for mentioning that, it looks interesting, particularly as it's
hooked up to perlbal.

Moritz, I've done something similar before using POE to handle many
connections more cheaply than Apache (thanks to Matt Trout for the
suggestion). It was to serve XML formatted requests though you could return
JSON formatted data just as easily
http://search.cpan.org/perldoc?POE::Component::Server::TCP

Regards, Peter
http://perl.dragonstaff.co.uk

But I still need a lot of code which is run by catalyst, like
authentication and of course the whole dbic schema.
What about Catalyst::Engine::HTTP::POE?

cometd.com is not very informative ...

Am 05.03.2008 um 23:47 schrieb Peter Edwards:

In the article http://www.irishdev.com/NewsArticle.aspx?id!66
see that diagram of an SOA style architecture?
http://alex.dojotoolkit.org/wp-content/Comet.png

Browser UI = HTML + Javascript

Comet client = Javascript driven by timed events

talks persistent TCP to

Comet event bus = proxy server, e.g. perlbal or nginx or POE TCP daemon

talks HTTP or XML-RPC to

Server-side processing = your Catalyst app


You need to split the management of the many persistent TCP connections from
the generation of content.

In a typical web server setup you achieve this by using a scalable caching
proxy front end to do the TCP socket handling and a Catalyst backend for the
content.

In your case you might need something a bit smarter in the front end, hence
my mention of a custom POE daemon. Or you could write a plugin for perlbal
http://search.cpan.org/dist/Perlbal/.

You could make an XML-RPC or HTTP request from the front end to Catalyst to
authenticate, and thereafter run periodic front to backend GETs and cache
them (in the "comet event bus" part) ready to serve to the comet client via
TCP.

That way you don't need to tie up a 30MB Catalyst process just to hold open
a socket, are spreading the request load evenly across time and can write it
all in Perl.

Alternatively, if you have an enormous load the for the proxy you could use
nginx and write a C module to call out to your backend
http://www.riceonfire.org/emiller/nginx-modules-guide.html


Regards, Peter
http://perl.dragonstaff.co.uk

Theoretically this should work. When your application is in the state
of doing IO (sending data to the client, holding a socket open), other
requests can run. That sounds like it is what you want.

There are things that will block POE, though; handling a request of
course, as well as things that are technically IO (waiting for your
database to return a query). That can be fixed, though.

Regards,
Jonathan Rockway

If you're looking for in-depth discussion of Comet techniques I'd suggest
you have a look at cometdaily.com.

Cheers,

Patrick

Ah... after I clicked send I thought "what if it is a true PUSH using
Ajax to listen?" Most people think of Ajax polling as a push, though
it's just looks like push with some granularity. My apologies for
assuming...

Have you checked out COMET? See http://cometd.com/ for discussion of
all the issues with push. I haven't looked at it in a long time, but
it was an attempt to deal with your issue. The Lighttpd developers
are working on something similar called mod_mailbox. I don't pretend
to understand anything about either :)