Just an FYI, I'm currently working on a project to make it possible to run
Storm in an OSGI container. A very early version of the project is in


Currently, you can run the WordCountTopology sample using my fork of
storm-starter in an OSGI container, though Storm (at least using a
LocalCluster) doesn't seem to be a good citizen in that it will actually
shut down the JVM it's running in when a topology finishes. This of course
shuts the OSGI container down as well. Since Storm does it using the Java
Runtime halt() method, there isn't anyway to prevent it from shutting down
the JVM.

Over the short term, I'm trying to get the project into a state where Storm
will run in an OSGI container, behave and to work in non-local mode. It's
not an easy task due to the way Clojure works and the way Storm is packaged
(which gives my debugger heartburn and therefore it won't let me debug
anything contained in the Storm jar). I'm also planning to hook in a
couple features that Storm is currently lacking.

I need to figure out how to build Storm so I can make a fork that builds a
version of Storm that is debuggable and better behaved when coexisting with
other Java code. Keep checking back, it shouldn't be too much longer
before it should be actually usable. I need to deal with a few things in
Storm and Clojure that cause them not to cooperate. I'll try to get some
instructions added as well.

Just wanted to let you know that someone is on it and is making progress.

On Thursday, September 13, 2012 11:25:49 AM UTC-4, Rudy Bonefas wrote:

Heres why we think we need a jar of jars:

We building our app around Trident and we're going to let other groups
contribute Operations. Each groups Operations classes will be submitted to
us in a jar, along with their dependencies. So that different operators
submitted by different groups can operate together without encountering any
'jar hell' issues, i.e. collision of different dependency versions, my plan
was to have the submitted jars implement an OSGI Operation factory service
thus caching in on the OSGI benefit of isolated classloaders

If you don't want to support a jar of jars, is there a way that we could
provide our own class loader at the point where our operations are
instantiated on each supervisor?

On another note: How's your Big Data book coming? I had the chance to
review the first 4 chapters from Safari online. Looking forward to its

On Thursday, September 13, 2012 3:08:52 AM UTC-4, nathanmarz wrote:

I don't really see a need for it. It's not very hard to make uberjars
with all the classfiles in one jar.
On Tue, Sep 11, 2012 at 7:44 AM, Rudy Bonefas wrote:

I've seen some discussion on this in the forum. Particularly the use of
one-jar which seemed to fail on the Supervisor nodes. One-jar uses a
special class loader. How do the supervisor processes get launched and

Nathan, any chance of supporting one-jar or something like it in the
near future?

Twitter: @nathanmarz

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupstorm-user @
postedDec 7, '12 at 2:35a
activeDec 7, '12 at 2:35a

1 user in discussion

Ryan Moquin: 1 post



site design / logo © 2022 Grokbase