Search Discussions
-
I have been trying to implement a Kafka+Storm system using KafkaSpout. I seem to have run into compatibility issues among the different versions. I wanted to know which would be the best combination ...
Nitin Jain
Jul 24, 2013 at 7:10 pm
Dec 12, 2013 at 7:43 am -
Hi, You'll see attached a schema of the topology we're creating. It's quite complex as it's intended to store different kinds of data using different sources. The goal is to store pre-calculated ...
Laurent
Jul 3, 2013 at 9:01 am
Oct 30, 2013 at 8:35 am -
Hi. I have incoming stream of events and I need to aggregate events based on time window (for example 1 minute). I know that there is tick tuple but It is not suitable for me. I need to aggregate the ...
Oleg Ruchovets
Jul 17, 2013 at 4:40 pm
Jul 23, 2013 at 8:34 pm -
Hi, We’ve been feeling like our Storm cluster throughput was running slower than it could. We currently are seeing throughput of around 2-3K message/second/node in our production cluster. In doing a ...
Uniquename70
Jul 9, 2013 at 11:28 am
Jul 14, 2013 at 3:58 am -
hello, we have set up storm using michael-noll's tutorial. the zookeeper and the nimbus are on one node and there is another node for worker(supervisor). the zookeeper and nimbus runs fine the, ...
Muhammad Arslan Arshad Ch.
Jul 23, 2013 at 6:50 pm
Jul 29, 2013 at 7:06 am -
hi all: i wrote a topology to aggregate some log files and output a result file(csv). it works fine in my local cluster. but when i submit to the cluster. the topology doesn't output any file to the ...
Joey Chang
Jul 30, 2013 at 9:55 am
Jul 31, 2013 at 3:18 pm -
I have developed an CEP system with rule engine that in nutshell captures all clickstream data as events, and based on some conditions in events, for example, user visit specific article on a site, ...
Eugene Dvorkin
Jul 20, 2013 at 2:50 pm
Jul 21, 2013 at 6:33 am -
We've got a trident topology, and are testing out various parallelism options. However, whenever we increase the parallelism of any section above 1, the topology fails to run on the cluster - ~2000 ...
Simon Cooper
Jul 25, 2013 at 12:06 pm
Aug 1, 2013 at 2:17 pm -
Hi, I am new to twitter storm, I have successfully run the topology so far, but I am wondering If I can specify the spout in one server. I basically have just one shufflegrouping, So, the number of ...
Jiang Jacky
Jul 18, 2013 at 7:17 pm
Jul 18, 2013 at 10:16 pm -
hi everyone, I have a problem when I want to set the workers's java.library.path as java.library.path+=":.",because i want to download a so from outerside and put it in the "." directory ,then load ...
Bo Cui
Jul 3, 2013 at 9:26 am
Jul 8, 2013 at 3:02 am -
I've attempted to move a call to cluster/StormClusterState#remove-worker-heartbeat! from the shutdown* function defined in worker/mk-worker to supervisor/try-cleanup-worker. What I am trying to ...
Derek
Jul 24, 2013 at 8:02 pm
Jul 25, 2013 at 9:21 pm -
Nathan: Some of our use cases process multiple incoming streams (ex. user query stream, and twitter feed stream) in a batch manner. Transactional topologies have been a nice abstraction for such use ...
Andy
Jul 21, 2013 at 8:54 pm
Jul 24, 2013 at 5:03 pm -
Hi all, I'm interested in using storm. I would like to work with the source instead of compiled jars. If I run into bugs, I'd like to be able to fix them and contribute back. I have a few questions ...
Ramsey Gurley
Jul 15, 2013 at 3:30 pm
Jul 17, 2013 at 4:26 am -
Hello, I've run into a critical issue deploying topologies. This began to occur on version 0.8.2 after several minutes of ZK downtime. After ZK cluster came back up, no topology has been able to stay ...
Homer Strong
Jul 1, 2013 at 7:29 am
Jul 2, 2013 at 3:14 am -
Hi, I'm using the LocalCluster class to start my topology. 9 out of 10 times the topology kickoff will break with this exception. At this point, the nimbus and supervisors are not running. There is ...
Premal Shah
Jul 18, 2013 at 7:55 am
Oct 24, 2013 at 10:57 pm -
Hello. I ran a Trident topology on a Storm cluster of 4 computers. One is the nimbus, the others got the zookeeper and the supervisor. After a day, the zookeeper data directory contains 8.5 Gb on the ...
JF Picard
Jul 23, 2013 at 1:39 pm
Jul 31, 2013 at 12:04 pm -
hi, I am trying to create Topology using Python.I have gone through storm-starter Example where I found WordCounterTopology. There SplitSentence Bolt uses Python language, WordCount Bolt is written ...
Avirup Hitk Ece
Jul 19, 2013 at 7:13 pm
Jul 26, 2013 at 2:36 am -
I have 6 partitions on each Topic in Kafka. How do you deploy a new set of code for a given Topology and not lose messages? As soon as i kill the topology and the redeploy it..all the messages/events ...
Ryan medlin
Jul 6, 2013 at 1:13 am
Jul 8, 2013 at 2:41 am -
Hi all, I think I finally understand the basic State API. I have implemented my topology using a basic state object and associated updater. But I'm intrigued by the possibilities of the MapState API, ...
Tom Brown
Jul 31, 2013 at 12:12 am
Nov 11, 2013 at 9:24 am -
I am evaluating the combination of Storm with Esper vs. TridentTopology to calculate aggregates. My use case is to count all the occurrences of a specific string in a file. I am planning to use bolts ...
Kishore Kopalle
Jul 21, 2013 at 9:09 am
Aug 13, 2013 at 1:37 pm -
Dear all, I've created a spout that retrieves data from a custom database. Each node in our cluster has its own database. I would like to create spouts where each spout (instance of a spout) executes ...
John Gilmore
Jul 31, 2013 at 7:01 pm
Aug 3, 2013 at 1:57 am -
hi, I am looking for a rule editor which will work on top of storm.End user can edit something in that rule editor and then submit it ,which will result in generation of storm code. Can You suggest a ...
Avirup Hitk Ece
Jul 22, 2013 at 6:17 am
Jul 28, 2013 at 6:51 pm -
Hi, I encounter a problem like this: two executors of different bolts are running in the same worker process, and suddenly they just stop running, it seems that storm stops calling both bolts' ...
Ji Zhang
Jul 21, 2013 at 8:17 am
Jul 25, 2013 at 11:26 pm -
We have a setup of three AWS 8-core supervisor nodes. But Storm sees only 12 slots available as opposed to 24 that should have been available. This can be verified from Storm UI as well as the number ...
Kishore Kopalle
Jul 24, 2013 at 11:00 am
Jul 25, 2013 at 4:16 am -
Hi, We're not sure we'll be able to use AWS in our production servers, we may be forced to use some other local hosting provider, hence we can't use https://github.com/nathanmarz/storm-deploy/wiki ...
Guy
Jul 14, 2013 at 7:44 pm
Jul 20, 2013 at 7:07 am -
Hello All, I was glad to join this group and explored many topics around streaming. I was wondering if any one in this group was evaluated Flume vs Storm ? if so,can you share some insights. I need ...
Murali0623
Jul 16, 2013 at 5:51 pm
Jul 18, 2013 at 8:46 am -
We're investigating implementing a system on storm, and are choosing between normal, transactional, and trident topologies. It looks like transactional topologies have been obsoleted in favour of ...
Simon Cooper
Jul 15, 2013 at 9:08 am
Jul 17, 2013 at 1:43 pm -
Hi everybody, I had a recent problem with supervisor: Topology was working for about 4 days everything was fine, but suddenly performance dropped down dramatically. I checked CPU usage of supervisor ...
Sergey
Jul 12, 2013 at 4:19 pm
Jul 14, 2013 at 4:00 am -
Is there a way of combining trident streams and standard storm bolts in a single topology? ie have a topology like this: TridentSpout - Stream - Bolt - Bolt - Stream The reason is that I want to have ...
Simon Cooper
Jul 3, 2013 at 10:12 pm
Jul 11, 2013 at 9:05 am -
We want to use storm-kafka integration in our finance company. We have different types of financial transactions, each of log message type is different. We planned to tail logs with KAFKA and then ...
Nbitkin
Jul 2, 2013 at 11:21 am
Jul 3, 2013 at 5:29 am -
Hey All, I'm running a topology and I've recently run into worker heartbeat timeouts. 2013-07-23 02:31:54 supervisor [INFO] Shutting down and clearing state for id ...
Drew Goya
Jul 23, 2013 at 6:50 pm
Oct 11, 2013 at 2:16 am -
I have a simple Storm topology I'm running in local mode that has two spouts, which each emit a stream that first each goes to its own preprocessing bolts (I'll call them A and B), and then both go ...
Maria Moresco
Jul 31, 2013 at 4:14 pm
Jul 31, 2013 at 5:56 pm -
hi all: i want to know how to abandon a invalid tuple if i used acker component. i just use "return" in one of my bolts like: public void excute(Tupe tuple) { .... if (tuple is invalid) {return;} ...
Joey Chang
Jul 30, 2013 at 1:55 pm
Jul 30, 2013 at 5:50 pm -
Hi All, I'm interested in writing a Storm backend to Pig. I've written an initial draft for a proposal that can be found at https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal I ...
Pradeep Gollakota
Jul 25, 2013 at 6:57 pm
Jul 27, 2013 at 5:20 am -
We've been running some basic topologies using storm 0.8.2 for a while, and we're finally switching to use Trident. It would be very nice if we could use the "dynamic brokers for trident" version of ...
Tom Brown
Jul 22, 2013 at 10:37 pm
Jul 24, 2013 at 4:08 pm -
I set two supervisors in different server,but one supervisor the used slots is 4, antoher is 0(seperate server), I checked supervisor log, no any serious errors, what's matter? -- You received this ...
Storm dev
Jul 19, 2013 at 5:42 am
Jul 20, 2013 at 7:16 am -
Hi, Everyone I can run the storm in my cluster. But When I run the storm UI, I get the following error: Exception in thread "main" java.net.BindException: Address already in use at ...
Jiang Jacky
Jul 17, 2013 at 10:19 pm
Jul 18, 2013 at 2:32 pm -
Hi everybody, I'm struggling implement one logic in storm: I need to process no more than N msgs/s. It's easy to achieve with one spout ( simple count number of tuples that been passed to the bolts, ...
Sergey
Jul 14, 2013 at 2:50 pm
Jul 15, 2013 at 1:21 pm -
I have various files who follow a particular pattern lie this pattern = somerandomealphabets_<node no. <seqno for example files will be of the form somerandomealphabets_100123_001, ...
Ayush choubey
Jul 9, 2013 at 6:00 am
Jul 14, 2013 at 4:16 am -
We are working on a video surveillance application that would need to process images in real time. We are considering using Storm as the processing platform for our intruder detection feature, which ...
Shweta
Jul 8, 2013 at 9:14 pm
Jul 9, 2013 at 8:42 pm -
So, i was running storm in cluster mode and found that to process certain data, it took me 9 mins. While seeing log file, i found that, it had too many system.out operations, so i decided to remove ...
Ayush choubey
Jul 8, 2013 at 5:50 pm
Jul 9, 2013 at 3:27 pm -
I am using storm 0.8.2 but it seems that most people are using the latest version of storm and storm kafka both wip versions. I see things that say kafka is buggy otherwise but I am not 100% sure. ...
Ryan medlin
Jul 7, 2013 at 4:19 am
Jul 9, 2013 at 7:06 am -
Hi, Excuse if its a newb question, I just stumbled upon this awesome framework recently and still learning the ropes. I got the following link - ...
Abraham menacherry
Jul 2, 2013 at 8:07 am
Jul 2, 2013 at 1:46 pm -
I've been working on integrating a kafka spout with storm 0.8.2 (the latest release). We would like to use Kafka 0.8; however, the "blessed" ...
Randy Buck
Jul 26, 2013 at 2:36 am
Sep 6, 2013 at 6:31 am -
Storm is distributed with a bundle of jars providing its dependencies. Among these are Clojure 1.4. If I replace the jar in the distribution with Clojure 1.5.1 and run my application that way, how ...
Michael Blume
Jul 6, 2013 at 8:42 pm
Aug 28, 2013 at 10:10 pm -
hi all: we need to count many metrics from a data stream; is it good use one topology for all the metrics or one topology for one metric? -- 不学习，不知道 -- You received this message because you are ...
Panfei
Jul 29, 2013 at 5:11 am
Aug 12, 2013 at 7:39 am -
2
[storm-user] java.lang.UnsatisfiedLinkError: sun/misc/Unsafe.putOrderedLong(Ljava/lang/Object;JJ)V
I am trying to run the sample storm example code:ExclamationTopology.java in Local mode, I have packaged this java file to a jar file and ran the below command /home/storm/storm-0.8.1/bin # storm jar ...Chandra sekhar
Jul 31, 2013 at 1:47 am
Aug 1, 2013 at 10:07 am -
What's the logic for deciding which nodes of a cluster run the tasks of a topology? I've tried to read through nimbus.clj, but unfortunately my clojure isn't up to scratch :( Is there any scope for ...
Simon Cooper
Jul 31, 2013 at 9:32 am
Jul 31, 2013 at 10:34 am -
Since I struggled a bit while setting this up for myself, I decided to go through it one more time and describe the procedure in a form of tutorial ...
Marko Bonaći
Jul 25, 2013 at 6:57 pm
Jul 30, 2013 at 8:03 pm -
Hi, I've yet met another strange behavior I can't explain or resolve so i'm posting it here hoping to get some insight on what i did wrong. I'm using Trident with storm-0.9.0-wip16. My topology's ...
Laurent
Jul 29, 2013 at 9:02 am
Jul 30, 2013 at 9:13 am
Group Overview
| group | storm-user |
| discussions | 169 |
| posts | 491 |
| users | 158 |
| website | storm-project.net |
| irc | #storm-user |
