[storm-user] storm and spout performance

[storm-user] storm and spout performance

	Tony
	Dec 5, 2012 at 11:53 am

Hi,

I have been reading about storm for a bit and I'm trying to search for its
performance. I'm curious if I have a stream of 20k qps I want to send to
storm, can 1 spout handle it ? basically what is the maximum qps a spout
can handle?

Thanks

Search Discussions

Discussion Posts

Follow ups

Michael Rose: That's all dependent on how tight your code is and how fast your source of data is, network conditions, etc. Generally for 20k qps I'd expect your spout would have a parallelism greater than 1 reading from multiple brokers. A single spout is horizontally scalable under storm (in most cases). Without more information about your expected use case, "queries", hardware, etc. nobody can tell you how Storm will perform for you. -- Michael Rose (@Xorlev) Senior Platform Engineer, FullContact [email
Tony: Hi Michael, Thanks for the reply... well I'm assuming that the hardware is very simple let's say M1 Medium Instance 3.75 GiB of memory, 2 EC2 from amazon. So really my question is this then. Let's take the twitter word counter example and let's imagine the twitter feed gives 50 000 qps. From my understanding, you can only have 1 spout listening to that socket and can't be threaded to prevent duplicates otherwise if we have threads this would create multiple connections. What I would like to

Related Discussions

Discussion Navigation

view	thread \| post
posts	‹ prev \| 1 of 3 \| next ›

Discussion Overview

group	storm-user @
posted	Dec 5, '12 at 11:53a
active	Dec 7, '12 at 11:46a
posts	3
users	2
website	storm-project.net
irc	#storm-user

font	variable	Hotkey: f
user style	avatars	Hotkey: a

[storm-user] storm and spout performance

Search Discussions

Discussion Posts

Related Discussions

2 users in discussion