FAQ
Hi Nathan,


I have a problem: few node that run a topology crash because Storm goes in
out of memory.

I tried to set the High water mark limit of ZMQ queue in storm.yaml
(zmq.hwm) so that when the queue limit is reached the receiver or sender
should block. Unfortunately this solution doesn't work. Do you know why?

Thank you

- Daviode

Search Discussions

  • Vinh at Dec 18, 2012 at 6:57 pm
    Have you tried setting "topology.max.spout.pending" to limit the amount of pending tuples? Else, spouts will keep pushing tuples to the bolts, regardless of result.

    On Dec 18, 2012, at 10:50 AM, Davide Simoncelli wrote:

    Hi Nathan,


    I have a problem: few node that run a topology crash because Storm goes in out of memory.

    I tried to set the High water mark limit of ZMQ queue in storm.yaml (zmq.hwm) so that when the queue limit is reached the receiver or sender should block. Unfortunately this solution doesn't work. Do you know why?

    Thank you

    - Daviode
  • Michael Hopstock at Jan 11, 2013 at 11:09 pm
    I have a similar problem.
    I tried both settings but the spout is still running OOM, because the
    following bolts cannot consume the tuples due to slow I/O (in my case
    network).
    Since I'm not using anchoring, might this cause the problem? Also, my spout
    shows 0 acked tuples, is this normal using the non-anchored option?
    Btw, where is the best place to set those configs? I was setting the max
    spout pending thing both in the topology and in the config of the master,
    nodes and my local storm.yaml. Which one is used when deploying topologies?
    (I cannot use storm-deploy)

    Thanks,
    Michael

    Am Dienstag, 18. Dezember 2012 10:57:33 UTC-8 schrieb Vinh:
    Have you tried setting "topology.max.spout.pending" to limit the amount of
    pending tuples? Else, spouts will keep pushing tuples to the bolts,
    regardless of result.

    On Dec 18, 2012, at 10:50 AM, Davide Simoncelli wrote:

    Hi Nathan,


    I have a problem: few node that run a topology crash because Storm goes
    in out of memory.
    I tried to set the High water mark limit of ZMQ queue in storm.yaml
    (zmq.hwm) so that when the queue limit is reached the receiver or sender
    should block. Unfortunately this solution doesn't work. Do you know why?
    Thank you

    - Daviode

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupstorm-user @
postedDec 18, '12 at 6:50p
activeJan 11, '13 at 11:09p
posts3
users3
websitestorm-project.net
irc#storm-user

People

Translate

site design / logo © 2022 Grokbase