FAQ
Just wondering, does storm use direct memory (memory outside of heap).
Memory stats on worker process show high RES values, while at the same time
heap values remain pretty low when queried from inside workers.
zMQ queues are stored in direct memory?

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Nathan Marz at Jan 30, 2013 at 7:52 am
    ZeroMQ buffers in native memory. This is highly undesirable and is one of
    the reasons why we will be replacing zeromq with a pure-Java replacement.
    On Tue, Jan 29, 2013 at 8:30 AM, Ivan Balashov wrote:

    Just wondering, does storm use direct memory (memory outside of heap).
    Memory stats on worker process show high RES values, while at the same time
    heap values remain pretty low when queried from inside workers.
    zMQ queues are stored in direct memory?

    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Abee at Jan 30, 2013 at 12:27 pm
    Hi Nathan,
    message buffering and replaying them back to the desired nodes,
    when there are failure is one of the important feature of zeromq.
    what is the overhead of batching in zeromq?, can the batch size and
    time-interval for the batching be controlled in storm?
    so are you planing to to use netty alone with bookeeper (zookeeper
    subproject) which can maintain the transaction logs,
    or maintaining a map manged by NIO (DirectBuffer) overflowing it
    disk.
    Thanks
    abee
    On Jan 30, 12:52 pm, Nathan Marz wrote:
    ZeroMQ buffers in native memory. This is highly undesirable and is one of
    the reasons why we will be replacing zeromq with a pure-Java replacement.






    On Tue, Jan 29, 2013 at 8:30 AM, Ivan Balashov wrote:
    Just wondering, does storm use direct memory (memory outside of heap).
    Memory stats on worker process show high RES values, while at the same time
    heap values remain pretty low when queried from inside workers.
    zMQ queues are stored in direct memory?
    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visithttps://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ryan LeCompte at Jan 31, 2013 at 9:05 am
    Do you have an idea what the pure-Java replacement will be?

    On Tue, Jan 29, 2013 at 11:52 PM, Nathan Marz wrote:

    ZeroMQ buffers in native memory. This is highly undesirable and is one of
    the reasons why we will be replacing zeromq with a pure-Java replacement.

    On Tue, Jan 29, 2013 at 8:30 AM, Ivan Balashov wrote:

    Just wondering, does storm use direct memory (memory outside of heap).
    Memory stats on worker process show high RES values, while at the same time
    heap values remain pretty low when queried from inside workers.
    zMQ queues are stored in direct memory?

    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nathan Marz at Jan 31, 2013 at 9:08 am
    It will be specific to Storm's networking needs, not a full 0mq replacement
    On Tue, Jan 29, 2013 at 11:56 PM, Ryan LeCompte wrote:

    Do you have an idea what the pure-Java replacement will be?

    On Tue, Jan 29, 2013 at 11:52 PM, Nathan Marz wrote:

    ZeroMQ buffers in native memory. This is highly undesirable and is one of
    the reasons why we will be replacing zeromq with a pure-Java replacement.

    On Tue, Jan 29, 2013 at 8:30 AM, Ivan Balashov wrote:

    Just wondering, does storm use direct memory (memory outside of heap).
    Memory stats on worker process show high RES values, while at the same time
    heap values remain pretty low when queried from inside workers.
    zMQ queues are stored in direct memory?

    --
    You received this message because you are subscribed to the Google
    Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups
    "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupstorm-user @
postedJan 29, '13 at 4:30p
activeJan 31, '13 at 9:08a
posts5
users4
websitestorm-project.net
irc#storm-user

People

Translate

site design / logo © 2022 Grokbase