FAQ
Hi All,

Does anyone know how many bytes does one heart beat message contain? And
what about the return value of heart beat message?

Thanks,

Shen

Search Discussions

  • Harsh J at Jun 10, 2011 at 8:07 pm
    Shen,

    It would really vary depending on how many TaskStatus objects the
    TaskTracker has to report in the heartbeat. If there are none (zero
    activity), I think the byte value (per Visual VM here) comes to about
    60B which you can perhaps add upon.

    Note that I'm using OSX so perhaps the ResourceStats carried within
    aren't properly for me (some of these are Linux specific). It
    shouldn't exceed 100B in any case though, I think.

    Similarly, the response carries variable numbers of TaskTrackerAction
    objects, and other job related information. At zero activity I see it
    reported as 32B.

    HTH. These are in memory values -- network tx. is made with some form
    of serialization applied, and I'm unsure as to how much the variance
    would be.
    On Fri, Jun 10, 2011 at 9:49 PM, Shen LI wrote:
    Hi All,
    Does anyone know how many bytes does one heart beat message contain? And
    what about the return value of heart beat message?
    Thanks,
    Shen


    --
    Harsh J
  • Shen LI at Jun 12, 2011 at 12:48 am
    Hi Harsh,

    Thank you very much for your reply. Do you know if there is any reference
    about calculating heart beat message size that I can turn to? If I have 4
    living task under each TaskTracker, how many bytes will one heart beat
    message contain?

    Thanks,

    Shen
    On Fri, Jun 10, 2011 at 5:06 PM, Harsh J wrote:

    Shen,

    It would really vary depending on how many TaskStatus objects the
    TaskTracker has to report in the heartbeat. If there are none (zero
    activity), I think the byte value (per Visual VM here) comes to about
    60B which you can perhaps add upon.

    Note that I'm using OSX so perhaps the ResourceStats carried within
    aren't properly for me (some of these are Linux specific). It
    shouldn't exceed 100B in any case though, I think.

    Similarly, the response carries variable numbers of TaskTrackerAction
    objects, and other job related information. At zero activity I see it
    reported as 32B.

    HTH. These are in memory values -- network tx. is made with some form
    of serialization applied, and I'm unsure as to how much the variance
    would be.
    On Fri, Jun 10, 2011 at 9:49 PM, Shen LI wrote:
    Hi All,
    Does anyone know how many bytes does one heart beat message contain? And
    what about the return value of heart beat message?
    Thanks,
    Shen


    --
    Harsh J
  • Harsh J at Jun 12, 2011 at 4:53 am
    Shen,

    Perhaps you could write some tests in Java to measure object sizes; or
    you can run a profiler and measure it live (VisualVM lets you profile
    with visuals)?

    As Arun mentioned, it is really tough to say since every heartbeat
    would carry variable sizes (I'd just provided empty-ish values)
    depending on each of the Task's current, reported status and the
    commands being issued back. But I believe you could form a formula of
    sorts with careful-enough study.
    On Sun, Jun 12, 2011 at 6:17 AM, Shen LI wrote:
    Hi Harsh,
    Thank you very much for your reply. Do you know if there is any reference
    about calculating heart beat message size  that I can turn to? If I have 4
    living task under each TaskTracker,  how many bytes will one heart beat
    message contain?
    Thanks,
    Shen
    On Fri, Jun 10, 2011 at 5:06 PM, Harsh J wrote:

    Shen,

    It would really vary depending on how many TaskStatus objects the
    TaskTracker has to report in the heartbeat. If there are none (zero
    activity), I think the byte value (per Visual VM here) comes to about
    60B which you can perhaps add upon.

    Note that I'm using OSX so perhaps the ResourceStats carried within
    aren't properly for me (some of these are Linux specific). It
    shouldn't exceed 100B in any case though, I think.

    Similarly, the response carries variable numbers of TaskTrackerAction
    objects, and other job related information. At zero activity I see it
    reported as 32B.

    HTH. These are in memory values -- network tx. is made with some form
    of serialization applied, and I'm unsure as to how much the variance
    would be.
    On Fri, Jun 10, 2011 at 9:49 PM, Shen LI wrote:
    Hi All,
    Does anyone know how many bytes does one heart beat message contain? And
    what about the return value of heart beat message?
    Thanks,
    Shen


    --
    Harsh J


    --
    Harsh J
  • Arun C Murthy at Jun 10, 2011 at 8:17 pm
    It really depends... the problem is that the TT's msg includes user-
    counters.

    So, in bad cases it could be infinite. hadoop-0.20.203 limits
    #counters for precisely this reason.

    In hadoop-0.20.203 it could be as big as multiple KB, but no more.
    On Jun 10, 2011, at 9:49 PM, Shen LI wrote:

    Hi All,

    Does anyone know how many bytes does one heart beat message contain?
    And what about the return value of heart beat message?

    Thanks,

    Shen

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJun 10, '11 at 4:19p
activeJun 12, '11 at 4:53a
posts5
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Harsh J: 2 posts Shen LI: 2 posts Arun C Murthy: 1 post

People

Translate

site design / logo © 2022 Grokbase