[MapReduce-user] How many bytes does heart beat message and return value contain?

Shen,

It would really vary depending on how many TaskStatus objects the
TaskTracker has to report in the heartbeat. If there are none (zero
activity), I think the byte value (per Visual VM here) comes to about
60B which you can perhaps add upon.

Note that I'm using OSX so perhaps the ResourceStats carried within
aren't properly for me (some of these are Linux specific). It
shouldn't exceed 100B in any case though, I think.

Similarly, the response carries variable numbers of TaskTrackerAction
objects, and other job related information. At zero activity I see it
reported as 32B.

HTH. These are in memory values -- network tx. is made with some form
of serialization applied, and I'm unsure as to how much the variance
would be.

Hi Harsh,

Thank you very much for your reply. Do you know if there is any reference
about calculating heart beat message size that I can turn to? If I have 4
living task under each TaskTracker, how many bytes will one heart beat
message contain?

Thanks,

Shen

Shen,

Perhaps you could write some tests in Java to measure object sizes; or
you can run a profiler and measure it live (VisualVM lets you profile
with visuals)?

As Arun mentioned, it is really tough to say since every heartbeat
would carry variable sizes (I'd just provided empty-ish values)
depending on each of the Task's current, reported status and the
commands being issued back. But I believe you could form a formula of
sorts with careful-enough study.

It really depends... the problem is that the TT's msg includes user-
counters.

So, in bad cases it could be infinite. hadoop-0.20.203 limits
#counters for precisely this reason.

In hadoop-0.20.203 it could be as big as multiple KB, but no more.