FAQ
Hi,
For my research project, I need to modify Hadoop JobTracker to collect some
statistics of TaskTracker nodes.
For example, I would like to piggy-back heartbeat messages sent from
TaskTrackers to JobTracker with some extra information related to the
resource utilization and other statistics.
I am able to checkout Hadoop common code base and build in Eclipse.
Specifically, which files should I look at, and after the above mentioned
modification, how to collect the statistics?

Thanks,
Bikash

Search Discussions

  • Nan Zhu at Mar 18, 2011 at 6:12 pm
    Hi, bikash

    I also worked on this several months ago,

    If I don't have a wrong memory, the involved files should be:
    JobTracker.java, TaskTracker.java,

    the most interesting functions should be JobTracker.heartbeat(), and
    TaskTracker.transmitHeartbeat(), I think you will also need to add more
    member variables in TaskTrackerStatus.java

    take care about the write() and read() in TaskTrackerStatus.java,

    Best,

    Nan
    On Sat, Mar 19, 2011 at 1:59 AM, bikash sharma wrote:

    Hi,
    For my research project, I need to modify Hadoop JobTracker to collect some
    statistics of TaskTracker nodes.
    For example, I would like to piggy-back heartbeat messages sent from
    TaskTrackers to JobTracker with some extra information related to the
    resource utilization and other statistics.
    I am able to checkout Hadoop common code base and build in Eclipse.
    Specifically, which files should I look at, and after the above mentioned
    modification, how to collect the statistics?

    Thanks,
    Bikash


    --
    Nan Zhu
    School of Software,5501
    Shanghai Jiao Tong University
    800,Dongchuan Road,Shanghai,China
    E-Mail: [email protected]

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMar 18, '11 at 5:59p
activeMar 18, '11 at 6:12p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Nan Zhu: 1 post Bikash sharma: 1 post

People

Translate

site design / logo © 2023 Grokbase