FAQ
I want to get metrics information (e.g. number of Maps, number of
Reduces, memory use, load) by APIs. I found two useful classes -
ClusterStatus and ClusterMetrics. My question is how I can get
instances of that two classes? From JobClient or JobTracker? Any
suggested alternative way to get the information?

Thanks

Gerald

Search Discussions

  • Ted Yu at Jan 22, 2011 at 9:47 pm
    You can use the following code:
    JobClient jc = new JobClient(jobConf);
    int numReduces = jc.getClusterStatus().getMaxReduceTasks();

    For 0.20.3, you can use:
    ClusterMetrics metrics = jobTracker.getClusterMetrics();
    On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo wrote:

    I want to get metrics information (e.g. number of Maps, number of
    Reduces, memory use, load) by APIs. I found two useful classes -
    ClusterStatus and ClusterMetrics. My question is how I can get
    instances of that two classes? From JobClient or JobTracker? Any
    suggested alternative way to get the information?

    Thanks

    Gerald
  • Zhenhua Guo at Jan 23, 2011 at 2:38 am
    Thanks!
    How to get JobTracker object?

    Gerald
    On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu wrote:
    You can use the following code:
    JobClient jc = new JobClient(jobConf);
    int numReduces = jc.getClusterStatus().getMaxReduceTasks();

    For 0.20.3, you can use:
    ClusterMetrics metrics = jobTracker.getClusterMetrics();
    On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo wrote:

    I want to get metrics information (e.g. number of Maps, number of
    Reduces, memory use, load) by APIs. I found two useful classes -
    ClusterStatus and ClusterMetrics. My question is how I can get
    instances of that two classes? From JobClient or JobTracker? Any
    suggested alternative way to get the information?

    Thanks

    Gerald
  • Ted Yu at Jan 23, 2011 at 3:00 am
    In the test code, JobTracker is returned from:

    mr = new MiniMRCluster(0, 0, 0, "file:///", 1, null, null, null,
    conf);
    jobTracker = mr.getJobTrackerRunner().getJobTracker();

    I guess it is not exposed in non-test code.
    On Sat, Jan 22, 2011 at 6:38 PM, Zhenhua Guo wrote:

    Thanks!
    How to get JobTracker object?

    Gerald
    On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu wrote:
    You can use the following code:
    JobClient jc = new JobClient(jobConf);
    int numReduces = jc.getClusterStatus().getMaxReduceTasks();

    For 0.20.3, you can use:
    ClusterMetrics metrics = jobTracker.getClusterMetrics();
    On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo wrote:

    I want to get metrics information (e.g. number of Maps, number of
    Reduces, memory use, load) by APIs. I found two useful classes -
    ClusterStatus and ClusterMetrics. My question is how I can get
    instances of that two classes? From JobClient or JobTracker? Any
    suggested alternative way to get the information?

    Thanks

    Gerald
  • Edward Capriolo at Jan 23, 2011 at 2:21 pm

    On Sat, Jan 22, 2011 at 9:59 PM, Ted Yu wrote:
    In the test code, JobTracker is returned from:

    mr = new MiniMRCluster(0, 0, 0, "file:///", 1, null, null, null,
    conf);
    jobTracker = mr.getJobTrackerRunner().getJobTracker();

    I guess it is not exposed in non-test code.
    On Sat, Jan 22, 2011 at 6:38 PM, Zhenhua Guo wrote:

    Thanks!
    How to get JobTracker object?

    Gerald
    On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu wrote:
    You can use the following code:
    JobClient jc = new JobClient(jobConf);
    int numReduces = jc.getClusterStatus().getMaxReduceTasks();

    For 0.20.3, you can use:
    ClusterMetrics metrics = jobTracker.getClusterMetrics();
    On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo wrote:

    I want to get metrics information (e.g. number of Maps, number of
    Reduces, memory use, load) by APIs. I found two useful classes -
    ClusterStatus and ClusterMetrics. My question is how I can get
    instances of that two classes? From JobClient or JobTracker? Any
    suggested alternative way to get the information?

    Thanks

    Gerald
    Correct JobTracker is the JobTracker itself that does not have any interface.
    However using JobClient you can mine most of the information out from
    the Job Tracker.

    My cacti graphing package takes that exact approach to pull
    information that is not a JMXCounter such as.
    http://www.jointhegrid.com/hadoop-cacti-jtg-walk/maps_v_reduces.jsp

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJan 22, '11 at 5:58p
activeJan 23, '11 at 2:21p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase