Hai guys !

Can i access the Job counters displayed in WEB GUI in Hadoop code when the
job finished their execution ?
If so, how can i access the values like "average task run time" and
counters "FILE/HDFS BYTES READ/WRITTEN" immediately after the job has
completed in JobQueueTaskScheduler Code or in some other code file ?


Thanks,
Arun

Search Discussions

  • Devaraj K at Dec 12, 2011 at 3:31 am
    Hi Arun,

    You can get the Counters object from the job after completion and
    then find the counter whichever you want in Counters object using
    findCounter api. Please find the sample snippet for accessing the counter
    after Job completion


    Configuration conf = new Configuration();

    Cluster cluster = new Cluster(conf);

    Job job = Job.getInstance(cluster,conf);

    result = job.waitForCompletion(true);
    .....
    .....
    Counters counters = job.getCounters();

    Counter c1 = counters.findCounter(CustomCOUNTER.NOOFBADRECORDS);

    System.out.println(c1.getDisplayName()+":"+c1.getValue());





    Devaraj K

    -----Original Message-----
    From: ArunKumar
    Sent: Sunday, December 11, 2011 12:15 PM
    To: hadoop-user@lucene.apache.org
    Subject: Accessing Job counters displayed in WEB GUI in Hadoop Code

    Hai guys !

    Can i access the Job counters displayed in WEB GUI in Hadoop code when the
    job finished their execution ?
    If so, how can i access the values like "average task run time" and counters
    "FILE/HDFS BYTES READ/WRITTEN" immediately after the job has completed in
    JobQueueTaskScheduler Code or in some other code file ?


    Thanks,
    Arun

    --
    View this message in context:
    http://lucene.472066.n3.nabble.com/Accessing-Job-counters-displayed-in-WEB-G
    UI-in-Hadoop-Code-tp3576925p3576925.html
    Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedDec 11, '11 at 6:42a
activeDec 12, '11 at 3:31a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Arun k: 1 post Devaraj K: 1 post

People

Translate

site design / logo © 2022 Grokbase