FAQ
Job history log does not output final counters if job is failed or killed
-------------------------------------------------------------------------

Key: HADOOP-5706
URL: https://issues.apache.org/jira/browse/HADOOP-5706
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Mac Yang


The last log entry in job history contains important counters. This information is missing if the job is killed or failed.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Mac Yang (JIRA) at Apr 20, 2009 at 5:37 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700878#action_12700878 ]

    Mac Yang commented on HADOOP-5706:
    ----------------------------------


    Sample output from a normal job,

    Job JOBID="job_200904200538_0393" FINISH_TIME="1240248187540" JOB_STATUS="SUCCESS" FINISHED_MAPS="20" FINISHED_REDUCES="1" FAILED_MAPS="0" FAILED_REDUCES="0" COUNTERS="{(org\.apache\.hadoop\.mapred\.JobInProgress$Counter)(Job Counters )[(TOTAL_LAUNCHED_REDUCES)(Launched reduce tasks)(1)][(RACK_LOCAL_MAPS)(Rack-local map tasks)(2)][(TOTAL_LAUNCHED_MAPS)(Launched map tasks)(20)][(DATA_LOCAL_MAPS)(Data-local map tasks)(1)]}{(FileSystemCounters)(FileSystemCounters)[(FILE_BYTES_READ)(FILE_BYTES_READ)(83248)][(HDFS_BYTES_READ)(HDFS_BYTES_READ)(2019250)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(948688)][(HDFS_BYTES_WRITTEN)(HDFS_BYTES_WRITTEN)(56630)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(5315)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(72020)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(821560)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(5315)][(SPILLED_RECORDS)(Spilled Records)(1057000)][(MAP_OUTPUT_BYTES)(Map output bytes)(3373000)][(MAP_INPUT_BYTES)(Map input bytes)(2965720)][(MAP_OUTPUT_RECORDS)(Map output records)(528500)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(528500)]}" .

    Sample output from a killed job,

    Job JOBID="job_200904200538_0015" FINISH_TIME="1240209608007" JOB_STATUS="KILLED" FINISHED_MAPS="1" FINISHED_REDUCES="0" .

    Job history log does not output final counters if job is failed or killed
    -------------------------------------------------------------------------

    Key: HADOOP-5706
    URL: https://issues.apache.org/jira/browse/HADOOP-5706
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Mac Yang

    The last log entry in job history contains important counters. This information is missing if the job is killed or failed.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 20, '09 at 5:35p
activeApr 20, '09 at 5:37p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Mac Yang (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase