FAQ
Add more statistics in the web-ui to do with tasks
--------------------------------------------------

Key: HADOOP-1101
URL: https://issues.apache.org/jira/browse/HADOOP-1101
Project: Hadoop
Issue Type: Improvement
Reporter: Devaraj Das
Assigned To: Devaraj Das


Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Devaraj Das (JIRA) at Mar 15, 2007 at 5:23 am
    [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Devaraj Das updated HADOOP-1101:
    --------------------------------

    Attachment: 1101.patch

    This patch does the following (in analysejobhistory.jsp):
    1) Splits the Reduce phase into two - shuffle and the actual "reduce". It displays the data for each seperately.
    2) Displays the average time, and the transition time for the last task for each of the phases (map, shuffle, reduce).
    Add more statistics in the web-ui to do with tasks
    --------------------------------------------------

    Key: HADOOP-1101
    URL: https://issues.apache.org/jira/browse/HADOOP-1101
    Project: Hadoop
    Issue Type: Improvement
    Reporter: Devaraj Das
    Assigned To: Devaraj Das
    Attachments: 1101.patch


    Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Devaraj Das (JIRA) at Mar 15, 2007 at 5:25 am
    [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Devaraj Das updated HADOOP-1101:
    --------------------------------

    Status: Patch Available (was: Open)
    Add more statistics in the web-ui to do with tasks
    --------------------------------------------------

    Key: HADOOP-1101
    URL: https://issues.apache.org/jira/browse/HADOOP-1101
    Project: Hadoop
    Issue Type: Improvement
    Reporter: Devaraj Das
    Assigned To: Devaraj Das
    Attachments: 1101.patch


    Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Devaraj Das (JIRA) at Mar 15, 2007 at 5:27 am
    [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Devaraj Das updated HADOOP-1101:
    --------------------------------

    Fix Version/s: 0.12.1
    Description: Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. (was: Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.)
    Affects Version/s: 0.12.0
    Add more statistics in the web-ui to do with tasks
    --------------------------------------------------

    Key: HADOOP-1101
    URL: https://issues.apache.org/jira/browse/HADOOP-1101
    Project: Hadoop
    Issue Type: Improvement
    Affects Versions: 0.12.0
    Reporter: Devaraj Das
    Assigned To: Devaraj Das
    Fix For: 0.12.1

    Attachments: 1101.patch


    Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nigel Daley (JIRA) at Mar 16, 2007 at 6:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Nigel Daley updated HADOOP-1101:
    --------------------------------

    Fix Version/s: (was: 0.12.1)
    0.13.0

    deferring to 0.13.0
    Add more statistics in the web-ui to do with tasks
    --------------------------------------------------

    Key: HADOOP-1101
    URL: https://issues.apache.org/jira/browse/HADOOP-1101
    Project: Hadoop
    Issue Type: Improvement
    Affects Versions: 0.12.0
    Reporter: Devaraj Das
    Assigned To: Devaraj Das
    Fix For: 0.13.0

    Attachments: 1101.patch


    Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Doug Cutting (JIRA) at Mar 23, 2007 at 9:02 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Doug Cutting updated HADOOP-1101:
    ---------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Devaraj.
    Add more statistics in the web-ui to do with tasks
    --------------------------------------------------

    Key: HADOOP-1101
    URL: https://issues.apache.org/jira/browse/HADOOP-1101
    Project: Hadoop
    Issue Type: Improvement
    Affects Versions: 0.12.0
    Reporter: Devaraj Das
    Assigned To: Devaraj Das
    Fix For: 0.13.0

    Attachments: 1101.patch


    Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 9, '07 at 6:59a
activeMar 23, '07 at 9:02p
posts6
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Doug Cutting (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase