FAQ
[ https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717939#action_12717939 ]

Hadoop QA commented on HADOOP-2141:
-----------------------------------

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12410208/2141.5.patch
against trunk revision 783059.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 22 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

-1 contrib tests. The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/488/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/488/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/488/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/488/console

This message is automatically generated.
speculative execution start up condition based on completion time
-----------------------------------------------------------------

Key: HADOOP-2141
URL: https://issues.apache.org/jira/browse/HADOOP-2141
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Affects Versions: 0.21.0
Reporter: Koji Noguchi
Assignee: Andy Konwinski
Fix For: 0.21.0

Attachments: 2141.4.patch, 2141.5.patch, 2141.patch, HADOOP-2141-v2.patch, HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch, HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch, HADOOP-2141.v8.patch


We had one job with speculative execution hang.
4 reduce tasks were stuck with 95% completion because of a bad disk.
Devaraj pointed out
bq . One of the conditions that must be met for launching a speculative instance of a task is that it must be at least 20% behind the average progress, and this is not true here.
It would be nice if speculative execution also starts up when tasks stop making progress.
Devaraj suggested
bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative execution check.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Amareshwari Sriramadasu (JIRA) at Jun 12, 2009 at 10:41 am
    [ https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718790#action_12718790 ]

    Amareshwari Sriramadasu commented on HADOOP-2141:
    -------------------------------------------------

    one minor comment:
    Initialization for stats can be done inline. And comments to the stats, describing what the stats are for, can be added.
    speculative execution start up condition based on completion time
    -----------------------------------------------------------------

    Key: HADOOP-2141
    URL: https://issues.apache.org/jira/browse/HADOOP-2141
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.21.0
    Reporter: Koji Noguchi
    Assignee: Andy Konwinski
    Fix For: 0.21.0

    Attachments: 2141.4.patch, 2141.5.patch, 2141.6.patch, 2141.patch, HADOOP-2141-v2.patch, HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch, HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch, HADOOP-2141.v8.patch


    We had one job with speculative execution hang.
    4 reduce tasks were stuck with 95% completion because of a bad disk.
    Devaraj pointed out
    bq . One of the conditions that must be met for launching a speculative instance of a task is that it must be at least 20% behind the average progress, and this is not true here.
    It would be nice if speculative execution also starts up when tasks stop making progress.
    Devaraj suggested
    bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative execution check.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amareshwari Sriramadasu (JIRA) at Jun 15, 2009 at 9:41 am
    [ https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719488#action_12719488 ]

    Amareshwari Sriramadasu commented on HADOOP-2141:
    -------------------------------------------------

    Some minor comments :
    {code}
    + //runningMapStats used to maintain the RUNNING reduce tasks' statistics
    + private DataStatistics runningReduceTaskStats = new DataStatistics();
    {code}
    The comment should say runningReduceStats

    In TaskInProgress.updateStatus() method, taskStatuses.put(taskid, status) can be moved up in the if condition and else can be removed.
    speculative execution start up condition based on completion time
    -----------------------------------------------------------------

    Key: HADOOP-2141
    URL: https://issues.apache.org/jira/browse/HADOOP-2141
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.21.0
    Reporter: Koji Noguchi
    Assignee: Andy Konwinski
    Fix For: 0.21.0

    Attachments: 2141.4.patch, 2141.5.patch, 2141.6.patch, 2141.7.patch, 2141.8.2.patch, 2141.patch, HADOOP-2141-v2.patch, HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch, HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch, HADOOP-2141.v8.patch


    We had one job with speculative execution hang.
    4 reduce tasks were stuck with 95% completion because of a bad disk.
    Devaraj pointed out
    bq . One of the conditions that must be met for launching a speculative instance of a task is that it must be at least 20% behind the average progress, and this is not true here.
    It would be nice if speculative execution also starts up when tasks stop making progress.
    Devaraj suggested
    bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative execution check.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amareshwari Sriramadasu (JIRA) at Jun 15, 2009 at 11:21 am
    [ https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719521#action_12719521 ]

    Amareshwari Sriramadasu commented on HADOOP-2141:
    -------------------------------------------------

    +1 patch looks good to me.
    speculative execution start up condition based on completion time
    -----------------------------------------------------------------

    Key: HADOOP-2141
    URL: https://issues.apache.org/jira/browse/HADOOP-2141
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.21.0
    Reporter: Koji Noguchi
    Assignee: Andy Konwinski
    Fix For: 0.21.0

    Attachments: 2141.4.patch, 2141.5.patch, 2141.6.patch, 2141.7.patch, 2141.8.2.patch, 2141.8.3.patch, 2141.patch, HADOOP-2141-v2.patch, HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch, HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch, HADOOP-2141.v8.patch


    We had one job with speculative execution hang.
    4 reduce tasks were stuck with 95% completion because of a bad disk.
    Devaraj pointed out
    bq . One of the conditions that must be met for launching a speculative instance of a task is that it must be at least 20% behind the average progress, and this is not true here.
    It would be nice if speculative execution also starts up when tasks stop making progress.
    Devaraj suggested
    bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative execution check.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Jun 17, 2009 at 7:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720841#action_12720841 ]

    Hudson commented on HADOOP-2141:
    --------------------------------

    Integrated in Hadoop-trunk #869 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/869/])

    speculative execution start up condition based on completion time
    -----------------------------------------------------------------

    Key: HADOOP-2141
    URL: https://issues.apache.org/jira/browse/HADOOP-2141
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.21.0
    Reporter: Koji Noguchi
    Assignee: Andy Konwinski
    Fix For: 0.21.0

    Attachments: 2141.4.patch, 2141.5.patch, 2141.6.patch, 2141.7.patch, 2141.8.2.patch, 2141.8.3.patch, 2141.patch, HADOOP-2141-v2.patch, HADOOP-2141-v3.patch, HADOOP-2141-v4.patch, HADOOP-2141-v5.patch, HADOOP-2141-v6.patch, HADOOP-2141.patch, HADOOP-2141.v7.patch, HADOOP-2141.v8.patch


    We had one job with speculative execution hang.
    4 reduce tasks were stuck with 95% completion because of a bad disk.
    Devaraj pointed out
    bq . One of the conditions that must be met for launching a speculative instance of a task is that it must be at least 20% behind the average progress, and this is not true here.
    It would be nice if speculative execution also starts up when tasks stop making progress.
    Devaraj suggested
    bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative execution check.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 10, '09 at 6:35a
activeJun 17, '09 at 7:20p
posts5
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase