FAQ
In TaskTracker, the notification for waking up the completion-events fetcher thread may be lost
-----------------------------------------------------------------------------------------------

Key: HADOOP-5245
URL: https://issues.apache.org/jira/browse/HADOOP-5245
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.21.0


In the TaskTracker, there is a completion events fetcher thread that fetches new events from the JobTracker. Normally, the fetcher thread would sleep for heartbeatInterval amount of time per cycle. When a reduce task asks for completion events from the corresponding TaskTracker and the TaskTracker currently doesn't have anything to hand out, a notification is sent to the completion events fetcher thread to wake up and fetch new events if any. Sometimes this notification could be lost and the reduce task would be idle for a few seconds. This hurts the performance of jobs like terasort.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Devaraj Das (JIRA) at Feb 13, 2009 at 5:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Devaraj Das updated HADOOP-5245:
    --------------------------------

    Attachment: 5245.patch

    Straightforward patch.
    In TaskTracker, the notification for waking up the completion-events fetcher thread may be lost
    -----------------------------------------------------------------------------------------------

    Key: HADOOP-5245
    URL: https://issues.apache.org/jira/browse/HADOOP-5245
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Fix For: 0.21.0

    Attachments: 5245.patch


    In the TaskTracker, there is a completion events fetcher thread that fetches new events from the JobTracker. Normally, the fetcher thread would sleep for heartbeatInterval amount of time per cycle. When a reduce task asks for completion events from the corresponding TaskTracker and the TaskTracker currently doesn't have anything to hand out, a notification is sent to the completion events fetcher thread to wake up and fetch new events if any. Sometimes this notification could be lost and the reduce task would be idle for a few seconds. This hurts the performance of jobs like terasort.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedFeb 13, '09 at 4:32a
activeFeb 13, '09 at 5:26a
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Devaraj Das (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase