FAQ
Few tasks failed while creating the work directory for a job, when job tracker was restarted
--------------------------------------------------------------------------------------------

Key: HADOOP-5487
URL: https://issues.apache.org/jira/browse/HADOOP-5487
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Hemanth Yamijala


A randomwriter job was running when the job tracker restarted. After the jobtracker restarted, some tasktrackers were sent a reinit action. After this, some new tasks of the random writer were scheduled to be run on the same task trackers. These failed in the job localization while creating the work directory. However, the next attempts of the same job ran successfully and the job succeeded. This happened in about 1% of the total number of tasks.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Hemanth Yamijala (JIRA) at Mar 13, 2009 at 1:08 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681696#action_12681696 ]

    Hemanth Yamijala commented on HADOOP-5487:
    ------------------------------------------

    Following was the exception trace on such a task:

    java.io.IOException: Mkdirs failed to create /path/to/mapred-local/taskTracker/jobcache/job_200903130908_0051/work
    at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:829)
    at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1743)
    at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:97)
    at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1708)
    Few tasks failed while creating the work directory for a job, when job tracker was restarted
    --------------------------------------------------------------------------------------------

    Key: HADOOP-5487
    URL: https://issues.apache.org/jira/browse/HADOOP-5487
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Hemanth Yamijala

    A randomwriter job was running when the job tracker restarted. After the jobtracker restarted, some tasktrackers were sent a reinit action. After this, some new tasks of the random writer were scheduled to be run on the same task trackers. These failed in the job localization while creating the work directory. However, the next attempts of the same job ran successfully and the job succeeded. This happened in about 1% of the total number of tasks.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Amareshwari Sriramadasu (JIRA) at Apr 2, 2009 at 11:32 am
    [ https://issues.apache.org/jira/browse/HADOOP-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amareshwari Sriramadasu updated HADOOP-5487:
    --------------------------------------------

    Fix Version/s: 0.21.0
    Assignee: Amareshwari Sriramadasu
    Few tasks failed while creating the work directory for a job, when job tracker was restarted
    --------------------------------------------------------------------------------------------

    Key: HADOOP-5487
    URL: https://issues.apache.org/jira/browse/HADOOP-5487
    Project: Hadoop Core
    Issue Type: Bug
    Components: mapred
    Reporter: Hemanth Yamijala
    Assignee: Amareshwari Sriramadasu
    Fix For: 0.21.0


    A randomwriter job was running when the job tracker restarted. After the jobtracker restarted, some tasktrackers were sent a reinit action. After this, some new tasks of the random writer were scheduled to be run on the same task trackers. These failed in the job localization while creating the work directory. However, the next attempts of the same job ran successfully and the job succeeded. This happened in about 1% of the total number of tasks.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 13, '09 at 1:06p
activeApr 2, '09 at 11:32a
posts3
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Amareshwari Sriramadasu (JIRA): 3 posts

People

Translate

site design / logo © 2022 Grokbase