FAQ
Hi

I get these messages in the TT log while running a job:

2009-07-23 02:03:59,091 INFO org.apache.hadoop.mapred.TaskTracker:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200907221738_0020/attempt_200907221738_0020_r_000000_0/output/file.out
in any of the configured local directories

Whats the problem?

Amandeep



Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz

Search Discussions

  • Aaron Kimball at Jul 24, 2009 at 6:00 pm
    Amandeep,

    Does the job fail after that happens? Are there any WARN or ERROR lines in
    the log nearby, or any exceptions?

    Three possibilities I can think of:

    You may have configured Hadoop to run under /tmp, and tmpwatch or another
    cleanup utility like that decided to throw away a bunch of files in the temp
    space while your job was running. In this case, you should consider moving
    hadoop.tmp.dir and mapred.local.dir out from under the default /tmp.

    You might be out of disk space?

    mapred.local.dir or hadoop.tmp.dir might be set to paths that Hadoop doesn't
    have the privileges to write to?

    - A

    On Thu, Jul 23, 2009 at 2:06 AM, Amandeep Khurana wrote:

    Hi

    I get these messages in the TT log while running a job:

    2009-07-23 02:03:59,091 INFO org.apache.hadoop.mapred.TaskTracker:
    org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find

    taskTracker/jobcache/job_200907221738_0020/attempt_200907221738_0020_r_000000_0/output/file.out
    in any of the configured local directories

    Whats the problem?

    Amandeep



    Amandeep Khurana
    Computer Science Graduate Student
    University of California, Santa Cruz
  • Amandeep Khurana at Jul 24, 2009 at 8:04 pm

    On Fri, Jul 24, 2009 at 11:00 AM, Aaron Kimball wrote:

    Amandeep,

    Does the job fail after that happens? Are there any WARN or ERROR lines in
    the log nearby, or any exceptions?
    These occur in the JT logs:
    2009-07-24 00:19:21,598 WARN org.apache.hadoop.mapred.JobInProgress:
    Running cache for maps missing!! Job details are missing
    .
    2009-07-24 00:19:21,598 WARN org.apache.hadoop.mapred.JobInProgress:
    Non-running cache for maps missing!! Job details are mis
    sing.

    There arent any WARN on ERROR messages in the TT logs. The exact message I'm
    getting is:
    2009-07-24 10:50:32,519 INFO org.apache.hadoop.mapred.TaskTracker:
    org.apache.hadoop.util.DiskChecker$DiskErrorException: Cou
    ld not find
    taskTracker/jobcache/job_200907221738_0103/attempt_200907221738_0103_m_000015_0/output/file.out
    in any of the con
    figured local directories



    Three possibilities I can think of:

    You may have configured Hadoop to run under /tmp, and tmpwatch or another
    cleanup utility like that decided to throw away a bunch of files in the
    temp
    space while your job was running. In this case, you should consider moving
    hadoop.tmp.dir and mapred.local.dir out from under the default /tmp.
    <name>hadoop.tmp.dir</name>
    <value>/hadoop/tmp/${user.name}</value>

    and...

    <name>mapred.local.dir</name>
    <value>${hadoop.tmp.dir}/mapred/local</value>

    Its creating the jobcache directory in mapred.local.dir/local/taskTracker/
    when the job is run...

    You might be out of disk space?
    Plenty available.

    mapred.local.dir or hadoop.tmp.dir might be set to paths that Hadoop
    doesn't
    have the privileges to write to?
    Nope.. They are set to paths that the user running hadoop can write to.
    My hdfs goes to a directory: /hadoop/hdfs
    Temp goes to: /hadoop/tmp

    I deleted the /hadoop/tmp directory and ran the jobs again. It created the
    directory again on its own.

    Cant figure out whats wrong. Any other pointers?

    - A

    On Thu, Jul 23, 2009 at 2:06 AM, Amandeep Khurana wrote:

    Hi

    I get these messages in the TT log while running a job:

    2009-07-23 02:03:59,091 INFO org.apache.hadoop.mapred.TaskTracker:
    org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find

    taskTracker/jobcache/job_200907221738_0020/attempt_200907221738_0020_r_000000_0/output/file.out
    in any of the configured local directories

    Whats the problem?

    Amandeep



    Amandeep Khurana
    Computer Science Graduate Student
    University of California, Santa Cruz

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 23, '09 at 9:05a
activeJul 24, '09 at 8:04p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Amandeep Khurana: 2 posts Aaron Kimball: 1 post

People

Translate

site design / logo © 2022 Grokbase