FAQ
Hi,

I have installed CDH through cloudera manager free edition. The setup halts
at starting Task trackers on the nodes. I have manually created
/data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
instructed in CDH documentation. Yet while i try to start mapreduce service

TaskTracker local dir /data/1/mapred/local error File
/data/1/mapred/local does not exist, removing from local dirs
TaskTracker local dir /data/2/mapred/local error File
/data/2/mapred/local does not exist, removing from local dirs

TaskTracker local dir /data/3/mapred/local error File
/data/3/mapred/local does not exist, removing from local dirs


Can not start task tracker because
org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local
directories are writable
  at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
  at org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:4040)


The directory permission structure is as follows,

drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data

and within data,
drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4

Kindly help resolving this.

--
Regards,
Praveen Bysani
http://www.praveenbysani.com

Search Discussions

  • Adam Smieszny at Mar 10, 2013 at 7:47 pm
    Can you do an ls -alh /data/*/mapred and share that output please?

    Thanks,
    Adam

    On Sun, Mar 10, 2013 at 9:47 AM, Praveen Bysani wrote:

    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs


    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data


    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4


    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156
  • Praveen Bysani at Mar 11, 2013 at 2:32 am
    Hi,

    The ouput for the command is,

    root@server:/# ls -alh /data/*/mapred
    /data/1/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/2/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/3/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/4/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local


    However i was able to start the TaskTracker successfully by creating a
    similar directory structure outside /data but instead a /mapred directory
    with identical file permissions. But it only lead me to a new problem when
    i tried to run a pig job as user 'hadoopuser' with the following errors in
    the grunt shell even after i configured the mapred.local.dir to valid
    fields in the cofiguration file and restarted the service,

    2013-03-11 10:19:40,645 [JobControl] INFO
    org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://
    server.epicoders.com:8020/user/hadoopuser/.staging/job_201303110217_0001
    2013-03-11 10:19:40,646 [JobControl] ERROR
    org.apache.hadoop.security.UserGroupInformation -
    PriviledgedActionException as:hadoopuser (auth:SIMPLE)
    cause:org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
             at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:396)
             at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
             at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
             at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
             at
    org.apache.hadoop.mapred.JobInProgress.(JobTracker.java:3507)
             ... 13 more

    2013-03-11 10:19:40,647 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2013-03-11 10:19:40,655 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
    recreate exception from backend error:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
             at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:396)
             at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
             at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
             at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
             at
    org.apache.hadoop.mapred.JobInProgress.(JobTracker.java:3507)
             ... 13 more

             at org.apache.hadoop.ipc.Client.call(Client.java:1225)
             at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
             at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
             at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
             at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:396)
             at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
             at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
             at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
             at java.lang.Thread.run(Thread.java:662)
             at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)

    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
    2013-03-11 10:19:40,667 [main] INFO
    org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

    HadoopVersion PigVersion UserId StartedAt FinishedAt
    Features
    2.0.0-cdh4.2.0 0.10.0-cdh4.2.0 hadoopuser 2013-03-11 10:19:34
    2013-03-11 10:19:40 LIMIT

    Failed!

    Failed Jobs:
    JobId Alias Feature Message Outputs
    N/A clean_log,log Message:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
             at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
             at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
             at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:396)
             at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
             at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
             at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
             at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
             at
    org.apache.hadoop.mapred.JobInProgress.(JobTracker.java:3507)
             ... 13 more

             at org.apache.hadoop.ipc.Client.call(Client.java:1225)
             at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
             at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
             at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
             at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:396)
             at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
             at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
             at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
             at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
             at java.lang.Thread.run(Thread.java:662)
             at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)


    Input(s):
    Failed to read data from "/user/hadoopuser/excite-small.log"




    On 11 March 2013 03:47, Adam Smieszny wrote:

    Can you do an ls -alh /data/*/mapred and share that output please?

    Thanks,
    Adam

    On Sun, Mar 10, 2013 at 9:47 AM, Praveen Bysani wrote:

    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs




    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data




    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4




    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156


    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com
  • Praveen Bysani at Mar 11, 2013 at 10:18 am
    Hello,

    I resolved this issue. It is because i am running both JobTracker and
    TaskTracker on the same node but have different mapred.local.dir values set
    in the configuration of TaskTracker and JobTracker. It is not straight
    forward from the documentation that these value could conflict.
    On 11 March 2013 10:31, Praveen Bysani wrote:

    Hi,

    The ouput for the command is,

    root@server:/# ls -alh /data/*/mapred
    /data/1/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/2/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/3/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/4/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local


    However i was able to start the TaskTracker successfully by creating a
    similar directory structure outside /data but instead a /mapred directory
    with identical file permissions. But it only lead me to a new problem when
    i tried to run a pig job as user 'hadoopuser' with the following errors in
    the grunt shell even after i configured the mapred.local.dir to valid
    fields in the cofiguration file and restarted the service,

    2013-03-11 10:19:40,645 [JobControl] INFO
    org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://
    server.epicoders.com:8020/user/hadoopuser/.staging/job_201303110217_0001
    2013-03-11 10:19:40,646 [JobControl] ERROR
    org.apache.hadoop.security.UserGroupInformation -
    PriviledgedActionException as:hadoopuser (auth:SIMPLE)
    cause:org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    2013-03-11 10:19:40,647 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2013-03-11 10:19:40,655 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
    recreate exception from backend error:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    at org.apache.hadoop.ipc.Client.call(Client.java:1225)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
    at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
    at java.lang.Thread.run(Thread.java:662)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)

    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
    2013-03-11 10:19:40,667 [main] INFO
    org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

    HadoopVersion PigVersion UserId StartedAt FinishedAt
    Features
    2.0.0-cdh4.2.0 0.10.0-cdh4.2.0 hadoopuser 2013-03-11 10:19:34
    2013-03-11 10:19:40 LIMIT

    Failed!

    Failed Jobs:
    JobId Alias Feature Message Outputs
    N/A clean_log,log Message:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    at org.apache.hadoop.ipc.Client.call(Client.java:1225)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
    at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
    at java.lang.Thread.run(Thread.java:662)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)


    Input(s):
    Failed to read data from "/user/hadoopuser/excite-small.log"





    On 11 March 2013 03:47, Adam Smieszny wrote:

    Can you do an ls -alh /data/*/mapred and share that output please?

    Thanks,
    Adam

    On Sun, Mar 10, 2013 at 9:47 AM, Praveen Bysani wrote:

    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs





    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data





    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4





    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156


    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com
  • Vinithra Varadharajan at Mar 15, 2013 at 3:29 am
    Praveen,

    Your problem is actually that /data has permissions 1700 and is owned by
    hdfs:hadoop. Even though its subdirs are owned by mapred:hadoop, it doesn't
    matter because /data cannot be accessed by user mapred.

    In other words, it is not necessary for mapred.local.dir of the JT and TT
    to match. I'm guessing your JT mapred.local.dir was a directory structure
    that user mapred can read/write to, which is why when you switched to it,
    it worked.

    -Vinithra
    On Mon, Mar 11, 2013 at 3:17 AM, Praveen Bysani wrote:

    Hello,

    I resolved this issue. It is because i am running both JobTracker and
    TaskTracker on the same node but have different mapred.local.dir values set
    in the configuration of TaskTracker and JobTracker. It is not straight
    forward from the documentation that these value could conflict.
    On 11 March 2013 10:31, Praveen Bysani wrote:

    Hi,

    The ouput for the command is,

    root@server:/# ls -alh /data/*/mapred
    /data/1/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/2/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/3/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local

    /data/4/mapred:
    total 12K
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 .
    drwxr-xr-x 3 mapred hadoop 4.0K Mar 10 21:09 ..
    drwxr-xr-x 2 mapred hadoop 4.0K Mar 10 21:09 local


    However i was able to start the TaskTracker successfully by creating a
    similar directory structure outside /data but instead a /mapred directory
    with identical file permissions. But it only lead me to a new problem when
    i tried to run a pig job as user 'hadoopuser' with the following errors in
    the grunt shell even after i configured the mapred.local.dir to valid
    fields in the cofiguration file and restarted the service,

    2013-03-11 10:19:40,645 [JobControl] INFO
    org.apache.hadoop.mapred.JobClient - Cleaning up the staging area hdfs://
    server.epicoders.com:8020/user/hadoopuser/.staging/job_201303110217_0001
    2013-03-11 10:19:40,646 [JobControl] ERROR
    org.apache.hadoop.security.UserGroupInformation -
    PriviledgedActionException as:hadoopuser (auth:SIMPLE)
    cause:org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    2013-03-11 10:19:40,647 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2013-03-11 10:19:40,655 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
    recreate exception from backend error:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    at org.apache.hadoop.ipc.Client.call(Client.java:1225)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
    at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
    at java.lang.Thread.run(Thread.java:662)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)

    2013-03-11 10:19:40,665 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
    2013-03-11 10:19:40,667 [main] INFO
    org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

    HadoopVersion PigVersion UserId StartedAt FinishedAt
    Features
    2.0.0-cdh4.2.0 0.10.0-cdh4.2.0 hadoopuser 2013-03-11 10:19:34
    2013-03-11 10:19:40 LIMIT

    Failed!

    Failed Jobs:
    JobId Alias Feature Message Outputs
    N/A clean_log,log Message:
    org.apache.hadoop.ipc.RemoteException(java.io.IOException):
    java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3509)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3475)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1695)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1691)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1689)
    Caused by: java.io.IOException: No valid local directories in property:
    mapred.local.dir
    at
    org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:1694)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:500)
    at
    org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:414)
    at
    org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3507)
    ... 13 more

    at org.apache.hadoop.ipc.Client.call(Client.java:1225)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
    at org.apache.hadoop.mapred.$Proxy10.submitJob(Unknown Source)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
    at
    org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
    at java.lang.Thread.run(Thread.java:662)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)


    Input(s):
    Failed to read data from "/user/hadoopuser/excite-small.log"





    On 11 March 2013 03:47, Adam Smieszny wrote:

    Can you do an ls -alh /data/*/mapred and share that output please?

    Thanks,
    Adam


    On Sun, Mar 10, 2013 at 9:47 AM, Praveen Bysani <praveen.iiith@gmail.com
    wrote:
    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs







    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data







    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4







    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156


    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com


    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com
  • Farah omer at Sep 30, 2013 at 3:53 pm
    I am having a similar issue, task tracker fails to start on one of the
    nodes.

    Upon running this command, i get the following. Can someone recommend what
    needs to be changed? Thanks.

    -bash-4.1$ sudo ls -alh /data
    total 92K
    drwxr-xr-x 10 hdfs hadoop 4.0K May 13 18:01 .
    dr-xr-xr-x. 37 root root 4.0K Feb 18 2013 ..
    drwxr-xr-x 3 root root 4.0K Mar 16 2012 flatfiles
    drwxrwxrwx 5 hdfs hadoop 4.0K Apr 24 23:37 hadoop-0.20
    drwxr-xr-x 2 hdfs hadoop 4.0K Jan 11 2013 log
    drwx------ 2 hdfs hadoop 16K Jan 6 2011 lost+found
    drwxr-xr-x 4 root root 4.0K Nov 19 2012 metamigrate
    drwxr-xr-x 4 root root 4.0K Feb 11 2011 nzdata
    -rw-r--r-- 1 root root 33K Jan 11 2013 out.out
    drwx------ 4 tedm tedm 4.0K May 15 17:01 tedm
    drwxrwxr-x 2 wxadmin wxadmin 4.0K May 24 2011 wx2

    On Sunday, 10 March 2013 09:47:24 UTC-4, Praveen Bysani wrote:

    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs


    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data


    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4


    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Lakshminarayana reddy at Jan 31, 2014 at 3:51 am
    Hi All,

    The problem might be with directory permissions of of the directory
    corresponding to hadoop.tmp.dir. While running jobs they will try to create
    temp files in hadoop.tmp.dir. If they unable to create the
    file/directories, those jobs will not start.
    In my case problem resolved by changing permminsions.

    Thanks,
    Laklshminarayana K
    On Sunday, 10 March 2013 19:17:24 UTC+5:30, Praveen Bysani wrote:

    Hi,

    I have installed CDH through cloudera manager free edition. The setup
    halts at starting Task trackers on the nodes. I have manually created
    /data/{1,2,3}/mapred/local with user permissions as mapred:hadoop as
    instructed in CDH documentation. Yet while i try to start mapreduce service

    TaskTracker local dir /data/1/mapred/local error File /data/1/mapred/local does not exist, removing from local dirs
    TaskTracker local dir /data/2/mapred/local error File /data/2/mapred/local does not exist, removing from local dirs


    TaskTracker local dir /data/3/mapred/local error File /data/3/mapred/local does not exist, removing from local dirs


    Can not start task tracker because org.apache.hadoop.util.DiskChecker$DiskErrorException: No mapred local directories are writable
    at org.apache.hadoop.mapred.TaskTracker$LocalStorage.checkDirs(TaskTracker.java:279)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1709)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:4040)


    The directory permission structure is as follows,

    drwx------ 10 hdfs hadoop 4096 Mar 10 21:09 data


    and within data,
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 1
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 2
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 3
    drwxr-xr-x 3 mapred hadoop 4096 Mar 10 21:09 4


    Kindly help resolving this.

    --
    Regards,
    Praveen Bysani
    http://www.praveenbysani.com
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 10, '13 at 1:47p
activeJan 31, '14 at 3:51a
posts7
users5
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase