FAQ
Dear All,
I installed CDH4 with cloudera manager. The system is running well. But when I want to run map reduce program is show error.

packageJobJar: [] [/usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.0.0.jar] /tmp/streamjob2452271508971633969.jar tmpDir=null
13/01/09 14:40:32 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/01/09 14:40:32 WARN snappy.LoadSnappy: Snappy native library is available
13/01/09 14:40:32 INFO snappy.LoadSnappy: Snappy native library loaded
13/01/09 14:40:32 INFO mapred.FileInputFormat: Total input paths to process : 4
13/01/09 14:40:32 INFO mapreduce.JobSubmitter: number of splits:5
13/01/09 14:40:32 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/01/09 14:40:32 WARN conf.Configuration: mapred.create.symlink is deprecated. Instead, use mapreduce.job.cache.symlink.create
13/01/09 14:40:32 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
13/01/09 14:40:32 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/01/09 14:40:32 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/01/09 14:40:32 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/01/09 14:40:32 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/01/09 14:40:32 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
13/01/09 14:40:32 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/01/09 14:40:33 INFO mapred.ResourceMgrDelegate: Submitted application application_1355984108641_0042 to ResourceManager at master.bridgetree.com/192.168.10.85:8032
13/01/09 14:40:33 INFO mapreduce.Job: The url to track the job: http://master.bridgetree.com:8088/proxy/application_1355984108641_0042/
13/01/09 14:40:33 INFO mapreduce.Job: Running job: job_1355984108641_0042
13/01/09 14:40:38 INFO mapreduce.Job: Job job_1355984108641_0042 running in uber mode : false
13/01/09 14:40:38 INFO mapreduce.Job: map 0% reduce 0%
13/01/09 14:40:41 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
... 23 more
Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more

13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stdout
13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stderr
13/01/09 14:40:44 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
... 23 more
Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more

13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stdout
13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stderr
13/01/09 14:40:48 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
... 23 more
Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more

13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stdout
13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stderr
13/01/09 14:40:51 INFO mapreduce.Job: Job job_1355984108641_0042 failed with state FAILED due to:
13/01/09 14:40:51 INFO mapreduce.Job: Counters: 6
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=61864
Total time spent by all reduces in occupied slots (ms)=0
13/01/09 14:40:51 ERROR streaming.StreamJob: Job not Successful!
Streaming Command Failed!

Can any one help to fix the problem.

Thanks & regards
Arindam Mitra

Search Discussions

  • bc Wong at Jan 9, 2013 at 6:24 pm
    What's the permission on "/home/hadoop/mapper.php"?
    On Wed, Jan 9, 2013 at 1:18 AM, Arindam Mitra wrote:


    Dear All,
    I installed CDH4 with cloudera manager. The system is running well. But
    when I want to run map reduce program is show error.

    packageJobJar: []
    [/usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.0.0.jar]
    /tmp/streamjob2452271508971633969.jar tmpDir=null
    13/01/09 14:40:32 INFO util.NativeCodeLoader: Loaded the native-hadoop
    library
    13/01/09 14:40:32 WARN snappy.LoadSnappy: Snappy native library is
    available
    13/01/09 14:40:32 INFO snappy.LoadSnappy: Snappy native library loaded
    13/01/09 14:40:32 INFO mapred.FileInputFormat: Total input paths to
    process : 4
    13/01/09 14:40:32 INFO mapreduce.JobSubmitter: number of splits:5
    13/01/09 14:40:32 WARN conf.Configuration: mapred.jar is deprecated.
    Instead, use mapreduce.job.jar
    13/01/09 14:40:32 WARN conf.Configuration: mapred.create.symlink is
    deprecated. Instead, use mapreduce.job.cache.symlink.create
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.value.class is
    deprecated. Instead, use mapreduce.job.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.value.class is
    deprecated. Instead, use mapreduce.map.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.job.name is deprecated.
    Instead, use mapreduce.job.name
    13/01/09 14:40:32 WARN conf.Configuration: mapred.input.dir is deprecated.
    Instead, use mapreduce.input.fileinputformat.inputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.dir is
    deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.map.tasks is deprecated.
    Instead, use mapreduce.job.maps
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.key.class is
    deprecated. Instead, use mapreduce.job.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.key.class is
    deprecated. Instead, use mapreduce.map.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.working.dir is
    deprecated. Instead, use mapreduce.job.working.dir
    13/01/09 14:40:33 INFO mapred.ResourceMgrDelegate: Submitted application
    application_1355984108641_0042 to ResourceManager at
    master.bridgetree.com/192.168.10.85:8032
    13/01/09 14:40:33 INFO mapreduce.Job: The url to track the job:
    http://master.bridgetree.com:8088/proxy/application_1355984108641_0042/
    13/01/09 14:40:33 INFO mapreduce.Job: Running job: job_1355984108641_0042
    13/01/09 14:40:38 INFO mapreduce.Job: Job job_1355984108641_0042 running
    in uber mode : false
    13/01/09 14:40:38 INFO mapreduce.Job: map 0% reduce 0%
    13/01/09 14:40:41 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_0, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stdout
    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stderr
    13/01/09 14:40:44 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_1, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stdout
    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stderr
    13/01/09 14:40:48 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_2, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stdout
    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stderr
    13/01/09 14:40:51 INFO mapreduce.Job: Job job_1355984108641_0042 failed
    with state FAILED due to:
    13/01/09 14:40:51 INFO mapreduce.Job: Counters: 6
    Job Counters
    Failed map tasks=4
    Launched map tasks=4
    Other local map tasks=3
    Data-local map tasks=1
    Total time spent by all maps in occupied slots (ms)=61864
    Total time spent by all reduces in occupied slots (ms)=0
    13/01/09 14:40:51 ERROR streaming.StreamJob: Job not Successful!
    Streaming Command Failed!

    Can any one help to fix the problem.

    Thanks & regards
    Arindam Mitra
  • bc Wong at Jan 10, 2013 at 9:47 am
    Please include the mailing list when replying. What's the mount options of
    /home/hadoop? Can you show `cat /proc/mounts'? Does it have noexec?

    Cheers,
    bc
    On Wed, Jan 9, 2013 at 11:21 PM, Arindam Mitra wrote:

    Dear bc,
    Thanks for reply. My present permission of mapper.php is

    -rwxrwxrwx 1 mapred hadoop 699 Jan 9 16:06 mapper.php
    -rwxrwxrwx 1 mapred hadoop 538 Jan 10 11:00 mapper.py
    -rwxrwxrwx 1 mapred hadoop 760 Jan 9 16:06 reducer.php
    -rwxrwxrwx 1 mapred hadoop 1028 Jan 10 11:01 reducer.py

    Thanks & regards
    Arindam Mitra
    Bridgetree Consultancy Pvt Ltd.
    Kolkata-700156, India.
    Mobile No:-8296042823
    Email: amitra@bridgetree.com
    Contact us: www.bridgetree.com
    ________________________________________
    From: bc Wong [bcwalrus@cloudera.com]
    Sent: Wednesday, January 09, 2013 11:53 PM
    To: Arindam Mitra
    Cc: scm-users@cloudera.org
    Subject: Re: Permission denied error on map reduce

    What's the permission on "/home/hadoop/mapper.php"?

    On Wed, Jan 9, 2013 at 1:18 AM, Arindam Mitra wrote:

    Dear All,
    I installed CDH4 with cloudera manager. The system is running well. But
    when I want to run map reduce program is show error.

    packageJobJar: []
    [/usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.0.0.jar]
    /tmp/streamjob2452271508971633969.jar tmpDir=null
    13/01/09 14:40:32 INFO util.NativeCodeLoader: Loaded the native-hadoop
    library
    13/01/09 14:40:32 WARN snappy.LoadSnappy: Snappy native library is
    available
    13/01/09 14:40:32 INFO snappy.LoadSnappy: Snappy native library loaded
    13/01/09 14:40:32 INFO mapred.FileInputFormat: Total input paths to
    process : 4
    13/01/09 14:40:32 INFO mapreduce.JobSubmitter: number of splits:5
    13/01/09 14:40:32 WARN conf.Configuration: mapred.jar is deprecated.
    Instead, use mapreduce.job.jar
    13/01/09 14:40:32 WARN conf.Configuration: mapred.create.symlink is
    deprecated. Instead, use mapreduce.job.cache.symlink.create
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.value.class is
    deprecated. Instead, use mapreduce.job.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.value.class is
    deprecated. Instead, use mapreduce.map.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.job.name<
    http://mapred.job.name> is deprecated. Instead, use mapreduce.job.name<
    http://mapreduce.job.name>
    13/01/09 14:40:32 WARN conf.Configuration: mapred.input.dir is deprecated.
    Instead, use mapreduce.input.fileinputformat.inputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.dir is
    deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.map.tasks is deprecated.
    Instead, use mapreduce.job.maps
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.key.class is
    deprecated. Instead, use mapreduce.job.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.key.class is
    deprecated. Instead, use mapreduce.map.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.working.dir is
    deprecated. Instead, use mapreduce.job.working.dir
    13/01/09 14:40:33 INFO mapred.ResourceMgrDelegate: Submitted application
    application_1355984108641_0042 to ResourceManager at
    master.bridgetree.com/192.168.10.85:8032
    13/01/09<http://master.bridgetree.com/192.168.10.85:8032
    13/01/09> 14:40:33 INFO mapreduce.Job: The url to track the job:
    http://master.bridgetree.com:8088/proxy/application_1355984108641_0042/
    13/01/09 14:40:33 INFO mapreduce.Job: Running job: job_1355984108641_0042
    13/01/09 14:40:38 INFO mapreduce.Job: Job job_1355984108641_0042 running
    in uber mode : false
    13/01/09 14:40:38 INFO mapreduce.Job: map 0% reduce 0%
    13/01/09 14:40:41 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_0, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stdout
    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stderr
    13/01/09 14:40:44 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_1, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stdout
    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stderr
    13/01/09 14:40:48 INFO mapreduce.Job: Task Id :
    attempt_1355984108641_0042_m_000000_2, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at
    org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program
    "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at
    org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission
    denied
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stdout
    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server
    returned HTTP response code: 400 for URL:
    http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stderr
    13/01/09 14:40:51 INFO mapreduce.Job: Job job_1355984108641_0042 failed
    with state FAILED due to:
    13/01/09 14:40:51 INFO mapreduce.Job: Counters: 6
    Job Counters
    Failed map tasks=4
    Launched map tasks=4
    Other local map tasks=3
    Data-local map tasks=1
    Total time spent by all maps in occupied slots (ms)=61864
    Total time spent by all reduces in occupied slots (ms)=0
    13/01/09 14:40:51 ERROR streaming.StreamJob: Job not Successful!
    Streaming Command Failed!

    Can any one help to fix the problem.

    Thanks & regards
    Arindam Mitra
  • Arindam Mitra at Jan 10, 2013 at 10:54 am
    Dear bc,
    The job is done. The file owner is mapred but I was try to run it from hadoop user. When I login to the mapred user and run than it is working fine.

    Thanks & regards
    Arindam Mitra
    Bridgetree Consultancy Pvt Ltd.
    Kolkata-700156, India.
    Mobile No:-8296042823
    Email: amitra@bridgetree.com
    Contact us: www.bridgetree.com
    ________________________________________
    From: bc Wong [bcwalrus@cloudera.com]
    Sent: Thursday, January 10, 2013 3:16 PM
    To: Arindam Mitra
    Cc: scm-users
    Subject: Re: Permission denied error on map reduce

    Please include the mailing list when replying. What's the mount options of /home/hadoop? Can you show `cat /proc/mounts'? Does it have noexec?

    Cheers,
    bc

    On Wed, Jan 9, 2013 at 11:21 PM, Arindam Mitra wrote:
    Dear bc,
    Thanks for reply. My present permission of mapper.php is

    -rwxrwxrwx 1 mapred hadoop 699 Jan 9 16:06 mapper.php
    -rwxrwxrwx 1 mapred hadoop 538 Jan 10 11:00 mapper.py
    -rwxrwxrwx 1 mapred hadoop 760 Jan 9 16:06 reducer.php
    -rwxrwxrwx 1 mapred hadoop 1028 Jan 10 11:01 reducer.py

    Thanks & regards
    Arindam Mitra
    Bridgetree Consultancy Pvt Ltd.
    Kolkata-700156, India.
    Mobile No:-8296042823<tel:8296042823>
    Email: amitra@bridgetree.com
    Contact us: www.bridgetree.com<http://www.bridgetree.com>
    ________________________________________
    From: bc Wong [bcwalrus@cloudera.com ]
    Sent: Wednesday, January 09, 2013 11:53 PM
    To: Arindam Mitra
    Cc: scm-users@cloudera.org
    Subject: Re: Permission denied error on map reduce

    What's the permission on "/home/hadoop/mapper.php"?

    On Wed, Jan 9, 2013 at 1:18 AM, Arindam Mitra <amitra@bridgetree.com wrote:

    Dear All,
    I installed CDH4 with cloudera manager. The system is running well. But when I want to run map reduce program is show error.

    packageJobJar: [] [/usr/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.0.0.jar] /tmp/streamjob2452271508971633969.jar tmpDir=null
    13/01/09 14:40:32 INFO util.NativeCodeLoader: Loaded the native-hadoop library
    13/01/09 14:40:32 WARN snappy.LoadSnappy: Snappy native library is available
    13/01/09 14:40:32 INFO snappy.LoadSnappy: Snappy native library loaded
    13/01/09 14:40:32 INFO mapred.FileInputFormat: Total input paths to process : 4
    13/01/09 14:40:32 INFO mapreduce.JobSubmitter: number of splits:5
    13/01/09 14:40:32 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
    13/01/09 14:40:32 WARN conf.Configuration: mapred.create.symlink is deprecated. Instead, use mapreduce.job.cache.symlink.create
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.job.name<http://mapred.job.name><http://mapred.job.name> is deprecated. Instead, use mapreduce.job.name<http://mapreduce.job.name><http://mapreduce.job.name>
    13/01/09 14:40:32 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
    13/01/09 14:40:32 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
    13/01/09 14:40:32 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
    13/01/09 14:40:32 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
    13/01/09 14:40:33 INFO mapred.ResourceMgrDelegate: Submitted application application_1355984108641_0042 to ResourceManager at master.bridgetree.com/192.168.10.85:8032<http://master.bridgetree.com/192.168.10.85:8032>
    13/01/09<http://master.bridgetree.com/192.168.10.85:8032
    13/01/09> 14:40:33 INFO mapreduce.Job: The url to track the job: http://master.bridgetree.com:8088/proxy/application_1355984108641_0042/
    13/01/09 14:40:33 INFO mapreduce.Job: Running job: job_1355984108641_0042
    13/01/09 14:40:38 INFO mapreduce.Job: Job job_1355984108641_0042 running in uber mode : false
    13/01/09 14:40:38 INFO mapreduce.Job: map 0% reduce 0%
    13/01/09 14:40:41 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_0, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
    at java.lang.UNIXProcess.(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stdout
    13/01/09 14:40:41 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_0&filter=stderr
    13/01/09 14:40:44 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_1, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
    at java.lang.UNIXProcess.(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stdout
    13/01/09 14:40:44 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_1&filter=stderr
    13/01/09 14:40:48 INFO mapreduce.Job: Task Id : attempt_1355984108641_0042_m_000000_2, Status : FAILED
    Error: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
    ... 17 more
    Caused by: java.lang.RuntimeException: configuration exception
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
    ... 22 more
    Caused by: java.io.IOException: Cannot run program "/home/hadoop/mapper.php": java.io.IOException: error=13, Permission denied
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
    ... 23 more
    Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
    at java.lang.UNIXProcess.(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 24 more

    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stdout
    13/01/09 14:40:48 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://slave.bridgetree.com:8080/tasklog?plaintext=true&attemptid=attempt_1355984108641_0042_m_000000_2&filter=stderr
    13/01/09 14:40:51 INFO mapreduce.Job: Job job_1355984108641_0042 failed with state FAILED due to:
    13/01/09 14:40:51 INFO mapreduce.Job: Counters: 6
    Job Counters
    Failed map tasks=4
    Launched map tasks=4
    Other local map tasks=3
    Data-local map tasks=1
    Total time spent by all maps in occupied slots (ms)=61864
    Total time spent by all reduces in occupied slots (ms)=0
    13/01/09 14:40:51 ERROR streaming.StreamJob: Job not Successful!
    Streaming Command Failed!

    Can any one help to fix the problem.

    Thanks & regards
    Arindam Mitra

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJan 9, '13 at 9:19a
activeJan 10, '13 at 10:54a
posts4
users2
websitecloudera.com
irc#hadoop

2 users in discussion

bc Wong: 2 posts Arindam Mitra: 2 posts

People

Translate

site design / logo © 2022 Grokbase