FAQ
Hi Guys,
My namenode suffered a boot disk failure. Luckily I was backed up (phew)
and recovered the cluster. I also updated to the latest CHD4.

However I can not run any streaming map-reduce now :-(

Here is the error:

$ ./get_email.sh
12/12/21 11:09:23 WARN streaming.StreamJob: -file option is deprecated,
please use generic option -files instead.
packageJobJar: [to_list_mapper.py, to_list, auto_get_email_reducer.py] []
/tmp/streamjob5605477836506937607.jar tmpDir=null
12/12/21 11:09:24 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
12/12/21 11:09:25 INFO mapred.FileInputFormat: Total input paths to process
: 8
12/12/21 11:09:26 INFO mapred.JobClient: Running job: job_201212202308_0005
12/12/21 11:09:27 INFO mapred.JobClient: map 0% reduce 0%
12/12/21 11:09:34 INFO mapred.JobClient: Task Id :
attempt_201212202308_0005_m_000003_0, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.streaming.PipeMapRunner not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:1010)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:389)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class org.apache.hadoop.streaming.PipeMapRunner not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1571)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1595)


$ ls -la /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
lrwxrwxrwx 1 root root 35 Dec 20 20:45
/usr/lib/hadoop-mapreduce/hadoop-streaming.jar ->
hadoop-streaming-2.0.0-cdh4.1.2.jar

Thanks
Paul

--

Search Discussions

  • Bob_the_web at Dec 21, 2012 at 3:42 pm
    Solved it.

    Wrong jar. Documentation could be clearer?

    Answer was to use this jar:

    /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.2.jar



    On Friday, December 21, 2012 11:15:38 AM UTC, bob_the_web wrote:

    Hi Guys,
    My namenode suffered a boot disk failure. Luckily I was backed up (phew)
    and recovered the cluster. I also updated to the latest CHD4.

    However I can not run any streaming map-reduce now :-(

    Here is the error:

    $ ./get_email.sh
    12/12/21 11:09:23 WARN streaming.StreamJob: -file option is deprecated,
    please use generic option -files instead.
    packageJobJar: [to_list_mapper.py, to_list, auto_get_email_reducer.py] []
    /tmp/streamjob5605477836506937607.jar tmpDir=null
    12/12/21 11:09:24 WARN mapred.JobClient: Use GenericOptionsParser for
    parsing the arguments. Applications should implement Tool for the same.
    12/12/21 11:09:25 INFO mapred.FileInputFormat: Total input paths to
    process : 8
    12/12/21 11:09:26 INFO mapred.JobClient: Running job: job_201212202308_0005
    12/12/21 11:09:27 INFO mapred.JobClient: map 0% reduce 0%
    12/12/21 11:09:34 INFO mapred.JobClient: Task Id :
    attempt_201212202308_0005_m_000003_0, Status : FAILED
    java.lang.RuntimeException: java.lang.RuntimeException:
    java.lang.ClassNotFoundException: Class
    org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
    at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:1010)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:389)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
    Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
    Class org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1571)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1595)


    $ ls -la /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
    lrwxrwxrwx 1 root root 35 Dec 20 20:45
    /usr/lib/hadoop-mapreduce/hadoop-streaming.jar ->
    hadoop-streaming-2.0.0-cdh4.1.2.jar

    Thanks
    Paul
    --
  • Abhijith Gopal at Jan 18, 2013 at 9:48 am
    Thanks for posting the solution. Helped me resolve the problem quickly.
    On Friday, 21 December 2012 21:12:30 UTC+5:30, bob_the_web wrote:

    Solved it.

    Wrong jar. Documentation could be clearer?

    Answer was to use this jar:


    /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.2.jar



    On Friday, December 21, 2012 11:15:38 AM UTC, bob_the_web wrote:

    Hi Guys,
    My namenode suffered a boot disk failure. Luckily I was backed up (phew)
    and recovered the cluster. I also updated to the latest CHD4.

    However I can not run any streaming map-reduce now :-(

    Here is the error:

    $ ./get_email.sh
    12/12/21 11:09:23 WARN streaming.StreamJob: -file option is deprecated,
    please use generic option -files instead.
    packageJobJar: [to_list_mapper.py, to_list, auto_get_email_reducer.py] []
    /tmp/streamjob5605477836506937607.jar tmpDir=null
    12/12/21 11:09:24 WARN mapred.JobClient: Use GenericOptionsParser for
    parsing the arguments. Applications should implement Tool for the same.
    12/12/21 11:09:25 INFO mapred.FileInputFormat: Total input paths to
    process : 8
    12/12/21 11:09:26 INFO mapred.JobClient: Running job:
    job_201212202308_0005
    12/12/21 11:09:27 INFO mapred.JobClient: map 0% reduce 0%
    12/12/21 11:09:34 INFO mapred.JobClient: Task Id :
    attempt_201212202308_0005_m_000003_0, Status : FAILED
    java.lang.RuntimeException: java.lang.RuntimeException:
    java.lang.ClassNotFoundException: Class
    org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
    at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:1010)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:389)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
    Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
    Class org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1571)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1595)


    $ ls -la /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
    lrwxrwxrwx 1 root root 35 Dec 20 20:45
    /usr/lib/hadoop-mapreduce/hadoop-streaming.jar ->
    hadoop-streaming-2.0.0-cdh4.1.2.jar

    Thanks
    Paul
    --
  • Iman saleh at May 17, 2014 at 2:11 am
    Thanks! That helped me as well.
    On Friday, December 21, 2012 10:42:30 AM UTC-5, bob_the_web wrote:

    Solved it.

    Wrong jar. Documentation could be clearer?

    Answer was to use this jar:


    /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.2.jar



    On Friday, December 21, 2012 11:15:38 AM UTC, bob_the_web wrote:

    Hi Guys,
    My namenode suffered a boot disk failure. Luckily I was backed up (phew)
    and recovered the cluster. I also updated to the latest CHD4.

    However I can not run any streaming map-reduce now :-(

    Here is the error:

    $ ./get_email.sh
    12/12/21 11:09:23 WARN streaming.StreamJob: -file option is deprecated,
    please use generic option -files instead.
    packageJobJar: [to_list_mapper.py, to_list, auto_get_email_reducer.py] []
    /tmp/streamjob5605477836506937607.jar tmpDir=null
    12/12/21 11:09:24 WARN mapred.JobClient: Use GenericOptionsParser for
    parsing the arguments. Applications should implement Tool for the same.
    12/12/21 11:09:25 INFO mapred.FileInputFormat: Total input paths to
    process : 8
    12/12/21 11:09:26 INFO mapred.JobClient: Running job:
    job_201212202308_0005
    12/12/21 11:09:27 INFO mapred.JobClient: map 0% reduce 0%
    12/12/21 11:09:34 INFO mapred.JobClient: Task Id :
    attempt_201212202308_0005_m_000003_0, Status : FAILED
    java.lang.RuntimeException: java.lang.RuntimeException:
    java.lang.ClassNotFoundException: Class
    org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
    at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:1010)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:389)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
    Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
    Class org.apache.hadoop.streaming.PipeMapRunner not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1571)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1595)


    $ ls -la /usr/lib/hadoop-mapreduce/hadoop-streaming.jar
    lrwxrwxrwx 1 root root 35 Dec 20 20:45
    /usr/lib/hadoop-mapreduce/hadoop-streaming.jar ->
    hadoop-streaming-2.0.0-cdh4.1.2.jar

    Thanks
    Paul
    --

    ---
    You received this message because you are subscribed to the Google Groups "CDH Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+unsubscribe@cloudera.org.
    For more options, visit https://groups.google.com/a/cloudera.org/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcdh-user @
categorieshadoop
postedDec 21, '12 at 11:15a
activeMay 17, '14 at 2:11a
posts4
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2018 Grokbase