FAQ
Hi,
I installed cloudera manager on my cluster and installed hdfs, mapreduce,
oozie services.

I cannot able to run streaming wordcount example on the cluster using
oozie. i am getting the following error and getting the job killed

job.properties

*nameNode=hdfs://smvm0:8020*
*jobTracker=smvm0:8021*
*queueName=default*
*examplesRoot=examples*
*
*
*oozie.use.system.libpath=true*
*oozie.libpath=/user/${user.name}/examples/apps/examples-lib*
*
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/streaming
*


workflow.xml

*<workflow-app xmlns="uri:oozie:workflow:0.2" name="streaming-wf">*
* <start to="streaming-node"/>*
* <action name="streaming-node">*
* <map-reduce>*
* <job-tracker>${jobTracker}</job-tracker>*
* <name-node>${nameNode}</name-node>*
* <prepare>*
* <delete
path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/streaming"/>
*
* </prepare>*
* <streaming>*
* <mapper>/home/srikanth/pythonex/mapper.py</mapper>*
* <reducer>/home/srikanth/pythonex/reducer.py</reducer>*
* </streaming>*
* <configuration>*
* <property>*
* <name>mapred.job.queue.name</name>*
* <value>${queueName}</value>*
* </property>*
* <property>*
* <name>mapred.input.dir</name>*
*
<value>/user/${wf:user()}/${examplesRoot}/input-data/text</value>*
* </property>*
* <property>*
* <name>mapred.output.dir</name>*
*
<value>/user/${wf:user()}/${examplesRoot}/output-data/streaming</value>*
* </property>*
* </configuration>*
* </map-reduce>*
* <ok to="end"/>*
* <error to="fail"/>*
* </action>*
* <kill name="fail">*
* <message>Streaming Map/Reduce failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>*
* </kill>*
* <end name="end"/>*
*</workflow-app> *


I uploaded the examples folder on to the hdfs and run job as a oozie user.
The error in the JobTracker log is

*2013-04-09 22:36:35,327 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201304031813_0052_m_000000_3:
java.lang.RuntimeException: Error in configuring object*
* at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)*
* at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
* at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
*
* at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
* at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
* at java.security.AccessController.doPrivileged(Native Method)*
* at javax.security.auth.Subject.doAs(Subject.java:396)*
* at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
*
* at org.apache.hadoop.mapred.Child.main(Child.java:262)*
*Caused by: java.lang.reflect.InvocationTargetException*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
*
*2013-04-09 22:36:35,438 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201304031813_0052_m_000001_3:
java.lang.RuntimeException: Error in configuring object*
* at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)*
* at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
* at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
*
* at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
* at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
* at java.security.AccessController.doPrivileged(Native Method)*
* at javax.security.auth.Subject.doAs(Subject.java:396)*
* at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
*
* at org.apache.hadoop.mapred.Child.main(Child.java:262)*
*Caused by: java.lang.reflect.InvocationTargetException*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
*


the error in the oozie webconsole is

*
*
*2013-04-09 22:36:41,235 WARN
org.apache.oozie.action.hadoop.MapReduceActionExecutor: USER[oozie]
GROUP[-] TOKEN[] APP[streaming-wf]
JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node] LauncherMapper
died, check Hadoop log for job [smvm0:8021:job_201304031813_0052]*
*2013-04-09 22:36:41,408 INFO
org.apache.oozie.command.wf.ActionEndXCommand: USER[oozie] GROUP[-] TOKEN[]
APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node] ERROR is
considered as FAILED for SLA*
*2013-04-09 22:36:42,006 INFO
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail] Start action
[0000005-130409205238858-oozie-oozi-W@fail] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]*
*2013-04-09 22:36:42,006 WARN
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail]
[***0000005-130409205238858-oozie-oozi-W@fail***]Action status=DONE*
*2013-04-09 22:36:42,006 WARN
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail]
[***0000005-130409205238858-oozie-oozi-W@fail***]Action updated in DB!*
*2013-04-09 22:36:42,127 WARN
org.apache.oozie.command.coord.CoordActionUpdateXCommand: USER[oozie]
GROUP[-] TOKEN[] APP[streaming-wf]
JOB[0000005-130409205238858-oozie-oozi-W] ACTION[-] E1100: Command
precondition does not hold before execution, [, coord action is null],
Error Code: E1100 *


Can anyone please help to overcome this error.

Thank you
srikanth.A

Search Discussions

  • Serega Sheypak at Apr 9, 2013 at 9:54 am
    1. Find *attempt_201304031813_0052_m_000000_3 *log folder and see there.
    There shoul be more info. This folder should be in your tasktracker log
    directory.
    2. Maybe there is no access to
    * <property>*
    * <name>mapred.input.dir</name>*
    *
    <value>/user/${wf:user()}/${examplesRoot}/input-data/text</value>*
    * </property>*
    * <property>*
    * <name>mapred.output.dir</name>*
    *
    <value>/user/${wf:user()}/${examplesRoot}/output-data/streaming</value>*
    * </property>*
    *I think "mapred" user should be granted to access your user folder. See
    hadoop fs -ls -R for **/user/${wf:user()}/${examplesRoot}/input-data/text
    and **/user/${wf:user()}/${examplesRoot}/output-data/streaming*


    2013/4/9 srikanth ayalasomayajulu <srikraj8341@gmail.com>
    Hi,
    I installed cloudera manager on my cluster and installed hdfs, mapreduce,
    oozie services.

    I cannot able to run streaming wordcount example on the cluster using
    oozie. i am getting the following error and getting the job killed

    job.properties

    *nameNode=hdfs://smvm0:8020*
    *jobTracker=smvm0:8021*
    *queueName=default*
    *examplesRoot=examples*
    *
    *
    *oozie.use.system.libpath=true*
    *oozie.libpath=/user/${user.name}/examples/apps/examples-lib*
    *oozie.wf.application.path=${nameNode}/user/${user.name
    }/${examplesRoot}/apps/streaming*


    workflow.xml

    *<workflow-app xmlns="uri:oozie:workflow:0.2" name="streaming-wf">*
    * <start to="streaming-node"/>*
    * <action name="streaming-node">*
    * <map-reduce>*
    * <job-tracker>${jobTracker}</job-tracker>*
    * <name-node>${nameNode}</name-node>*
    * <prepare>*
    * <delete
    path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/streaming"/>
    *
    * </prepare>*
    * <streaming>*
    * <mapper>/home/srikanth/pythonex/mapper.py</mapper>*
    * <reducer>/home/srikanth/pythonex/reducer.py</reducer>*
    * </streaming>*
    * <configuration>*
    * <property>*
    * <name>mapred.job.queue.name</name>*
    * <value>${queueName}</value>*
    * </property>*
    * <property>*
    * <name>mapred.input.dir</name>*
    *
    <value>/user/${wf:user()}/${examplesRoot}/input-data/text</value>*
    * </property>*
    * <property>*
    * <name>mapred.output.dir</name>*
    *
    <value>/user/${wf:user()}/${examplesRoot}/output-data/streaming</value>*
    * </property>*
    * </configuration>*
    * </map-reduce>*
    * <ok to="end"/>*
    * <error to="fail"/>*
    * </action>*
    * <kill name="fail">*
    * <message>Streaming Map/Reduce failed, error
    message[${wf:errorMessage(wf:lastErrorNode())}]</message>*
    * </kill>*
    * <end name="end"/>*
    *</workflow-app> *


    I uploaded the examples folder on to the hdfs and run job as a oozie user.
    The error in the JobTracker log is

    *2013-04-09 22:36:35,327 INFO org.apache.hadoop.mapred.TaskInProgress:
    Error from attempt_201304031813_0052_m_000000_3:
    java.lang.RuntimeException: Error in configuring object*
    * at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    *
    * at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
    * at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    *
    * at
    org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
    * at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
    * at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
    * at java.security.AccessController.doPrivileged(Native Method)*
    * at javax.security.auth.Subject.doAs(Subject.java:396)*
    * at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    *
    * at org.apache.hadoop.mapred.Child.main(Child.java:262)*
    *Caused by: java.lang.reflect.InvocationTargetException*
    * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
    * at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    *
    * at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
    *
    *2013-04-09 22:36:35,438 INFO org.apache.hadoop.mapred.TaskInProgress:
    Error from attempt_201304031813_0052_m_000001_3:
    java.lang.RuntimeException: Error in configuring object*
    * at
    org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    *
    * at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
    * at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
    *
    * at
    org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
    * at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
    * at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
    * at java.security.AccessController.doPrivileged(Native Method)*
    * at javax.security.auth.Subject.doAs(Subject.java:396)*
    * at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    *
    * at org.apache.hadoop.mapred.Child.main(Child.java:262)*
    *Caused by: java.lang.reflect.InvocationTargetException*
    * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
    * at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    *
    * at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
    *


    the error in the oozie webconsole is

    *
    *
    *2013-04-09 22:36:41,235 WARN
    org.apache.oozie.action.hadoop.MapReduceActionExecutor: USER[oozie]
    GROUP[-] TOKEN[] APP[streaming-wf]
    JOB[0000005-130409205238858-oozie-oozi-W]
    ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node]
    LauncherMapper died, check Hadoop log for job
    [smvm0:8021:job_201304031813_0052]*
    *2013-04-09 22:36:41,408 INFO
    org.apache.oozie.command.wf.ActionEndXCommand: USER[oozie] GROUP[-] TOKEN[]
    APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
    ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node] ERROR is
    considered as FAILED for SLA*
    *2013-04-09 22:36:42,006 INFO
    org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
    TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
    ACTION[0000005-130409205238858-oozie-oozi-W@fail] Start action
    [0000005-130409205238858-oozie-oozi-W@fail] with user-retry state :
    userRetryCount [0], userRetryMax [0], userRetryInterval [10]*
    *2013-04-09 22:36:42,006 WARN
    org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
    TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
    ACTION[0000005-130409205238858-oozie-oozi-W@fail]
    [***0000005-130409205238858-oozie-oozi-W@fail***]Action status=DONE*
    *2013-04-09 22:36:42,006 WARN
    org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
    TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
    ACTION[0000005-130409205238858-oozie-oozi-W@fail]
    [***0000005-130409205238858-oozie-oozi-W@fail***]Action updated in DB!*
    *2013-04-09 22:36:42,127 WARN
    org.apache.oozie.command.coord.CoordActionUpdateXCommand: USER[oozie]
    GROUP[-] TOKEN[] APP[streaming-wf]
    JOB[0000005-130409205238858-oozie-oozi-W] ACTION[-] E1100: Command
    precondition does not hold before execution, [, coord action is null],
    Error Code: E1100 *


    Can anyone please help to overcome this error.

    Thank you
    srikanth.A


Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedApr 9, '13 at 7:07a
activeApr 9, '13 at 9:54a
posts2
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase