Hi,
I installed cloudera manager on my cluster and installed hdfs, mapreduce,
oozie services.
I cannot able to run streaming wordcount example on the cluster using
oozie. i am getting the following error and getting the job killed
job.properties
*nameNode=hdfs://smvm0:8020*
*jobTracker=smvm0:8021*
*queueName=default*
*examplesRoot=examples*
*
*
*oozie.use.system.libpath=true*
*oozie.libpath=/user/${user.name}/examples/apps/examples-lib*
*
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/streaming
*
workflow.xml
*<workflow-app xmlns="uri:oozie:workflow:0.2" name="streaming-wf">*
* <start to="streaming-node"/>*
* <action name="streaming-node">*
* <map-reduce>*
* <job-tracker>${jobTracker}</job-tracker>*
* <name-node>${nameNode}</name-node>*
* <prepare>*
* <delete
path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/streaming"/>
*
* </prepare>*
* <streaming>*
* <mapper>/home/srikanth/pythonex/mapper.py</mapper>*
* <reducer>/home/srikanth/pythonex/reducer.py</reducer>*
* </streaming>*
* <configuration>*
* <property>*
* <name>mapred.job.queue.name</name>*
* <value>${queueName}</value>*
* </property>*
* <property>*
* <name>mapred.input.dir</name>*
*
<value>/user/${wf:user()}/${examplesRoot}/input-data/text</value>*
* </property>*
* <property>*
* <name>mapred.output.dir</name>*
*
<value>/user/${wf:user()}/${examplesRoot}/output-data/streaming</value>*
* </property>*
* </configuration>*
* </map-reduce>*
* <ok to="end"/>*
* <error to="fail"/>*
* </action>*
* <kill name="fail">*
* <message>Streaming Map/Reduce failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>*
* </kill>*
* <end name="end"/>*
*</workflow-app> *
I uploaded the examples folder on to the hdfs and run job as a oozie user.
The error in the JobTracker log is
*2013-04-09 22:36:35,327 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201304031813_0052_m_000000_3:
java.lang.RuntimeException: Error in configuring object*
* at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)*
* at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
* at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
*
* at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
* at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
* at java.security.AccessController.doPrivileged(Native Method)*
* at javax.security.auth.Subject.doAs(Subject.java:396)*
* at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
*
* at org.apache.hadoop.mapred.Child.main(Child.java:262)*
*Caused by: java.lang.reflect.InvocationTargetException*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
*
*2013-04-09 22:36:35,438 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201304031813_0052_m_000001_3:
java.lang.RuntimeException: Error in configuring object*
* at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)*
* at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)*
* at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
*
* at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:414)*
* at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)*
* at org.apache.hadoop.mapred.Child$4.run(Child.java:268)*
* at java.security.AccessController.doPrivileged(Native Method)*
* at javax.security.auth.Subject.doAs(Subject.java:396)*
* at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
*
* at org.apache.hadoop.mapred.Child.main(Child.java:262)*
*Caused by: java.lang.reflect.InvocationTargetException*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
*
the error in the oozie webconsole is
*
*
*2013-04-09 22:36:41,235 WARN
org.apache.oozie.action.hadoop.MapReduceActionExecutor: USER[oozie]
GROUP[-] TOKEN[] APP[streaming-wf]
JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node] LauncherMapper
died, check Hadoop log for job [smvm0:8021:job_201304031813_0052]*
*2013-04-09 22:36:41,408 INFO
org.apache.oozie.command.wf.ActionEndXCommand: USER[oozie] GROUP[-] TOKEN[]
APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@streaming-node] ERROR is
considered as FAILED for SLA*
*2013-04-09 22:36:42,006 INFO
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail] Start action
[0000005-130409205238858-oozie-oozi-W@fail] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10]*
*2013-04-09 22:36:42,006 WARN
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail]
[***0000005-130409205238858-oozie-oozi-W@fail***]Action status=DONE*
*2013-04-09 22:36:42,006 WARN
org.apache.oozie.command.wf.ActionStartXCommand: USER[oozie] GROUP[-]
TOKEN[] APP[streaming-wf] JOB[0000005-130409205238858-oozie-oozi-W]
ACTION[0000005-130409205238858-oozie-oozi-W@fail]
[***0000005-130409205238858-oozie-oozi-W@fail***]Action updated in DB!*
*2013-04-09 22:36:42,127 WARN
org.apache.oozie.command.coord.CoordActionUpdateXCommand: USER[oozie]
GROUP[-] TOKEN[] APP[streaming-wf]
JOB[0000005-130409205238858-oozie-oozi-W] ACTION[-] E1100: Command
precondition does not hold before execution, [, coord action is null],
Error Code: E1100 *
Can anyone please help to overcome this error.
Thank you
srikanth.A
