FAQ
Hi Jon,

YARN clients require a yarn-site.xml configured with address of the
RM, and a mapred-site.xml that carries at least this:

<property><name>mapreduce.framework.name</name><value>yarn</value></property>
On Tue, Aug 28, 2012 at 12:33 AM, Jon Ramos wrote:
Hello Cloudera Manager users,

I'm testing out YARN for the first time for my organization and am having
trouble running a simple YARN grep app. I was hoping somebody should help me
as I seem to be stuck. I used the example from the end of the Cloudera CDH4
Quick Start guide for this.


I have already loaded some XML files into HDFS here:

[root@cdh4-cm ~]# hadoop fs -ls yarn_input
Found 3 items
-rw-r--r-- 1 root supergroup 822 2012-08-27 18:39
yarn_input/core-site.xml
-rw-r--r-- 1 root supergroup 697 2012-08-27 18:39
yarn_input/hdfs-site.xml
-rw-r--r-- 1 root supergroup 1934 2012-08-27 18:39
yarn_input/mapred-site.xml


And I have properly set my environment variable:

[root@cdh4-cm ~]# echo $HADOOP_MAPRED_HOME
/usr/lib/hadoop-mapreduce


Here is the error I get when I try to run the example code:

Note: The Cloudera Manager UI shows that HDFS and YARN services are running
and healthy at the moment. I have also stopped the mapreduce1 service so
there is no incompatibility. Actually all other services hae been stopped.

[root@cdh4-cm ~]# hadoop jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep yarn_input
yarn_output 'dfs[a-z.]+'
12/08/27 18:53:41 INFO mapreduce.Cluster: Failed to use
org.apache.hadoop.mapred.LocalClientProtocolProvider due to error: Invalid
"mapreduce.jobtracker.address" configuration value for LocalJobRunner :
"cdh4-cm:8021"
12/08/27 18:53:41 ERROR security.UserGroupInformation:
PriviledgedActionException as:root (auth:SIMPLE) cause:java.io.IOException:
Cannot initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.
java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1192)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1191)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1220)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
at org.apache.hadoop.examples.Grep.run(Grep.java:77)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.examples.Grep.main(Grep.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
at
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)


What is going on here? How do I fix the "Invalid
"mapreduce.jobtracker.address" configuration value for LocalJobRunner :
"cdh4-cm:8021"???

Also, why is a JobTracker being used when I'm running a YARN example?
Shouldn't it be looking for a NodeManager or ResourceManager instead?

- J


--
Harsh J

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 9 | next ›
Discussion Overview
groupscm-users @
categorieshadoop
postedAug 27, '12 at 7:03p
activeAug 28, '12 at 6:57p
posts9
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase