FAQ
I just installed CDH5 beta and am trying to get my existing application
(compiled against CDH4.3, and the MRv1 (mapreduce.*) API) to run under YARN.

My appplication has a bunch of additional dependecies (hibernate. jdbc,
etc) so I used
option 3 from
http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

Basically, I added the additional jar files to a separate directory on each
node
(/opt/myextralibs/) and then added this to /etc/hadoop/conf/hadoop-env.sh

MYLIBS="/opt/myextralibs/jar1.jar:/opt/myextralibs/jar2.jar:/opt/myextralibs/jar3jar"
HADOOP_CLASSPATH=${MYLIBS}
HADOOP_TASKTRACKER_OPTS="-classpath ${MYLIBS} ${HADOOP_TASKTRACKER_OPTS}"

This doesn't seem to work in MRv2 since there's no task tracker.

Is there an equivalent environment variable for the YARN daemon(s) that I
need to set
in order to have the extra jars added to my job's classpath?

I noticed the yarn.application.classpath & mapreduce.application.classpath
settings
in the /etc/hadoop/conf/yarn-site.xml & /etc/hadoop/conf/mapred-site.xml
files, but
adding the additional; jars to those variables didnt seem to have any
effect.
(Yes, I restarted the yarn service before re-running my job.)


Alan

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedNov 15, '13 at 7:09p
activeNov 15, '13 at 7:09p
posts1
users1
websitecloudera.com
irc#hadoop

1 user in discussion

Alan Miller: 1 post

People

Translate

site design / logo © 2022 Grokbase