FAQ
In the old Hadoop mapreduce v1 framework (e.g. 1.0.3) the way to compile a job was using the already provided "hadoop-core-HADOOP_VERSION.jar" through the following command:

javac -cp ".../hadoop-core-HADOOP_VERSION.jar" -d class_dir Example.java

In the newest releases for Hadoop mapreduce v2 YARN (e.g. 2.0.1-alpha or the cloudera 4.0) a similar core.jar is not provided, though, to be used during compilation. The closest i have managed to get is by using the "hadoop-common-2.0.1-alpha.jar" in the /share/hadoop/common dir but this still leaves a lot of compilation errors.

Is there some other specific way or using different jars for compiling a YARN job? Or maybe for some reason you need to build the source yourself in the new releases to get the core.jar? I have searched quite a lot but can't find a relevant answer.

Thank you very much!

Search Discussions

  • Harsh J at Aug 11, 2012 at 3:55 pm
    Hi Pantazis,

    It is better to use maven or other such tools to develop MR Java
    programs, as that handles dependencies for you. In maven you may use
    the hadoop-client POM.

    Alternatively, if you have a hadoop setup available, do this:

    $ javac -cp `hadoop classpath`:. -d class_dir Example.java

    The command "hadoop classpath" will add all requisite jars from the
    installed version to the java(c) classpath automatically.

    On Sat, Aug 11, 2012 at 7:25 AM, Pantazis Deligiannis
    wrote:
    In the old Hadoop mapreduce v1 framework (e.g. 1.0.3) the way to compile a job was using the already provided "hadoop-core-HADOOP_VERSION.jar" through the following command:

    javac -cp ".../hadoop-core-HADOOP_VERSION.jar" -d class_dir Example.java

    In the newest releases for Hadoop mapreduce v2 YARN (e.g. 2.0.1-alpha or the cloudera 4.0) a similar core.jar is not provided, though, to be used during compilation. The closest i have managed to get is by using the "hadoop-common-2.0.1-alpha.jar" in the /share/hadoop/common dir but this still leaves a lot of compilation errors.

    Is there some other specific way or using different jars for compiling a YARN job? Or maybe for some reason you need to build the source yourself in the new releases to get the core.jar? I have searched quite a lot but can't find a relevant answer.

    Thank you very much!


    --
    Harsh J
  • Pantazis Deligiannis at Aug 11, 2012 at 4:17 pm
    Worked perfectly!

    Thanks a lot! :)
    On 11 Aug 2012, at 16:54, Harsh J wrote:

    Hi Pantazis,

    It is better to use maven or other such tools to develop MR Java
    programs, as that handles dependencies for you. In maven you may use
    the hadoop-client POM.

    Alternatively, if you have a hadoop setup available, do this:

    $ javac -cp `hadoop classpath`:. -d class_dir Example.java

    The command "hadoop classpath" will add all requisite jars from the
    installed version to the java(c) classpath automatically.

    On Sat, Aug 11, 2012 at 7:25 AM, Pantazis Deligiannis
    wrote:
    In the old Hadoop mapreduce v1 framework (e.g. 1.0.3) the way to compile a job was using the already provided "hadoop-core-HADOOP_VERSION.jar" through the following command:

    javac -cp ".../hadoop-core-HADOOP_VERSION.jar" -d class_dir Example.java

    In the newest releases for Hadoop mapreduce v2 YARN (e.g. 2.0.1-alpha or the cloudera 4.0) a similar core.jar is not provided, though, to be used during compilation. The closest i have managed to get is by using the "hadoop-common-2.0.1-alpha.jar" in the /share/hadoop/common dir but this still leaves a lot of compilation errors.

    Is there some other specific way or using different jars for compiling a YARN job? Or maybe for some reason you need to build the source yourself in the new releases to get the core.jar? I have searched quite a lot but can't find a relevant answer.

    Thank you very much!


    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedAug 11, '12 at 1:56a
activeAug 11, '12 at 4:17p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Pantazis Deligiannis: 2 posts Harsh J: 1 post

People

Translate

site design / logo © 2021 Grokbase