FAQ
Hi,

I want to add a jar file (that is required by mappers and reducers) to the
classpath. Initially I had copied the jar file to all the slave nodes in the
$HADOOP_HOME/lib directory and it was working fine.

However when I tried the libjars option to add jar files -

$HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar


I got this error-

java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

Can someone please tell me what needs to be fixed here ?

Thanks,
Taran

Search Discussions

  • Mahadev Konar at Oct 6, 2008 at 9:47 pm
    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev

    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes in the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar


    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran
  • Tarandeep Singh at Oct 6, 2008 at 9:56 pm
    thanks Mahadev for the reply.
    So that means I have to copy my jar file in the $HADOOP_HOME/lib folder on
    all slave machines like before.

    One more question- I am adding a conf file (just like HADOOP_SITE.xml) via
    -conf option and I am able to query parameters in mapper/reducers. But is
    there a way I can query the parameters in my job driver class -

    public class jobDriver extends Configured
    {
    someMethod( )
    {
    ToolRunner.run( new MyJob( ), commandLineArgs);
    // I want to query parameters present in my conf file here
    }
    }

    public class MyJob extends Configured implements Tool
    {
    }

    Thanks,
    Taran
    On Mon, Oct 6, 2008 at 2:46 PM, Mahadev Konar wrote:

    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev

    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes in the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar

    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran
  • Taeho Kang at Oct 7, 2008 at 2:37 am
    Adding your jar files in the $HADOOP_HOME/lib folder works, but you would
    have to restart all your tasktrackers to have your jar files loaded.

    If you repackage your map-reduce jar file (e.g. hadoop-0.18.0-examples.jar)
    with your jar file and run your job with the newly repackaged jar file, it
    would work, too.
    On Tue, Oct 7, 2008 at 6:55 AM, Tarandeep Singh wrote:

    thanks Mahadev for the reply.
    So that means I have to copy my jar file in the $HADOOP_HOME/lib folder on
    all slave machines like before.

    One more question- I am adding a conf file (just like HADOOP_SITE.xml) via
    -conf option and I am able to query parameters in mapper/reducers. But is
    there a way I can query the parameters in my job driver class -

    public class jobDriver extends Configured
    {
    someMethod( )
    {
    ToolRunner.run( new MyJob( ), commandLineArgs);
    // I want to query parameters present in my conf file here
    }
    }

    public class MyJob extends Configured implements Tool
    {
    }

    Thanks,
    Taran
    On Mon, Oct 6, 2008 at 2:46 PM, Mahadev Konar wrote:

    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev

    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes
    in
    the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar

    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran
  • Mahadev Konar at Oct 7, 2008 at 3:34 am
    You can just add the jar to the env variable HADOOP_CLASSPATH

    If using bash

    Just do this :

    Export HADOOP_CLASSPATH=path to your class path on the client

    And then use the libjars option.

    mahadev

    On 10/6/08 2:55 PM, "Tarandeep Singh" wrote:

    thanks Mahadev for the reply.
    So that means I have to copy my jar file in the $HADOOP_HOME/lib folder on
    all slave machines like before.

    One more question- I am adding a conf file (just like HADOOP_SITE.xml) via
    -conf option and I am able to query parameters in mapper/reducers. But is
    there a way I can query the parameters in my job driver class -

    public class jobDriver extends Configured
    {
    someMethod( )
    {
    ToolRunner.run( new MyJob( ), commandLineArgs);
    // I want to query parameters present in my conf file here
    }
    }

    public class MyJob extends Configured implements Tool
    {
    }

    Thanks,
    Taran
    On Mon, Oct 6, 2008 at 2:46 PM, Mahadev Konar wrote:

    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev

    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes in the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar

    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran
  • Amareshwari Sriramadasu at Oct 7, 2008 at 3:49 am
    Hi,

    From 0.19, the jars added using -libjars are available on the client
    classpath also, fixed by HADOOP-3570.

    Thanks
    Amareshwari

    Mahadev Konar wrote:
    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev


    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes in the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar


    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran
  • Kyle Lau at Oct 17, 2008 at 5:26 pm
    That's right. I have success with the 0.18 (from EC2) and 0.18.1 (my
    local installation) as well.

    Kyle

    On Tue, 2008-10-07 at 09:13 +0530, Amareshwari Sriramadasu wrote:
    Hi,

    From 0.19, the jars added using -libjars are available on the client
    classpath also, fixed by HADOOP-3570.

    Thanks
    Amareshwari

    Mahadev Konar wrote:
    HI Tarandeep,
    the libjars options does not add the jar on the client side. Their is an
    open jira for that ( id ont remember which one)...

    Oyu have to add the jar to the

    HADOOP_CLASSPATH on the client side so that it gets picked up on the client
    side as well.


    mahadev


    On 10/6/08 2:30 PM, "Tarandeep Singh" wrote:

    Hi,

    I want to add a jar file (that is required by mappers and reducers) to the
    classpath. Initially I had copied the jar file to all the slave nodes in the
    $HADOOP_HOME/lib directory and it was working fine.

    However when I tried the libjars option to add jar files -

    $HADOOP_HOME/bin/hadoop jar myApp.jar -conf $MY_CONF_FILE -libjars jdom.jar


    I got this error-

    java.lang.NoClassDefFoundError: org/jdom/input/SAXBuilder

    Can someone please tell me what needs to be fixed here ?

    Thanks,
    Taran

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 6, '08 at 9:31p
activeOct 17, '08 at 5:26p
posts7
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase