FAQ
Hi,

I just setup 3 node hadoop cluster using the latest version from website ,
0.21.0

I am able to start all the daemons, when I run jps I see datanode, namenode,
secondary, tasktracker, but I was running a test and trying to run the
following command: ./hadoop dfs -ls, and I get the following error:

DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/fs/FsShell
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: org.apache.hadoop.fs.FsShell. Program will
exit.

If i try this command instead:

./hadoop hdfs -ls
Exception in thread "main" java.lang.NoClassDefFoundError: hdfs
Caused by: java.lang.ClassNotFoundException: hdfs
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: hdfs. Program will exit.

Does anyone know what the command really is I should be using?

Thanks

Search Discussions

  • Tom White at Sep 15, 2010 at 10:37 pm
    Hi Mike,

    What do you get if you type "./hadoop classpath"? Does it contain the
    Hadoop common JAR?

    To avoid the deprecation warning you should use "hadoop fs", not "hadoop dfs".

    Tom
    On Wed, Sep 15, 2010 at 12:53 PM, Mike Franon wrote:
    Hi,

    I just setup 3 node hadoop cluster using the latest version from website ,
    0.21.0

    I am able to start all the daemons, when I run jps I see datanode, namenode,
    secondary, tasktracker, but I was running a test and trying to run the
    following command: ./hadoop dfs -ls, and I get the following error:

    DEPRECATED: Use of this script to execute hdfs command is deprecated.
    Instead use the hdfs command for it.

    Exception in thread "main" java.lang.NoClassDefFoundError:
    org/apache/hadoop/fs/FsShell
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: org.apache.hadoop.fs.FsShell.  Program will
    exit.

    If i try this command instead:

    ./hadoop hdfs -ls
    Exception in thread "main" java.lang.NoClassDefFoundError: hdfs
    Caused by: java.lang.ClassNotFoundException: hdfs
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: hdfs.  Program will exit.

    Does anyone know what the command really is I should be using?

    Thanks
  • Mike Franon at Sep 16, 2010 at 1:33 pm
    Hi Tom,

    Great that worked, using ./hadoop -fs ls, and when I did classpath it showed
    commonjar.

    I had another question not sure why it keeps putting files in /tmp, when in
    my configs I have it set to something else. Most of the files begine with
    hadoop-hadoop*.pid, and some Jetty folders.

    Here are my configs

    <property>
    <name>hadoop.tmp.dir</name>
    <value>/usr/local/hadoop/hadoop-${user.name}</value>
    <description>A base for other temporary directories.</description>
    </property>


    <property>
    <name>fs.default.name</name>
    <value>hdfs://server1:9000/</value>
    </property>
    </configuration>


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.name.dir</name>
    <value>/usr/local/hadoop/hdfs/name</value>
    </property>
    <property>
    <name>dfs.data.dir</name>
    <value>/usr/local/hadoop/hdfs/data</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>3</value>
    </property>
    </configuration>

    Thanks,
    Mike
    On Wed, Sep 15, 2010 at 6:36 PM, Tom White wrote:

    Hi Mike,

    What do you get if you type "./hadoop classpath"? Does it contain the
    Hadoop common JAR?

    To avoid the deprecation warning you should use "hadoop fs", not "hadoop
    dfs".

    Tom
    On Wed, Sep 15, 2010 at 12:53 PM, Mike Franon wrote:
    Hi,

    I just setup 3 node hadoop cluster using the latest version from website ,
    0.21.0

    I am able to start all the daemons, when I run jps I see datanode, namenode,
    secondary, tasktracker, but I was running a test and trying to run the
    following command: ./hadoop dfs -ls, and I get the following error:

    DEPRECATED: Use of this script to execute hdfs command is deprecated.
    Instead use the hdfs command for it.

    Exception in thread "main" java.lang.NoClassDefFoundError:
    org/apache/hadoop/fs/FsShell
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: org.apache.hadoop.fs.FsShell. Program will
    exit.

    If i try this command instead:

    ./hadoop hdfs -ls
    Exception in thread "main" java.lang.NoClassDefFoundError: hdfs
    Caused by: java.lang.ClassNotFoundException: hdfs
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: hdfs. Program will exit.

    Does anyone know what the command really is I should be using?

    Thanks
  • Sharma, Avani at Sep 16, 2010 at 5:15 pm
    You need to set the PID dir separately using HADOOP_PID_DIR in conf/hadoop-env.sh

    -Avani
    -----Original Message-----
    From: Mike Franon
    Sent: Thursday, September 16, 2010 6:33 AM
    To: common-user@hadoop.apache.org
    Subject: Re: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FsShell

    Hi Tom,

    Great that worked, using ./hadoop -fs ls, and when I did classpath it showed
    commonjar.

    I had another question not sure why it keeps putting files in /tmp, when in
    my configs I have it set to something else. Most of the files begine with
    hadoop-hadoop*.pid, and some Jetty folders.

    Here are my configs

    <property>
    <name>hadoop.tmp.dir</name>
    <value>/usr/local/hadoop/hadoop-${user.name}</value>
    <description>A base for other temporary directories.</description>
    </property>


    <property>
    <name>fs.default.name</name>
    <value>hdfs://server1:9000/</value>
    </property>
    </configuration>


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.name.dir</name>
    <value>/usr/local/hadoop/hdfs/name</value>
    </property>
    <property>
    <name>dfs.data.dir</name>
    <value>/usr/local/hadoop/hdfs/data</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>3</value>
    </property>
    </configuration>

    Thanks,
    Mike
    On Wed, Sep 15, 2010 at 6:36 PM, Tom White wrote:

    Hi Mike,

    What do you get if you type "./hadoop classpath"? Does it contain the
    Hadoop common JAR?

    To avoid the deprecation warning you should use "hadoop fs", not "hadoop
    dfs".

    Tom
    On Wed, Sep 15, 2010 at 12:53 PM, Mike Franon wrote:
    Hi,

    I just setup 3 node hadoop cluster using the latest version from website ,
    0.21.0

    I am able to start all the daemons, when I run jps I see datanode, namenode,
    secondary, tasktracker, but I was running a test and trying to run the
    following command: ./hadoop dfs -ls, and I get the following error:

    DEPRECATED: Use of this script to execute hdfs command is deprecated.
    Instead use the hdfs command for it.

    Exception in thread "main" java.lang.NoClassDefFoundError:
    org/apache/hadoop/fs/FsShell
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FsShell
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: org.apache.hadoop.fs.FsShell. Program will
    exit.

    If i try this command instead:

    ./hadoop hdfs -ls
    Exception in thread "main" java.lang.NoClassDefFoundError: hdfs
    Caused by: java.lang.ClassNotFoundException: hdfs
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: hdfs. Program will exit.

    Does anyone know what the command really is I should be using?

    Thanks
  • Mike Franon at Sep 20, 2010 at 1:06 pm
    Great thanks for the help!
    On Thu, Sep 16, 2010 at 1:14 PM, Sharma, Avani wrote:

    You need to set the PID dir separately using HADOOP_PID_DIR in
    conf/hadoop-env.sh

    -Avani
    -----Original Message-----
    From: Mike Franon
    Sent: Thursday, September 16, 2010 6:33 AM
    To: common-user@hadoop.apache.org
    Subject: Re: Exception in thread "main" java.lang.NoClassDefFoundError:
    org/apache/hadoop/fs/FsShell

    Hi Tom,

    Great that worked, using ./hadoop -fs ls, and when I did classpath it
    showed
    commonjar.

    I had another question not sure why it keeps putting files in /tmp, when in
    my configs I have it set to something else. Most of the files begine with
    hadoop-hadoop*.pid, and some Jetty folders.

    Here are my configs

    <property>
    <name>hadoop.tmp.dir</name>
    <value>/usr/local/hadoop/hadoop-${user.name}</value>
    <description>A base for other temporary directories.</description>
    </property>


    <property>
    <name>fs.default.name</name>
    <value>hdfs://server1:9000/</value>
    </property>
    </configuration>


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.name.dir</name>
    <value>/usr/local/hadoop/hdfs/name</value>
    </property>
    <property>
    <name>dfs.data.dir</name>
    <value>/usr/local/hadoop/hdfs/data</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>3</value>
    </property>
    </configuration>

    Thanks,
    Mike
    On Wed, Sep 15, 2010 at 6:36 PM, Tom White wrote:

    Hi Mike,

    What do you get if you type "./hadoop classpath"? Does it contain the
    Hadoop common JAR?

    To avoid the deprecation warning you should use "hadoop fs", not "hadoop
    dfs".

    Tom

    On Wed, Sep 15, 2010 at 12:53 PM, Mike Franon <kongfranon@gmail.com>
    wrote:
    Hi,

    I just setup 3 node hadoop cluster using the latest version from
    website
    ,
    0.21.0

    I am able to start all the daemons, when I run jps I see datanode, namenode,
    secondary, tasktracker, but I was running a test and trying to run the
    following command: ./hadoop dfs -ls, and I get the following error:

    DEPRECATED: Use of this script to execute hdfs command is deprecated.
    Instead use the hdfs command for it.

    Exception in thread "main" java.lang.NoClassDefFoundError:
    org/apache/hadoop/fs/FsShell
    Caused by: java.lang.ClassNotFoundException:
    org.apache.hadoop.fs.FsShell
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: org.apache.hadoop.fs.FsShell. Program will
    exit.

    If i try this command instead:

    ./hadoop hdfs -ls
    Exception in thread "main" java.lang.NoClassDefFoundError: hdfs
    Caused by: java.lang.ClassNotFoundException: hdfs
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    Could not find the main class: hdfs. Program will exit.

    Does anyone know what the command really is I should be using?

    Thanks

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 15, '10 at 7:54p
activeSep 20, '10 at 1:06p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase