[Hadoop-common-user] Error in Cluster Startup: NameNode is not formatted

Boyu-

You didn't do anything stupid. I've forgotten to format a NameNode
too myself.

If you check the QuickStart guide at http://hadoop.apache.org/core/docs/current/quickstart.html
you'll see that formatting the NameNode is the first of the
Execution section (near the bottom of the page).

The command to format the NameNode is:

hadoop namenode -format

A warning though, you should only format your NameNode once. Just
like formatting any filesystem, you can loss data if you (re)format.

Good luck.

-Matt

Matt,

Thanks a lot for your reply! I did formatted the namenode. But I got the
same error again. And actually I successfully run the example jar file once,
but after that one time, I couldn't get it run again. I clean the /tmp dir
every time before I format namenode again(I am just testing it, so I don't
worry about losing data:). Still, I got the same error when I execute the
bin/start-dfs.sh . I checked my conf, and I can't figure out why. Here is my
conf file:

I really appreciate if you could take a look at it. Thanks a lot.


<configuration>

<property>
<name>fs.default.name</name>
<value>hdfs://hostname1:9000</value>
</property>


<property>
<name>mapred.job.tracker</name>
<value>hostname2:9001</value>
</property>



<property>
<name>dfs.data.dir</name>
<value>/data/zhang/hadoop/dfs/data</value>
<description>Determines where on the local filesystem an DFS data node
should store its blocks. If this is a comma-delimited
list of directories, then data will be stored in all named
directories, typically on different devices.
Directories that do not exist are ignored.
</description>
</property>


<property>
<name>mapred.local.dir</name>
<value>/data/zhang/hadoop/mapred/local</value>
<description>The local directory where MapReduce stores intermediate
data files. May be a comma-separated list of
directories on different devices in order to spread disk i/o.
Directories that do not exist are ignored.
</description>
</property>
</configuration>


-----Original Message-----
From: Matt Massie
Sent: Friday, June 26, 2009 4:31 PM
To: [email protected]
Subject: Re: Error in Cluster Startup: NameNode is not formatted

Boyu-

You didn't do anything stupid. I've forgotten to format a NameNode
too myself.

If you check the QuickStart guide at
http://hadoop.apache.org/core/docs/current/quickstart.html
you'll see that formatting the NameNode is the first of the
Execution section (near the bottom of the page).

The command to format the NameNode is:

hadoop namenode -format

A warning though, you should only format your NameNode once. Just
like formatting any filesystem, you can loss data if you (re)format.

Good luck.

-Matt

Sometimes the metadata gets corrupted. Its happened with me on multiple
occasions during the initial stages of setting up the cluster. What I did
was simply delete the entire directory where the metadata and the actual
data is being stored by hdfs. Since I was playing around with the systems
and didnt care much about the data, I could do so. If it doesnt spoil
anything for you, go ahead and try it. It might work.

Secondly, you've specified the dfs.data.dir parameter but havent specified
the metadata directory. AFAIK, it will take /tmp as the default. Since /tmp
gets cleaned up, you'll lose your metadata and that could be causing the
system to not come up. Specify that parameter in the config file.

Amandeep


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz

The property "dfs.name.dir" allows you to control where Hadoop writes
NameNode metadata.

You should have a property like

<property>
<name>dfs.name.dir</name>
<value>/data/zhang/hadoop/name/data</value>
</property>

to make sure the NameNode data isn't being deleted when you delete the
files in /tmp.

-Matt