[Hadoop-common-user] Reason of Formatting Namenode

Formatting the NameNode initializes the FSNameSystem in the
dfs.name.dir directories, to prepare for use.

The format command typically writes a VERSION file that specifies what
the NamespaceID for this FS instance is, what was its ctime, and what
is the version (of the file's layout) in use.

This is helpful in making every NameNode instance unique, among other
things. DataNode blocks carry the namespace-id information that lets
them relate blocks to a NameNode (and thereby validate, etc.).

Thanks Harsh, i.e why if we again format namenode after loading some
data INCOMATIBLE NAMESPACE ID's error occurs.


Best Regards,

Adarsh Sharma




Harsh J wrote:

If you do not tell where you NN to store data it stores it to /tmp.
And your operating system cleans up temp.

The reason for the error you see is datanodes don't like to suddenly
connect to new namenodes. So as a safety they do not start up until
they are cleared.

On the first run you want namenode to initialize its directories (where it
store VERSION file, fsimage and edits).
On the subsequent formats - you are making sure you have a new EMPTY file
system. If you don't do format NameNode will load up fsimage and edits.
There is also matter of generating new space id, which is matched against
Datanode's ones. So if you format Namenode you need to cleanup data from
Datanodes.

On the other hand, if you just add Datanodes to a running cluster - you
don't have to format NN.

Boris.