FAQ

On Tue, Aug 11, 2009 at 4:45 AM, Mayuran Yogarajah wrote:

Hello,

If you are interested, you could try to trace one of these block ids in
NameNode log to see what happened it. We are always eager to hear about
irrecoverable errors. Please mention hadoop version you are using.

I'm using Hadoop 0.18.3. I just checked namenode log for one of the bad
blocks. I see entries from Saturday saying:
ask 1.1.1.6:50010 to replicate blk_1697509332927954816_8724 to datanode(s)
< all other data nodes >

I only loaded this data Saturday, and the .6 data node became full at some
point.
When data is first loaded into the cluster, does the name node send the
data to as many nodes as
it can to satisfy the replication factor, or does it send it to one node
and ask that node send it to others?
A DN is instructed by the NN (this actually happens when the DN sends a
heartbeat to NN) to replicate its block to the first replication target DN.
The NN chooses replication targets based on some logic (2nd replica is
always placed on a node in a different rack; 3rd replica is placed on a DN
in the same rack as the first one).

--
Harish Mallipeddi
http://blog.poundbang.in

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 5 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 10, '09 at 10:08p
activeAug 11, '09 at 4:44p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase