FAQ
Hi all,

we are using hadoop 0.20.2 with hbase 0.20.6 on about 30 nodes. Our
cluster is under heavy write load, causing hbase to do a lot of
compactions, which in turn causes many files with many new blocks to be
created in HDFS. Now, the problem is, that several datanodes are
periodically running out of disk space (they consume the space very
rapidly, we have seen 5 TiB to be exhausted in single day). We have
investigated a bit this problem and created a patch in
FSNamesystem.invalidateWorkForOneNode(). The patch changes selection of
node to invalidate - in original version the first node is selected, we
changed this to random node. After aplying the patch the problem seems
to disappear and all our DNs are having constant and balanced disk
usage. The question is, is our problem a general issue, or are we
missing something? What is the reason to take the first node to
invalidate, when this can potentially cause starvation of other nodes?

Thanx,
Jan

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-user @
categorieshadoop
postedFeb 11, '11 at 9:24a
activeFeb 11, '11 at 9:24a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Jan Lukavský: 1 post

People

Translate

site design / logo © 2022 Grokbase