FAQ
When files are being loaded into HDFS, if the node has multiple entries for
dfs.data.dir, how does Hadoop pick which directory to store files in? Does
it intelligently pick the partition that has the most amount of space
available,
or is it round robin, or perhaps random?

We keep running into a problem where a DataNode keeps running out of space
because the data was being written to the partition with less space
available.

Here's some info about the cluster:
7 nodes, all identical hardware, running Hadoop 0.18.3.

Any feedback would be greatly appreciated.

thanks,
M

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 1 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 25, '10 at 6:01p
activeJun 25, '10 at 6:01p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Mayuran Yogarajah: 1 post

People

Translate

site design / logo © 2021 Grokbase