FAQ
Changing the ordering of dfs.data.dir won't change anything, because
dfs.data.dir is written to in a round-robin fashion.

Kris, I think you're stuck with the hack you're performing :(. Sorry I
don't have better news.

Alex
On Tue, Aug 25, 2009 at 1:16 PM, Ted Dunning wrote:

Change the ordering of the volumes in the ocnfig files.

On Tue, Aug 25, 2009 at 12:51 PM, Kris Jirapinyo <kjirapinyo@biz360.com
wrote:
Hi all,
I know this has been filed as a JIRA improvement already
http://issues.apache.org/jira/browse/HDFS-343, but is there any good
workaround at the moment? What's happening is I have added a few new EBS
volumes to half of the cluster, but Hadoop doesn't want to write to them.
When I try to do cluster rebalancing, since the new disks make the
percentage used lower, it fills up the first two existing local disks,
which
is exactly what I don't want to happen. Currently, I just delete several
subdirs from dfs, since I know that with a replication factor of 3, it'll
be
ok, so that fixes the problems in the short term. But I still cannot get
Hadoop to use those new larger disks efficiently. Any thoughts?

-- Kris.


--
Ted Dunning, CTO
DeepDyve

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 11 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 25, '09 at 7:51p
activeAug 26, '09 at 5:40p
posts11
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase