FAQ
In case of multiple folders from different disk drives are used for DFS on a data node, what is the best way to balance their disk usage?

As I understand, hadoop writes data to these folders in a round-robin fashion. Most time, it reaches a balance status. But if you add/remove disks, you want a way to rebalance multiple volumes on a single data node similar to cluster-wide rebalance.

Thanks,

Michael

Search Discussions

  • Allen Wittenauer at Sep 9, 2010 at 5:21 pm

    On Sep 9, 2010, at 10:13 AM, jiang licht wrote:

    In case of multiple folders from different disk drives are used for DFS on a data node, what is the best way to balance their disk usage?

    As I understand, hadoop writes data to these folders in a round-robin fashion. Most time, it reaches a balance status. But if you add/remove disks, you want a way to rebalance multiple volumes on a single data node similar to cluster-wide rebalance.

    Since this comes up pretty much every week, I've added it to the FAQ:

    http://wiki.apache.org/hadoop/FAQ#A31

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 9, '10 at 5:14p
activeSep 9, '10 at 5:21p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Allen Wittenauer: 1 post Jiang licht: 1 post

People

Translate

site design / logo © 2021 Grokbase