Grokbase › Groups › Hadoop › hdfs-user › December 2010
FAQ
Hi!

I have unbalanced disks on one of my nodes after a disk failure with
disk replacements.

http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
has a solution, but it starts with "Take down HDFS".

Is that really necessary - shouldn't taking down just that datanode,
moving around the blocks, then start the datanode be good enough, or
will that mess up some datastructure in the namenode?

Thanks,
\EF
--
Erik Forsberg <forsberg@opera.com>
Developer, Opera Software - http://www.opera.com/

Search Discussions

  • Travis Crawford at Dec 16, 2010 at 2:41 pm ⇧

    On Dec 16, 2010, at 6:34 AM, Erik Forsberg wrote:

    Hi!

    I have unbalanced disks on one of my nodes after a disk failure with
    disk replacements.

    http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
    has a solution, but it starts with "Take down HDFS".

    Is that really necessary - shouldn't taking down just that datanode,
    moving around the blocks, then start the datanode be good enough, or
    will that mess up some datastructure in the namenode?
    You're right - that should say take the DataNode down, move blocks
    around, then restart the DataNode. You might be able to just replace
    the disk and have them unbalanced for a while though. As data is
    created and deleted things will naturally even out.

    --Travis


    Thanks,
    \EF
    --
    Erik Forsberg <forsberg@opera.com>
    Developer, Opera Software - http://www.opera.com/
  • Allen Wittenauer at Dec 16, 2010 at 7:34 pm ⇧

    On Dec 16, 2010, at 6:32 AM, Erik Forsberg wrote:
    http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
    has a solution, but it starts with "Take down HDFS".

    Is that really necessary - shouldn't taking down just that datanode,
    moving around the blocks, then start the datanode be good enough, or
    will that mess up some datastructure in the namenode?
    It won't mess up the namenode, but it is going to be busy replicating those blocks on that datanode while you move stuff around. Depending upon how much data is involved, you might find that after you bring the datanode back up the namenode will put you back in a state of major unbalance when it sends a wave of deletions for the extra replicas.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-user @
categorieshadoop
postedDec 16, '10 at 2:34p
activeDec 16, '10 at 7:34p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Erik Forsberg: 1 post Travis Crawford: 1 post Allen Wittenauer: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase