FAQ
Hi all,

When I invoke the balancer command, I found that it seems hdfs want to
balance the block number of each data node rather than the percentage of
each machine's remaining capacity.
Not sure if I am correct, could any help on this ?


Thank you

Jeff Zhang

Search Discussions

  • Brian Bockelman at Dec 18, 2009 at 12:37 pm
    Hey Jeff,

    I have not encountered this.

    http://dcache-head.unl.edu:8088/dfshealth.jsp

    The nodes that are 10x bigger have 10x more blocks on them. What makes you think this is true?

    Brian
    On Dec 18, 2009, at 4:29 AM, Jeff Zhang wrote:

    Hi all,

    When I invoke the balancer command, I found that it seems hdfs want to
    balance the block number of each data node rather than the percentage of
    each machine's remaining capacity.
    Not sure if I am correct, could any help on this ?


    Thank you

    Jeff Zhang
  • Jeff Zhang at Dec 18, 2009 at 1:06 pm
    It's my mistake, the balancer process is so slow that make me think it
    balance regarding the block

    Jeff Zhang
    On Fri, Dec 18, 2009 at 8:36 PM, Brian Bockelman wrote:

    Hey Jeff,

    I have not encountered this.

    http://dcache-head.unl.edu:8088/dfshealth.jsp

    The nodes that are 10x bigger have 10x more blocks on them. What makes you
    think this is true?

    Brian
    On Dec 18, 2009, at 4:29 AM, Jeff Zhang wrote:

    Hi all,

    When I invoke the balancer command, I found that it seems hdfs want to
    balance the block number of each data node rather than the percentage of
    each machine's remaining capacity.
    Not sure if I am correct, could any help on this ?


    Thank you

    Jeff Zhang
  • Todd Lipcon at Dec 18, 2009 at 5:04 pm
    Hi Jeff,

    You can tune dfs.datanode.balance.bandwidthPerSec in order to change the
    speed at which the balancer moves data around. It defaults to 1MB/sec so as
    to avoid using significant cluster resources, but you could certainly bump
    it up for an individual balancer run if you need it to go quicker.

    -Todd
    On Fri, Dec 18, 2009 at 5:05 AM, Jeff Zhang wrote:

    It's my mistake, the balancer process is so slow that make me think it
    balance regarding the block

    Jeff Zhang

    On Fri, Dec 18, 2009 at 8:36 PM, Brian Bockelman <[email protected]
    wrote:
    Hey Jeff,

    I have not encountered this.

    http://dcache-head.unl.edu:8088/dfshealth.jsp

    The nodes that are 10x bigger have 10x more blocks on them. What makes you
    think this is true?

    Brian
    On Dec 18, 2009, at 4:29 AM, Jeff Zhang wrote:

    Hi all,

    When I invoke the balancer command, I found that it seems hdfs want to
    balance the block number of each data node rather than the percentage
    of
    each machine's remaining capacity.
    Not sure if I am correct, could any help on this ?


    Thank you

    Jeff Zhang

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 18, '09 at 10:30a
activeDec 18, '09 at 5:04p
posts4
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2023 Grokbase