Grokbase › Groups › Hadoop › common-user › June 2009
FAQ
hello everyone,

I have added 7 nodes to my 3 node cluster. I followed the following steps to
do this

1. added the node's ip to conf/slaves at master
2. ran bin/start-balance.sh at each node

As i loaded the data when the size of the cluster was three which is now
TEN. Can i do anything to redistribute the data among all the nodes?

Any ideas appreciated.

Thanks and Regards

Asif.

Search Discussions

  • Alex Loddengaard at Jun 24, 2009 at 10:51 pm ⇧
    Hi,

    Running the rebalancer script (by the way, you only need to run it once)
    redistributes all of your data for you. That is, after you've run the
    rebalancer, your data should be stored evenly among your 10 nodes.

    Alex
    On Wed, Jun 24, 2009 at 2:50 PM, asif md wrote:

    hello everyone,

    I have added 7 nodes to my 3 node cluster. I followed the following steps
    to
    do this

    1. added the node's ip to conf/slaves at master
    2. ran bin/start-balance.sh at each node

    As i loaded the data when the size of the cluster was three which is now
    TEN. Can i do anything to redistribute the data among all the nodes?

    Any ideas appreciated.

    Thanks and Regards

    Asif.
  • Asif md at Jun 24, 2009 at 11:22 pm ⇧
    @Alex

    Thanks.

    http://wiki.apache.org/hadoop/FAQ#6

    has anyone any experience with this?

    Please suggest.
    On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard wrote:

    Hi,

    Running the rebalancer script (by the way, you only need to run it once)
    redistributes all of your data for you. That is, after you've run the
    rebalancer, your data should be stored evenly among your 10 nodes.

    Alex
    On Wed, Jun 24, 2009 at 2:50 PM, asif md wrote:

    hello everyone,

    I have added 7 nodes to my 3 node cluster. I followed the following steps
    to
    do this

    1. added the node's ip to conf/slaves at master
    2. ran bin/start-balance.sh at each node

    As i loaded the data when the size of the cluster was three which is now
    TEN. Can i do anything to redistribute the data among all the nodes?

    Any ideas appreciated.

    Thanks and Regards

    Asif.
  • Konstantin Shvachko at Jun 25, 2009 at 12:28 am ⇧
    These links should help you to rebalance the nodes:

    http://developer.yahoo.com/hadoop/tutorial/module2.html#rebalancing
    http://hadoop.apache.org/core/docs/current/hdfs_user_guide.html#Rebalancer
    http://hadoop.apache.org/core/docs/current/commands_manual.html#balancer
    http://issues.apache.org/jira/secure/attachment/12368261/RebalanceDesign6.pdf

    --Konstantin

    asif md wrote:
    @Alex

    Thanks.

    http://wiki.apache.org/hadoop/FAQ#6

    has anyone any experience with this?

    Please suggest.
    On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard wrote:

    Hi,

    Running the rebalancer script (by the way, you only need to run it once)
    redistributes all of your data for you. That is, after you've run the
    rebalancer, your data should be stored evenly among your 10 nodes.

    Alex
    On Wed, Jun 24, 2009 at 2:50 PM, asif md wrote:

    hello everyone,

    I have added 7 nodes to my 3 node cluster. I followed the following steps
    to
    do this

    1. added the node's ip to conf/slaves at master
    2. ran bin/start-balance.sh at each node

    As i loaded the data when the size of the cluster was three which is now
    TEN. Can i do anything to redistribute the data among all the nodes?

    Any ideas appreciated.

    Thanks and Regards

    Asif.
  • Asif md at Jun 25, 2009 at 12:37 am ⇧
    @Konstantin

    I'll try those . Thanks.

    More comments are welcome.


    On Wed, Jun 24, 2009 at 7:27 PM, Konstantin Shvachko wrote:

    These links should help you to rebalance the nodes:

    http://developer.yahoo.com/hadoop/tutorial/module2.html#rebalancing
    http://hadoop.apache.org/core/docs/current/hdfs_user_guide.html#Rebalancer
    http://hadoop.apache.org/core/docs/current/commands_manual.html#balancer

    http://issues.apache.org/jira/secure/attachment/12368261/RebalanceDesign6.pdf

    --Konstantin


    asif md wrote:
    @Alex

    Thanks.

    http://wiki.apache.org/hadoop/FAQ#6

    has anyone any experience with this?

    Please suggest.

    On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard <alex@cloudera.com>
    wrote:

    Hi,
    Running the rebalancer script (by the way, you only need to run it once)
    redistributes all of your data for you. That is, after you've run the
    rebalancer, your data should be stored evenly among your 10 nodes.

    Alex

    On Wed, Jun 24, 2009 at 2:50 PM, asif md wrote:

    hello everyone,
    I have added 7 nodes to my 3 node cluster. I followed the following
    steps
    to
    do this

    1. added the node's ip to conf/slaves at master
    2. ran bin/start-balance.sh at each node

    As i loaded the data when the size of the cluster was three which is now
    TEN. Can i do anything to redistribute the data among all the nodes?

    Any ideas appreciated.

    Thanks and Regards

    Asif.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 24, '09 at 9:50p
activeJun 25, '09 at 12:37a
posts5
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Asif md: 3 posts Alex Loddengaard: 1 post Konstantin Shvachko: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase