Grokbase › Groups › Hadoop › common-user › August 2011
FAQ
Hi all,
I've tried to make a rack topology script. I've written it in python
and it works if I call it with the following arguments:
10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
10.2.0.33

The output is:
/rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
Should the output be on newlines or is any whitespace sufficient?

Additionally, my cluster's datanodes have DNS names such as:
r1dn02
r2dn05
etc...

I restarted the namenode in my running cluster (after configuring the
topology script setting in core-site.xml).
I ran a job and checked what the job tracker thinks the rack id's are
and it showed default-rack.
Can anyone tell me what I'm doing wrong?

Thanks,
tim

Search Discussions

  • Joey Echeverria at Aug 19, 2011 at 4:54 pm ⇧
    Did you restart the JobTracker?

    -Joey
    On Fri, Aug 19, 2011 at 12:45 PM, modemide wrote:
    Hi all,
    I've tried to make a rack topology script.  I've written it in python
    and it works if I call it with the following arguments:
    10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26  10.2.0.31
    10.2.0.33

    The output is:
    /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
    Should the output be on newlines or is any whitespace sufficient?

    Additionally, my cluster's datanodes have DNS names such as:
    r1dn02
    r2dn05
    etc...

    I restarted the namenode in my running cluster (after configuring the
    topology script setting in core-site.xml).
    I ran a job and checked what the job tracker thinks the rack id's are
    and it showed default-rack.
    Can anyone tell me what I'm doing wrong?

    Thanks,
    tim


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434
  • Modemide at Aug 19, 2011 at 5:16 pm ⇧
    Ha, what a silly mistake.

    Thank you Joey.

    Do you also happen to know of an easier way to tell which racks the
    jobtracker/namenode think each node is in?


    On 8/19/11, Joey Echeverria wrote:
    Did you restart the JobTracker?

    -Joey
    On Fri, Aug 19, 2011 at 12:45 PM, modemide wrote:
    Hi all,
    I've tried to make a rack topology script.  I've written it in python
    and it works if I call it with the following arguments:
    10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26  10.2.0.31
    10.2.0.33

    The output is:
    /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
    Should the output be on newlines or is any whitespace sufficient?

    Additionally, my cluster's datanodes have DNS names such as:
    r1dn02
    r2dn05
    etc...

    I restarted the namenode in my running cluster (after configuring the
    topology script setting in core-site.xml).
    I ran a job and checked what the job tracker thinks the rack id's are
    and it showed default-rack.
    Can anyone tell me what I'm doing wrong?

    Thanks,
    tim


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434
  • Joey Echeverria at Aug 21, 2011 at 2:22 pm ⇧
    Not that I know of.

    -Joey
    On Fri, Aug 19, 2011 at 1:16 PM, modemide wrote:
    Ha, what a silly mistake.

    Thank you Joey.

    Do you also happen to know of an easier way to tell which racks the
    jobtracker/namenode think each node is in?


    On 8/19/11, Joey Echeverria wrote:
    Did you restart the JobTracker?

    -Joey
    On Fri, Aug 19, 2011 at 12:45 PM, modemide wrote:
    Hi all,
    I've tried to make a rack topology script.  I've written it in python
    and it works if I call it with the following arguments:
    10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26  10.2.0.31
    10.2.0.33

    The output is:
    /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
    Should the output be on newlines or is any whitespace sufficient?

    Additionally, my cluster's datanodes have DNS names such as:
    r1dn02
    r2dn05
    etc...

    I restarted the namenode in my running cluster (after configuring the
    topology script setting in core-site.xml).
    I ran a job and checked what the job tracker thinks the rack id's are
    and it showed default-rack.
    Can anyone tell me what I'm doing wrong?

    Thanks,
    tim


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434
  • Edward Capriolo at Aug 21, 2011 at 11:21 pm ⇧

    On Sun, Aug 21, 2011 at 10:22 AM, Joey Echeverria wrote:

    Not that I know of.

    -Joey
    On Fri, Aug 19, 2011 at 1:16 PM, modemide wrote:
    Ha, what a silly mistake.

    Thank you Joey.

    Do you also happen to know of an easier way to tell which racks the
    jobtracker/namenode think each node is in?


    On 8/19/11, Joey Echeverria wrote:
    Did you restart the JobTracker?

    -Joey
    On Fri, Aug 19, 2011 at 12:45 PM, modemide wrote:
    Hi all,
    I've tried to make a rack topology script. I've written it in python
    and it works if I call it with the following arguments:
    10.2.0.1 10.2.0.11 10.2.0.11 10.2.0.12 10.2.0.21 10.2.0.26 10.2.0.31
    10.2.0.33

    The output is:
    /rack0 /rack1 /rack1 /rack1 /rack2 /rack2 /rack3 /rack3
    Should the output be on newlines or is any whitespace sufficient?

    Additionally, my cluster's datanodes have DNS names such as:
    r1dn02
    r2dn05
    etc...

    I restarted the namenode in my running cluster (after configuring the
    topology script setting in core-site.xml).
    I ran a job and checked what the job tracker thinks the rack id's are
    and it showed default-rack.
    Can anyone tell me what I'm doing wrong?

    Thanks,
    tim


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434


    --
    Joseph Echeverria
    Cloudera, Inc.
    443.305.9434
    If you run the hdfs balancer application the application displays the
    topology it learns from from the topology script. Assuming your jobtracker
    started with the same configuration you have your answer.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 19, '11 at 4:46p
activeAug 21, '11 at 11:21p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Joey Echeverria: 2 posts Modemide: 2 posts Edward Capriolo: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase