FAQ
Just to make sure my head is on straight: Each node in the grid reads its own configuration file (hadoop-site.xml, hadoop-default.xml) and configures itself appropriately, correct?

I am asking because I have nodes with 2 different disk configurations. For some I want a single directory for dfs.data.dir, but for others I want a comma-separated list of directories.

Also, is it true that ALL nodes should indicate the master node with mapred.job.tracker ?

Thanks,
C G



---------------------------------
Get easy, one-click access to your favorites. Make Yahoo! your homepage.

Search Discussions

  • Arun C Murthy at Nov 27, 2007 at 3:45 am

    C G wrote:
    Just to make sure my head is on straight: Each node in the grid reads its own configuration file (hadoop-site.xml, hadoop-default.xml) and configures itself appropriately, correct?

    I am asking because I have nodes with 2 different disk configurations. For some I want a single directory for dfs.data.dir, but for others I want a comma-separated list of directories.
    Yes. This is the intended behaviour.

    However, you should be aware of HADOOP-1245 (scheduled for
    hadoop-0.16.0) which fixes one important bug w.r.t configuring
    TaskTrackers independently of the JobTracker.
    Also, is it true that ALL nodes should indicate the master node with mapred.job.tracker ?
    mapred.job.tracker is the JobTracker, so yes.

    Of course you can have multiple map-reduce clusters with different
    JobTrackers.

    Arun
    Thanks,
    C G



    ---------------------------------
    Get easy, one-click access to your favorites. Make Yahoo! your homepage.
  • Ted Dunning at Nov 27, 2007 at 7:24 pm
    The standard trick here is to put all possible (conceivable) directories in
    the list. As the datanodes come up, they will ignore directories that don't
    exist.

    On 11/26/07 1:21 PM, "C G" wrote:

    Just to make sure my head is on straight: Each node in the grid reads its own
    configuration file (hadoop-site.xml, hadoop-default.xml) and configures itself
    appropriately, correct?

    I am asking because I have nodes with 2 different disk configurations. For
    some I want a single directory for dfs.data.dir, but for others I want a
    comma-separated list of directories.

    Also, is it true that ALL nodes should indicate the master node with
    mapred.job.tracker ?

    Thanks,
    C G



    ---------------------------------
    Get easy, one-click access to your favorites. Make Yahoo! your homepage.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedNov 26, '07 at 9:48p
activeNov 27, '07 at 7:24p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

C G: 1 post Ted Dunning: 1 post Arun C Murthy: 1 post

People

Translate

site design / logo © 2022 Grokbase