Grokbase › Groups › Hadoop › scm-users › July 2013
FAQ
Hi, I have a general question. How to configure the new datanodes and HDFS
services to make sure they work for the map-reduce job?

I found that the number of cluster nodes was only 1 even I already add
multiple datanodes in the same cluster and add the HDFS instances.

Do I need also add the task-tracker for each datanode?
Thanks a lot!

<https://lh6.googleusercontent.com/-8pLtmgtcc5g/Ueswvch9ZaI/AAAAAAAAAIM/z10xetLoRWI/s1600/ClusterNodes.PNG>

Search Discussions

  • Harsh J at Jul 21, 2013 at 4:20 am ⇧
    Yes, you need to add both a DataNode and TaskTracker service to any added
    slave node for MR to work over all HDFS nodes.

    On Sun, Jul 21, 2013 at 6:29 AM, GIS Song wrote:

    Hi, I have a general question. How to configure the new datanodes and
    HDFS services to make sure they work for the map-reduce job?

    I found that the number of cluster nodes was only 1 even I already add
    multiple datanodes in the same cluster and add the HDFS instances.

    Do I need also add the task-tracker for each datanode?
    Thanks a lot!


    <https://lh6.googleusercontent.com/-8pLtmgtcc5g/Ueswvch9ZaI/AAAAAAAAAIM/z10xetLoRWI/s1600/ClusterNodes.PNG>


    --
    Harsh J
  • GIS Song at Jul 21, 2013 at 5:28 am ⇧
    Thank you Harsh.
    Yes, I started both the DataNode service and the TaskTracker service in
    each slave node. But in the Jobtacker webUI page, I still only saw 1 node
    in the cluster as shown in previous picture.

    Is there any other settings? or they will automatically add nodes when
    running new MR tasks.

    Thanks,
    Song
    On Saturday, July 20, 2013 9:20:10 PM UTC-7, Harsh J wrote:

    Yes, you need to add both a DataNode and TaskTracker service to any added
    slave node for MR to work over all HDFS nodes.


    On Sun, Jul 21, 2013 at 6:29 AM, GIS Song <songg...@gmail.com<javascript:>
    wrote:
    Hi, I have a general question. How to configure the new datanodes and
    HDFS services to make sure they work for the map-reduce job?

    I found that the number of cluster nodes was only 1 even I already add
    multiple datanodes in the same cluster and add the HDFS instances.

    Do I need also add the task-tracker for each datanode?
    Thanks a lot!


    <https://lh6.googleusercontent.com/-8pLtmgtcc5g/Ueswvch9ZaI/AAAAAAAAAIM/z10xetLoRWI/s1600/ClusterNodes.PNG>


    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 21, '13 at 12:59a
activeJul 21, '13 at 5:28a
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

GIS Song: 2 posts Harsh J: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase