FAQ
Hi,

I am currently trying to tune a CDH 4.0.1 cluster running HDFS, YARN,
and HBase managed by Cloudera Manager 4.0.3 (Free Edition).

In CM, there are a number of options for setting mapreduce.*
configuration properties on the YARN client page.

Some of the explanations in the GUI still refer to JobTracker and
TaskTracker, e.g.,
- mapreduce.jobtracker.handler.count,
- mapreduce.tasktracker.map.tasks.maximum,
- mapreduce.tasktracker.reduce.tasks.maximum

I wonder whether these and a number of other mapreduce.* (e.g.,
mapreduce.job.reduces) properties are observed by the MR2
ApplicationMaster or not.

Can anyone clarify or point to respective documentation?

Thanks,
Martin

Search Discussions

  • Mg at Aug 16, 2012 at 5:37 pm
    Hi,

    thanks.

    But there are many more mapreduce.* properties. ...

    Does anyone have more information on those with respect to whether they
    are relevant to YARN, and if so, what they exactly mean in the YARN context?

    I am trying to optimize settings in order to use as much RAM and CPU as
    possible. The cluster has 8 node managers each with 96GB RAM and 16 CPUs
    (cpuinfo: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz)

    Cheers,
    Martin
    On 16.08.2012 18:40, anil gupta wrote:
    Hi,

    AFAIK, these properties are being ignored by YARN:
    - mapreduce.tasktracker.map.tasks.maximum,
    - mapreduce.tasktracker.reduce.tasks.maximum

    Thanks,
    Anil Gupta


    On Thu, Aug 16, 2012 at 9:28 AM, mg wrote:

    Hi,

    I am currently trying to tune a CDH 4.0.1 (i~ hadoop 2.0.0-alpha)
    cluster running HDFS, YARN, and HBase managed by Cloudera Manager
    4.0.3 (Free Edition).


    In CM, there are a number of options for setting mapreduce.*
    configuration properties on the YARN client page.

    Some of the properties and their explanations in the GUI still refer
    to JobTracker and TaskTracker, e.g.,

    - mapreduce.jobtracker.handler.__count,
    - mapreduce.tasktracker.map.__tasks.maximum,
    - mapreduce.tasktracker.reduce.__tasks.maximum

    I wonder whether these and a number of other mapreduce.* (e.g.,
    mapreduce.job.reduces) properties are observed by the MR2
    ApplicationMaster (and how), or not.


    Can anyone clarify or point to respective documentation?

    Thanks,
    Martin




    --
    Thanks & Regards,
    Anil Gupta

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedAug 16, '12 at 4:22p
activeAug 16, '12 at 5:37p
posts2
users1
websitecloudera.com
irc#hadoop

1 user in discussion

Mg: 2 posts

People

Translate

site design / logo © 2022 Grokbase