FAQ
I'm using the new Job class:

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html

There is a way to set the number of reduce tasks:

setNumReduceTasks(int tasks)

However, I don't see how to set the number of MAP tasks?

I tried to set it through mapred-site.xml :

<property>
<name>mapred.map.tasks</name>
<value>500</value>
</property>

It doesn't work either (launched map task is still small).

I'm wondering, do I have to rename the prefix from "mapred" to
"mapreduce"? like this (for all configurations?):

<property>
<name>mapreduce.map.tasks</name>
<value>500</value>
</property>

I added both, and it still doesn't work.

However, the old way of (using JobConf) works.

Is the new Job class is intended to not support altering the number of
map tasks?

FYI, setting the number of reduce tasks seemed to work using the new Job class.

Felix Halim

Search Discussions

  • Hemanth Yamijala at Jun 22, 2010 at 1:43 am
    Felix,
    I'm using the new Job class:

    http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html

    There is a way to set the number of reduce tasks:

    setNumReduceTasks(int tasks)

    However, I don't see how to set the number of MAP tasks?

    I tried to set it through mapred-site.xml :

    <property>
    <name>mapred.map.tasks</name>
    <value>500</value>
    </property>

    It doesn't work either (launched map task is still small).

    I'm wondering, do I have to rename the prefix from "mapred" to
    "mapreduce"? like this (for all configurations?):

    <property>
    <name>mapreduce.map.tasks</name>
    <value>500</value>
    </property>

    I added both, and it still doesn't work.
    As documented in the mapreduce tutorial as well as the Java
    documentation (http://bit.ly/9HKclu), the number of map tasks is
    primarily determined by the number of input splits generated for the
    input data.

    Thanks
    Hemanth

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 21, '10 at 7:08p
activeJun 22, '10 at 1:43a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Felix Halim: 1 post Hemanth Yamijala: 1 post

People

Translate

site design / logo © 2022 Grokbase