FAQ

On Feb 27, 2008, at 7:29 PM, Zhang, Guibin wrote:

Question: How can I configure the hadoop to distribute less
tasks to weak nodes and distribute more tasks to strong nodes?
mapred.tasktracker.map.tasks.maximum is the number of tasks to run on
that task track simultaneously. 75 is almost certainly too high.
mapred.map.tasks is only relevant on the submitting node, because
that is where the planning takes place. More reasonable values
(depending on your hardware) are:

strong:
mapred.tasktracker.map.tasks.maximum = 8
mapred.tasktracker.reduce.tasks.maximum = 4

weak:
mapred.tasktracker.map.tasks.maximum = 4
mapred.tasktracker.reduce.tasks.maximum = 2

client:
mapred.map.tasks = nodes * avgerage
(mapred.tasktracker.map.tasks.maximum)
mapred.reduce.tasks = 95% * nodes * average
(mapred.tasktracker.reduce.tasks.maximum)

-- Owen

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 3 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedFeb 28, '08 at 3:31a
activeFeb 28, '08 at 3:35p
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase