FAQ
Hello,

I am running compute intensive job using Hadoop Streaming (hadoop
version 0.19.1), and my mapper input has several thousand small files.
My system has 4 nodes and 8 cores per node.

I want to run 8 mappers per node to use all 8 cores, but whatever the
mapred.map.tasks value is, I can see only two mappers per node (when I
check it using ps -ef|grep mapper_name).

Any suggestions?

Thank you very much,

-seunghwa

Search Discussions

  • Todd Lipcon at May 6, 2009 at 8:24 pm

    On Wed, May 6, 2009 at 1:10 PM, Seunghwa Kang wrote:

    Hello,

    I am running compute intensive job using Hadoop Streaming (hadoop
    version 0.19.1), and my mapper input has several thousand small files.
    My system has 4 nodes and 8 cores per node.

    I want to run 8 mappers per node to use all 8 cores, but whatever the
    mapred.map.tasks value is, I can see only two mappers per node (when I
    check it using ps -ef|grep mapper_name).

    Any suggestions?
    Set mapred.tasktracker.map.tasks.maximum and
    mapred.tasktracker.reduce.tasks.maximum to your desired values

    -Todd

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 6, '09 at 8:11p
activeMay 6, '09 at 8:24p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Todd Lipcon: 1 post Seunghwa Kang: 1 post

People

Translate

site design / logo © 2022 Grokbase