FAQ
Hi,
Is there any way in which we can measure the resource usage of each
map/reduce task running?
I was trying to use sar utility to track each process resource usage,
however it seems these individual map/reduce tasks are not listed as
processes when I do ps -ex.

Thanks,
Bikash

Search Discussions

  • Bikash sharma at Mar 1, 2011 at 1:59 pm
    Hi,
    As a follow-up question, do map/reduce tasks run as threads or processes?
    On Tue, Feb 22, 2011 at 10:35 AM, bikash sharma wrote:

    Hi,
    Is there any way in which we can measure the resource usage of each
    map/reduce task running?
    I was trying to use sar utility to track each process resource usage,
    however it seems these individual map/reduce tasks are not listed as
    processes when I do ps -ex.

    Thanks,
    Bikash
  • Harsh J at Mar 1, 2011 at 5:11 pm
    Hello,
    On Tue, Mar 1, 2011 at 7:29 PM, bikash sharma wrote:
    Hi,
    As a follow-up question, do map/reduce tasks run as threads or processes?
    Every launched Task runs as an independent process, communicating over
    a network interface (lo) with the TaskTracker for reporting/etc.
    purposes.

    --
    Harsh J
    www.harshj.com
  • Christopher Smith at Mar 1, 2011 at 5:45 pm
    Actually, that is configurable. Multiple tasks can be run from the
    same JVM, and you can also use the MultithreadedMapper
    (http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/map/MultithreadedMapper.html)
    to at least make map tasks run in the same thread.
    On Tue, Mar 1, 2011 at 9:09 AM, Harsh J wrote:
    Hello,
    On Tue, Mar 1, 2011 at 7:29 PM, bikash sharma wrote:
    Hi,
    As a follow-up question, do map/reduce tasks run as threads or processes?
    Every launched Task runs as an independent process, communicating over
    a network interface (lo) with the TaskTracker for reporting/etc.
    purposes.

    --
    Harsh J
    www.harshj.com


    --
    Chris

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedFeb 22, '11 at 3:36p
activeMar 1, '11 at 5:45p
posts4
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase