FAQ
Is there any sense in setting mapred.child.java.opts to a high value
if all we're using is Hadoop streaming ? We've set it to 512MB but
I don't know if it even matters..

thanks

Search Discussions

  • Jason Venner at Sep 15, 2009 at 2:35 am
    For streaming, you just need enough space to the run the child that forks
    and handles the streaming mapper/reducer and enough space to handle sorting
    the date with out the task getting stuck in GC hell.

    On Fri, Sep 11, 2009 at 12:30 PM, Mayuran Yogarajah wrote:

    Is there any sense in setting mapred.child.java.opts to a high value
    if all we're using is Hadoop streaming ? We've set it to 512MB but
    I don't know if it even matters..

    thanks


    --
    Pro Hadoop, a book to guide you from beginner to hadoop mastery,
    http://www.amazon.com/dp/1430219424?tag=jewlerymall
    www.prohadoopbook.com a community for Hadoop Professionals

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedSep 11, '09 at 7:30p
activeSep 15, '09 at 2:35a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Mayuran Yogarajah: 1 post Jason Venner: 1 post

People

Translate

site design / logo © 2021 Grokbase