FAQ
I'm running a mapreduce using Hbase table as input with some distributed
cache file and all works well.

However, when I set:

c.set("mapred.child.java.opts", "-Xmx512m") in the java code and
using the exact same input and exact same distributed cache I'm getting
the following:



on the master side:



09/08/11 08:19:05 INFO mapred.JobClient: Task Id :
attempt_200908110722_0016_m_000001_0, Status : FAILED

java.io.IOException: Task process exit with nonzero status of 134.

at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)



09/08/11 08:19:05 WARN mapred.JobClient: Error reading task
outputhttp://domU-12-31-39-02-95-D2.compute-1.internal:50060/tasklog?pla
intext=true&taskid=attempt_200908110722_0016_m_000001_0&filter=stdout

09/08/11 08:19:05 WARN mapred.JobClient: Error reading task
outputhttp://domU-12-31-39-02-95-D2.compute-1.internal:50060/tasklog?pla
intext=true&taskid=attempt_200908110722_0016_m_000001_0&filter=stderr









On the slave log I'm getting as if I do not have enough space. But I run
these on EC2 with 4 GB or memory so that's insane.



Error occurred during initialization of VM

Could not reserve enough space for object heap

#

# An unexpected error has been detected by Java Runtime Environment:

#

# SIGSEGV (0xb) at pc=0xf7ec40da, pid=4948, tid=4158831504

#

# Java VM: Java HotSpot(TM) Server VM (10.0-b23 mixed mode linux-x86)

# Problematic frame:

# C [libc.so.6+0x6d0da] cfree+0x7a

#

# An error report file with more information is saved as:

#
/mnt/hadoop/mapred/local/taskTracker/jobcache/job_200908110722_0016/atte
mpt_200908110722_0016_m_000001_0/work/hs_err_pid4948.log

#

# If you would like to submit a bug report, please visit:

# http://java.sun.com/webapps/bugreport/crash.jsp

#





Furthermore, I have checked the job_XXXXXXXXX_YYYY_conf.xml and the only
difference between the two runs (successful and non-successful ) is the
size

<
<property><name>mapred.child.java.opts</name><value>-Xmx512m</value></pr
operty>

>
<property><name>mapred.child.java.opts</name><value>-Xmx200m</value></pr
operty>





Any idea regarding this proble is welcome



Thanks

-Yair

Search Discussions

  • Yair Even-Zohar at Aug 11, 2009 at 3:05 pm
    Sorry to bug you guys again but I found the problem.
    An old hadoop-site that was in the class path and had limit on the
    "mapred.child.ulimit" to 500000

    Thanks
    -Yair

    -----Original Message-----
    From: Yair Even-Zohar
    Sent: Tuesday, August 11, 2009 4:11 PM
    To: common-user@hadoop.apache.org; hbase-user@hadoop.apache.org
    Subject: problem setting mapred.child.java.opts

    I'm running a mapreduce using Hbase table as input with some distributed
    cache file and all works well.

    However, when I set:

    c.set("mapred.child.java.opts", "-Xmx512m") in the java code and
    using the exact same input and exact same distributed cache I'm getting
    the following:



    on the master side:



    09/08/11 08:19:05 INFO mapred.JobClient: Task Id :
    attempt_200908110722_0016_m_000001_0, Status : FAILED

    java.io.IOException: Task process exit with nonzero status of 134.

    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)



    09/08/11 08:19:05 WARN mapred.JobClient: Error reading task
    outputhttp://domU-12-31-39-02-95-D2.compute-1.internal:50060/tasklog?pla
    intext=true&taskid=attempt_200908110722_0016_m_000001_0&filter=stdout

    09/08/11 08:19:05 WARN mapred.JobClient: Error reading task
    outputhttp://domU-12-31-39-02-95-D2.compute-1.internal:50060/tasklog?pla
    intext=true&taskid=attempt_200908110722_0016_m_000001_0&filter=stderr









    On the slave log I'm getting as if I do not have enough space. But I run
    these on EC2 with 4 GB or memory so that's insane.



    Error occurred during initialization of VM

    Could not reserve enough space for object heap

    #

    # An unexpected error has been detected by Java Runtime Environment:

    #

    # SIGSEGV (0xb) at pc=0xf7ec40da, pid=4948, tid=4158831504

    #

    # Java VM: Java HotSpot(TM) Server VM (10.0-b23 mixed mode linux-x86)

    # Problematic frame:

    # C [libc.so.6+0x6d0da] cfree+0x7a

    #

    # An error report file with more information is saved as:

    #
    /mnt/hadoop/mapred/local/taskTracker/jobcache/job_200908110722_0016/atte
    mpt_200908110722_0016_m_000001_0/work/hs_err_pid4948.log

    #

    # If you would like to submit a bug report, please visit:

    # http://java.sun.com/webapps/bugreport/crash.jsp

    #





    Furthermore, I have checked the job_XXXXXXXXX_YYYY_conf.xml and the only
    difference between the two runs (successful and non-successful ) is the
    size

    <
    <property><name>mapred.child.java.opts</name><value>-Xmx512m</value></pr
    operty>

    >
    <property><name>mapred.child.java.opts</name><value>-Xmx200m</value></pr
    operty>





    Any idea regarding this proble is welcome



    Thanks

    -Yair

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 11, '09 at 3:05p
activeAug 11, '09 at 3:05p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Yair Even-Zohar: 2 posts

People

Translate

site design / logo © 2022 Grokbase