FAQ
I'm seeing this error when a job runs:

Shuffling 35338524 bytes (35338524 raw bytes) into RAM from attempt_201001051549_0036_m_000003_0
Map output copy failure: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1198)


I originally had mapred.child.java.opts set to 200M. If I boost this up
to 512M the error goes away.
I'm trying to understand whats going on though. Can anyone explain?
Also are there any other parameters that
I should be tweaking to help with this?

thank you very much,
M

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 3 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedJan 7, '10 at 9:17p
activeJan 8, '10 at 6:59p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase