This is technically a java issue but I thought other hadoop users would find
it interesting.

When we upgraded from old jvm to a newer (32bit) jvm of 1.6.0_21, we started
seeing users' tasks having issues at random places

1. Tasks running 2-3 times slower
2. Tasks failing with OutOfMemory
3. Reducer failing with OutOfMemory while pulling map outpus

Our first response was to increase the heapsize but what actually was
happening was that NewRatio has changed from 8 to 2 after we upgraded jvm to


So with -Xmx640m, older jvm (1.6.0_17 or before)

old generation went up to

PS Old Generation
capacity = 596574208 (568.9375MB)
used = 596574200 (568.9374923706055MB)
free = 8 (7.62939453125E-6MB)
99.99999865901007% used

but newer jvm(1.6.0_21) only went up to

PS Old Generation
capacity = 447414272 (426.6875MB)
used = 447414264 (426.68749237060547MB)
free = 8 (7.62939453125E-6MB)
99.99999821194797% used

causing all sorts of issues. To bring back the old behavior, we needed to
add -XX:NewRatio=8 to jvm param.

If you have MAPREDUCE-1207 in your version, you can set


Otherwise, you can set it on 'mapred.child.java.opts' and ask users to also
set it whenever they overwrite that conf.

Maybe this is a known issue to everyone but thought I would share it since
we spent so much time on this simple(?) issue :)

You can check the NewRatio value by running jmap -histo on the jvm.


Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
postedApr 8, '11 at 9:11p
activeApr 8, '11 at 9:11p

1 user in discussion

Koji Noguchi: 1 post



site design / logo © 2022 Grokbase