Grokbase Groups Pig user May 2008
FAQ
Dear Tanton,

This means the MemoryManager is not successful at reclaiming memory. Did
that happen in Map phase or Reduce phase? Did you use a custom chunker? How
big is your split?

Pi
On Fri, May 23, 2008 at 3:52 PM, Tanton Gibbs wrote:

I upgraded to hadoop 17 and the latest Pig from svn.

I'm now getting a ton of lines in my log files that say:

2008-05-23 00:49:27,832 INFO
org.apache.pig.impl.util.SpillableMemoryManager: low memory handler
called init = 1441792(1408K) used = 483176072(471851K) committed =
641335296(626304K) max = 954466304(932096K)

In addition, jobs on big files are running very slowly.

Does anyone have any ideas as to what I could have screwed up?

Thanks!
Tanton

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 12 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 23, '08 at 5:52a
activeMay 30, '08 at 4:32p
posts12
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase