FAQ

On Wed, Aug 12, 2009 at 1:15 PM, Mayuran Yogarajah wrote:

I had 3 jobs running and I saw something a bit odd. Two of the tasks
are reducing, one of them is using all the reducers so the other is
waiting, this is OK. However the 3rd job is still in the mapping phase
and even though the web interface shows map capacity at 96 I only
see about 7-12 mappers actually running. I'm wondering if theres
some setting I need to change, perhaps I've hit some system limit. Can
someone point me in the right direction please?
Are there any pending mappers remaining? Are you using any scheduler?

The other thing was that with the two jobs that are in the reducing phase,
the reducer for one job wouldn't actually start until all the mappers of
the
_other_ job completed which seems kind of odd. Is this expected ?
Reducers dont really start the "reduce" phase till the mappers are
completed. However, the process gets spawned off and the copying of the
intermediate keys from the mapper starts off.

thanks,
M

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 9 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 12, '09 at 8:15p
activeAug 12, '09 at 10:24p
posts9
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase