I'm running into an inconsistent problem where with multiple reducers (and extremely small amounts of data) I'll hit a situation where a Reducer task attempt will hang at 33% until the JobTracker starts up a 2nd attempt. It does not seem to be tied to any 1 machine, and when I change the number of reducers to 1, it runs fine every time.

log4j:ERROR Failed to flush writer,


at java.io.FileOutputStream.writeBytes(Native Method)

at java.io.FileOutputStream.write(FileOutputStream.java:260)

at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)

at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)

at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)

at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)

at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)

at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:58)

at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:316)

at org.apache.log4j.WriterAppender.append(WriterAppender.java:160)

at org.apache.hadoop.mapred.TaskLogAppender.append(TaskLogAppender.java:58)

at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)

at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)

at org.apache.log4j.Category.callAppenders(Category.java:206)

at org.apache.log4j.Category.forcedLog(Category.java:391)

at org.apache.log4j.Category.log(Category.java:856)

at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:199)

at org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler.freeHost(ShuffleScheduler.java:345)

at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:152)

Aaron Baff | Developer | Telescope, Inc.

email: aaron.baff@telescope.tv | office: 424 270 2913 | www.telescope.tv<http://www.telescope.tv>

<http://twitter.com/Telescope_Inc>AMERICAN IDOL is back and better than ever with a new team of judges on Season 10! Voting begins Tuesday, March 1st so be sure to tune in and vote to make sure your favorite contestant makes it through each week (check local listings for dates & times). More info at www.americanidol.com<http://www.americanidol.com/>.

The information contained in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Any views expressed in this message are those of the individual and may not necessarily reflect the views of Telescope Inc. or its associated companies.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
postedFeb 25, '11 at 8:39p
activeFeb 25, '11 at 8:39p

1 user in discussion

Aaron Baff: 1 post



site design / logo © 2022 Grokbase