FAQ

Mayuran Yogarajah wrote:
There are always a few 'Failed/Killed Task Attempts' and when I view the
logs for
these I see:

- some that are empty, ie stdout/stderr/syslog logs are all blank
- several that say:

2009-06-06 20:47:15,309 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
java.io.IOException: Filesystem closed
at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:195)
at org.apache.hadoop.dfs.DFSClient.access$600(DFSClient.java:59)
at
org.apache.hadoop.dfs.DFSClient$DFSInputStream.close(DFSClient.java:1359)
at java.io.FilterInputStream.close(FilterInputStream.java:159)
at
org.apache.hadoop.mapred.LineRecordReader$LineReader.close(LineRecordReader.java:103)

at
org.apache.hadoop.mapred.LineRecordReader.close(LineRecordReader.java:301)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.close(MapTask.java:173)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:231)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)



Any idea why this happens? I don't understand why I'd be seeing these
only as the mappers get to
100%.
Seen this when something in the same process got a FileSystem reference
by FileSystem.get() and then called close() on it -it closes the client
for every thread/class that has a reference to the same object.


We're planning on adding more diagnostics, by tracking who closed the
filesystem
https://issues.apache.org/jira/browse/HADOOP-5933

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 3 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 7, '09 at 12:55a
activeJun 8, '09 at 5:42p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase