FAQ
Thanks! Because of your information, I managed to find out that my crashes
have to do with the somewhat standard 'Too many open files'.
I added some 'close' and 'disconnect' to my InputStreams and
HttpURLConnections, but that only works up to a certain point.
The strange thing is that I do execute 'ulimit -n 32768', but I'm thinking
it somehow 'won't' stick.
The other alternative is that Hadoop is using 32768 file descriptors, but
that seems a bit over the top, as I'm using at most 200 threads (each of
which only sets up one HttpURLConnection).

Suggestions are welcome :-)
On Mon, Aug 3, 2009 at 3:58 PM, Jason Venner wrote:

That generally means that the process that is running the task, crashed.
The actual map/reduce task is run in a separate jvm by the task tracker,
and that JVM is exiting abnormally.
This used to happen to my jobs quite a bit when they were using a buggy
native library via jni.

If you are trying to use the colorspace transforms via the java imaging
APIs, it is not thread safe (at least through 1.6.10 under linux).

There may be additional information available in the per task logs.

2009/8/3 Mathias De Maré <mathias.demare@gmail.com>

I'm getting a rather cryptic error while running a Map job with
MultithreadedMapper (no idea if it has anything to do with the
MultithreadedMapper).
It only occurs sometimes, occurs at different times during the Map
(sometimes at the start, sometimes at a random location), and it doesn't
really give any information.

Task Id : attempt_200908031207_0009_m_000000_0, Status : FAILED
java.io.IOException: Task process exit with nonzero status of 255.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)

I'm a bit at a loss. The only results I'm finding from Google are a few
other people who have had the same problem, but nobody with a solution.

Mathias De Maré


--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 3 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 3, '09 at 1:39p
activeAug 3, '09 at 5:47p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Mathias De Maré: 2 posts Jason Venner: 1 post

People

Translate

site design / logo © 2022 Grokbase