[Hadoop-common-user] runtime exceptions not killing job

On Mon, Mar 17, 2008 at 6:14 PM, Matt Kent wrote:
I recently upgraded from Hadoop 0.14.1 to 0.16.1. Previously in 0.14.1,
if a map or reduce task threw a runtime exception such as an NPE, the
task, and ultimately the job, would fail in short order. I was running
on job on my local 0.16.1 cluster today, and when the reduce tasks
started throwing NPEs, the tasks just hung. Eventually they timed out
and were killed, but is this expected behavior in 0.16.1? I'd prefer the
job to fail quickly if NPEs are being thrown.

Matt

--
Matt Kent
Co-Founder
Persai
1221 40th St #113
Emeryville, CA 94608
matt@persai.com

On Mar 17, 2008, at 8:17 PM, Konstantin Shvachko wrote:

Usually a build takes 2 hours or less.
This one is stuck and I don't see changes in the QUEUE OF PENDING
PATCHES when I submit a patch.
I guess something is wrong with Hadson.
Could anybody please check.
--Konstantin

On Mar 17, 2008, at 3:14 PM, Matt Kent wrote:

I recently upgraded from Hadoop 0.14.1 to 0.16.1. Previously in
0.14.1, if a map or reduce task threw a runtime exception such as
an NPE, the task, and ultimately the job, would fail in short
order. I was running on job on my local 0.16.1 cluster today, and
when the reduce tasks started throwing NPEs, the tasks just hung.
Eventually they timed out and were killed, but is this expected
behavior in 0.16.1? I'd prefer the job to fail quickly if NPEs are
being thrown.

This sounds like a bug. Tasks should certainly fail immediately if an
exception is thrown. Do you know where the exception is being thrown?
Can you get a stack trace of the task from jstack after the exception
and before the task times out?

Thanks,
Owen