Arun C Murthy
|| at Aug 12, 2008 at 11:53 pm
On Aug 12, 2008, at 11:52 AM, Stuart Sierra wrote:
I've seen this question before, but haven't found an answer. If I run
a Hadoop job on a cluster (EC2), how can I download the stdout/stderr
logs from each mapper/reducer task? Are they stored somewhere in
HDFS, or just on the machine that ran the task?
They are usually just present on the nodes, however if you running
Hadoop on Demand (HoD - http://hadoop.apache.org/core/docs/current/hod.html)
you get that feature with it.
OTOH - you might be interested in Chukwa (http://issues.apache.org/jira/browse/HADOOP-3719
) - it's coming in hadoop-0.19 though, which is a while away.