FAQ
Hello, list,

I've seen this question before, but haven't found an answer. If I run
a Hadoop job on a cluster (EC2), how can I download the stdout/stderr
logs from each mapper/reducer task? Are they stored somewhere in
HDFS, or just on the machine that ran the task?

-Stuart

Search Discussions

  • Arun C Murthy at Aug 12, 2008 at 11:53 pm

    On Aug 12, 2008, at 11:52 AM, Stuart Sierra wrote:

    Hello, list,

    I've seen this question before, but haven't found an answer. If I run
    a Hadoop job on a cluster (EC2), how can I download the stdout/stderr
    logs from each mapper/reducer task? Are they stored somewhere in
    HDFS, or just on the machine that ran the task?
    They are usually just present on the nodes, however if you running
    Hadoop on Demand (HoD - http://hadoop.apache.org/core/docs/current/hod.html)
    you get that feature with it.

    OTOH - you might be interested in Chukwa (http://issues.apache.org/jira/browse/HADOOP-3719
    ) - it's coming in hadoop-0.19 though, which is a while away.

    Arun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 12, '08 at 6:53p
activeAug 12, '08 at 11:53p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Arun C Murthy: 1 post Stuart Sierra: 1 post

People

Translate

site design / logo © 2022 Grokbase