FAQ
I have found that HOD writes a series of log files to directories on the
virtual cluster master, if you specify log directories.
The interesting part is figuring out which machine was the virtual
cluster master, if you have a decent sized pool of machines.

It would be so nice if HOD could copy this back to the submission node.

I suppose we could configure syslog up but then we have to configure the
syslogd on each of the submit hosts to accept from any compute node...


--
Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested

Search Discussions

  • Hemanth Yamijala at Feb 28, 2008 at 4:34 am

    Jason Venner wrote:
    I have found that HOD writes a series of log files to directories on
    the virtual cluster master, if you specify log directories.
    The interesting part is figuring out which machine was the virtual
    cluster master, if you have a decent sized pool of machines.
    Can you explain what you mean by 'virtual cluster master' ? I guess you
    could mean the node which ran the 'ringmaster' process - the master
    process in a hod virtual cluster, or the hadoop 'jobtracker'. Both this
    information is stored by hod for an allocated cluster. You could
    retrieve it by using
    hod -o list,
    and
    hod -o "info cluster-dir"

    If the cluster is deallocated, you can get the ringmaster node still by
    using torque commands:

    qstat (to get torque job ids)
    and
    qstat -n <jobid>

    This will print a list of nodes, the first one is the ringmaster.
    It would be so nice if HOD could copy this back to the submission node.

    I suppose we could configure syslog up but then we have to configure
    the syslogd on each of the submit hosts to accept from any compute
    node...
    Syslog configuration has a bug which will be addressed in Hadoop 0.16.1.

    Thanks
    hemanth
  • Jason Venner at Feb 28, 2008 at 5:21 am
    You are my hero of the day Hemanth, these 3 things will make our life so
    much simpler.

    Not knowing torque at all and trying to deal with two systems that are
    essentially unknown has been making the learning curve quite steep.


    Hemanth Yamijala wrote:
    Jason Venner wrote:
    I have found that HOD writes a series of log files to directories on
    the virtual cluster master, if you specify log directories.
    The interesting part is figuring out which machine was the virtual
    cluster master, if you have a decent sized pool of machines.
    Can you explain what you mean by 'virtual cluster master' ? I guess
    you could mean the node which ran the 'ringmaster' process - the
    master process in a hod virtual cluster, or the hadoop 'jobtracker'.
    Both this information is stored by hod for an allocated cluster. You
    could retrieve it by using
    hod -o list,
    and
    hod -o "info cluster-dir"

    If the cluster is deallocated, you can get the ringmaster node still
    by using torque commands:

    qstat (to get torque job ids)
    and
    qstat -n <jobid>

    This will print a list of nodes, the first one is the ringmaster.
    It would be so nice if HOD could copy this back to the submission node.

    I suppose we could configure syslog up but then we have to configure
    the syslogd on each of the submit hosts to accept from any compute
    node...
    Syslog configuration has a bug which will be addressed in Hadoop 0.16.1.

    Thanks
    hemanth
    --
    Jason Venner
    Attributor - Publish with Confidence <http://www.attributor.com/>
    Attributor is hiring Hadoop Wranglers, contact if interested
  • Vinod KV at Feb 29, 2008 at 5:08 am

    Jason Venner wrote:
    I have found that HOD writes a series of log files to directories on
    the virtual cluster master, if you specify log directories.
    The interesting part is figuring out which machine was the virtual
    cluster master, if you have a decent sized pool of machines.
    In torque set up, HOD starts the "virtual cluster master" called
    ringmaster on the first node in the node list returned by torque. So
    doing a "qstat -f torque-jobid" or better "qstat -an jobid" will show
    you the list of nodes allocated for the current hod allocation, the
    first of them being the ringmaster.
    It would be so nice if HOD could copy this back to the submission node.
    HOD currently doesn't copy any logs, neither of the master nor of the
    hodrings, to the submission node. Ideally, one should be able to figure
    out anything wrong just by looking at the error codes.
    I suppose we could configure syslog up but then we have to configure
    the syslogd on each of the submit hosts to accept from any compute
    node...
    Yeah, that should be the case when using syslogd. But a better way would
    be to set up syslog-ng which would need setup at one centralized
    location, and let hod write all logs to that. JFYI, there is one issue
    still to be resolved ( HADOOP-2809
    <https://issues.apache.org/jira/browse/HADOOP-2809> ); once it is done
    you should be able to log to syslog/sylog-ng.

    Hope that helps,

    Thanks,
    Vinod

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedFeb 28, '08 at 3:07a
activeFeb 29, '08 at 5:08a
posts4
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase