FAQ
[HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
---------------------------------------------------------------------------------

Key: HADOOP-2947
URL: https://issues.apache.org/jira/browse/HADOOP-2947
Project: Hadoop Core
Issue Type: Improvement
Components: contrib/hod
Affects Versions: 0.16.0
Reporter: Hemanth Yamijala
Fix For: 0.17.0


Copied from internal bug details from Koji:

==========================
Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.

Please have the stdout of JT/TT redirected to a file.

Adding stderr.
If thread has an uncaught exception, it prints out to stderr and dies.
==========================

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Hemanth Yamijala (JIRA) at Mar 17, 2008 at 4:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala reassigned HADOOP-2947:
    ----------------------------------------

    Assignee: Vinod Kumar Vavilapalli
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Fix For: 0.17.0


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Mar 18, 2008 at 10:05 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Attachment: HADOOP-2947

    Attaching a patch. stdout and stderr are now saved along with hadoop.log in the hadoop log directory. Thus, stdout is redirected to $HADOOP_LOG_DIR/<daemon-name>.out and stderr to $HADOOP_LOG_DIR/<daemon-name>.err.

    Made changes to simpleCommand to enable starting commands with stderr and/or stdout set. And made related changes in HadoopCommand.

    Added testcases. Also tested by giving a sigquit to jobtracker and verifying that the standard output indeed goes to the jobtracker.out file. Didn't get to check stderr.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Fix For: 0.17.0

    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Mar 18, 2008 at 10:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Status: Patch Available (was: Open)

    Running it through Hudson.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Fix For: 0.17.0

    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Mar 19, 2008 at 7:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580263#action_12580263 ]

    Hadoop QA commented on HADOOP-2947:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12378113/HADOOP-2947
    against trunk revision 619744.

    @author +1. The patch does not contain any @author tags.

    tests included +1. The patch appears to include 4 new or modified tests.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new javac compiler warnings.

    release audit +1. The applied patch does not generate any new release audit warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests -1. The patch failed core unit tests.

    contrib tests +1. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1992/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1992/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1992/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1992/console

    This message is automatically generated.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Fix For: 0.17.0

    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Mar 21, 2008 at 8:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581025#action_12581025 ]

    Hemanth Yamijala commented on HADOOP-2947:
    ------------------------------------------

    After looking at the changes, we realize there are some issues with the patch. Also, this is requiring changes to a core functionality of HOD. So, moving this out of the 0.17 list.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Mar 21, 2008 at 8:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-2947:
    -------------------------------------

    Fix Version/s: (was: 0.17.0)
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Mar 27, 2008 at 9:53 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-2947:
    -------------------------------------

    Priority: Blocker (was: Major)
    Fix Version/s: 0.17.0

    Marking this as a blocker, after discussions with Mukund and Sameer. We are close to the fix, and can be done in a couple of days, and it seems a useful thing to have.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Mar 31, 2008 at 4:32 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Status: Open (was: Patch Available)
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Mar 31, 2008 at 4:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Attachment: HADOOP-2947.2

    Earlier patch kind of does a radical surgery on simpleCommand, though, as Hemanth suggested, this could be done without disturbing much of the current code. Attaching a new patch. This uses the current simpleCommand framework to redirect the stderr and stdout to files. Redirecting these on all daemons to <daemon-name.out> and <daemon-name.err> in the corresponding log directories, which when log-destination-uri is specified, will be archived along with hadoop logs onto dfs.

    Tested stdout by doing a sigquit, tested stderr by forcing namenode to fail in intializing(for e.g removed logging jar from hadoop lib dir). Tested to make sure that streams are getting redirected to respective files as desired.

    Added test cases also.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Mar 31, 2008 at 5:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583755#action_12583755 ]

    Hemanth Yamijala commented on HADOOP-2947:
    ------------------------------------------

    Functionality works well with the patch. Also code looks good.

    One minor point is the test cases. The current tests are using ls to check output. The output of these programs could be environment specific, and the tests could fail because of that.

    For e.g. I verified on FreeBSD that when ls fails, what comes out on stderr is something like
    ls: <filename>: No such file or directory,

    whereas on Linux it says:
    /bin/ls: <filename>: No such file or directory.

    So, I think we should have tests that aren't dependent on platforms like this.

    Other than this, +1
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Apr 1, 2008 at 5:52 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Attachment: HADOOP-2947.3

    Added a helper program that prints sample text to stderr, and changed the test-case testRedirectedStderr to call this helper program instead.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Apr 1, 2008 at 5:56 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Status: Patch Available (was: Open)
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Apr 1, 2008 at 7:56 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584045#action_12584045 ]

    Hemanth Yamijala commented on HADOOP-2947:
    ------------------------------------------

    - The patch assumes current working directory while referring to the helper program. This will not work
    - Can we also change the stdout test to use the helper program. I like having the expected output of the test under our complete control. For e.g. say ls is aliased to something in some environment, this test would break again.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Apr 1, 2008 at 9:40 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Status: Open (was: Patch Available)
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Apr 1, 2008 at 9:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Attachment: HADOOP-2947.4

    Incorporating the suggested changes.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vinod Kumar Vavilapalli (JIRA) at Apr 1, 2008 at 9:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vinod Kumar Vavilapalli updated HADOOP-2947:
    --------------------------------------------

    Status: Patch Available (was: Open)
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Apr 1, 2008 at 4:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584195#action_12584195 ]

    Hemanth Yamijala commented on HADOOP-2947:
    ------------------------------------------

    +1 for the changes.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Apr 1, 2008 at 4:56 pm
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584212#action_12584212 ]

    Hadoop QA commented on HADOOP-2947:
    -----------------------------------

    +1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12379020/HADOOP-2947.4
    against trunk revision 643282.

    @author +1. The patch does not contain any @author tags.

    tests included +1. The patch appears to include 7 new or modified tests.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new javac compiler warnings.

    release audit +1. The applied patch does not generate any new release audit warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests +1. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2112/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2112/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2112/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2112/console

    This message is automatically generated.
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Apr 2, 2008 at 5:19 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-2947:
    -------------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Vinod !
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Apr 17, 2008 at 5:20 am
    [ https://issues.apache.org/jira/browse/HADOOP-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-2947:
    -------------------------------------

    Release Note: The stdout and stderr streams of daemons are redirected to files that are created under the hadoop log directory. Users can now send kill 3 signals to the daemons to get stack traces and thread dumps for debugging.
    Hadoop Flags: [Reviewed]
    [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging
    ---------------------------------------------------------------------------------

    Key: HADOOP-2947
    URL: https://issues.apache.org/jira/browse/HADOOP-2947
    Project: Hadoop Core
    Issue Type: Improvement
    Components: contrib/hod
    Affects Versions: 0.16.0
    Reporter: Hemanth Yamijala
    Assignee: Vinod Kumar Vavilapalli
    Priority: Blocker
    Fix For: 0.17.0

    Attachments: HADOOP-2947, HADOOP-2947.2, HADOOP-2947.3, HADOOP-2947.4


    Copied from internal bug details from Koji:
    ==========================
    Sometimes JobTracker/TaskTracker starts consuming 99% cpu and stops responding to 'jstack' call. In those cases,
    usually it still responds to kill -QUIT signal which forces the jvm to dump the stack to stdout.
    Please have the stdout of JT/TT redirected to a file.
    Adding stderr.
    If thread has an uncaught exception, it prints out to stderr and dies.
    ==========================
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 6, '08 at 8:26a
activeApr 17, '08 at 5:20a
posts21
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hemanth Yamijala (JIRA): 21 posts

People

Translate

site design / logo © 2022 Grokbase