FAQ
Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
-------------------------------------------------------------------------------------------------------------------

Key: HADOOP-6106
URL: https://issues.apache.org/jira/browse/HADOOP-6106
Project: Hadoop Common
Issue Type: Improvement
Components: util
Reporter: Hemanth Yamijala
Assignee: Sreekanth Ramakrishnan


In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Hemanth Yamijala (JIRA) at Jun 25, 2009 at 10:59 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724025#action_12724025 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    Code was being reviewed in MAPREDUCE-211. Sreekanth, can you please put up the latest patch here ?
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan

    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 25, 2009 at 11:05 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-6106:
    -------------------------------------------

    Attachment: mapred-211-common-3.patch

    Attaching Shell timeout feature patch.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 25, 2009 at 11:48 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-6106:
    -------------------------------------

    Attachment: HADOOP-6106.patch

    New patch which Sreekanth and I worked on together.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 25, 2009 at 11:52 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724057#action_12724057 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    The patch contains the following changes:

    - Converted the timedOut variable to an atomic boolean, as it was being accessed from the timer task as well as from the ShellCommandExecutor.
    - Creating the Timer only if the timeout interval is > 0.
    - Setting completed variable at exactly the same places as the previous code in order not to change contract.
    - Cancelling timer in the finally block of the code.
    - Refactored the constructors of ShellCommandExecutor to all reach one constructor.

    Sreekanth, can you please run ant test and test-patch so I can commit this ?
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 25, 2009 at 1:44 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-6106:
    -------------------------------------------

    Attachment: HADOOP-6106-1.patch

    Attaching latest patch fixing findbugs warning.

    * Changing {{ShellTimeoutTimerTask}} to private static class.

    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 25, 2009 at 1:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724089#action_12724089 ]

    Sreekanth Ramakrishnan commented on HADOOP-6106:
    ------------------------------------------------

    Output from ant test-patch
    {noformat}
    [exec]
    [exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] -1 javac. The applied patch generated 64 javac compiler warnings (more than the trunk's current 124 warnings).
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] -1 release audit. The applied patch generated 271 release audit warnings (more than the trunk's current 269 warnings).
    {noformat}

    Release audit is flagged because of changes to {{Shell}} and {{ShellCommandExecutor}} checking javac warnings does not point to any of the changes which were made in this patch.

    All tests passes successfully on local box.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 25, 2009 at 2:40 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724109#action_12724109 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    Sigh. Found one more problem. In the timer task timeout, the variable timedout must be set up before the process.destroy, because the exception would be thrown asynchronously when the process is destroyed.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 26, 2009 at 2:43 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-6106:
    -------------------------------------------

    Attachment: HADOOP-6106-2.patch

    Attaching patch as per Hemanth's comment.

    Running ant test and test-patch again.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Jun 26, 2009 at 3:45 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724374#action_12724374 ]

    Sreekanth Ramakrishnan commented on HADOOP-6106:
    ------------------------------------------------

    output from ant test-patch
    {noformat}
    [exec]
    [exec]
    [exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] -1 javac. The applied patch generated 64 javac compiler warnings (more than the trunk's current 124 warnings).
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] -1 release audit. The applied patch generated 271 release audit warnings (more than the trunk's current 269 warnings).
    [exec]
    {noformat}

    All test cases passed locally.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 26, 2009 at 6:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724397#action_12724397 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    +1 for the changes.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 26, 2009 at 6:19 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala resolved HADOOP-6106.
    --------------------------------------

    Resolution: Fixed
    Fix Version/s: 0.21.0
    Hadoop Flags: [Reviewed]

    I just committed this. Thanks, Sreekanth !
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Fix For: 0.21.0

    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Jun 26, 2009 at 11:09 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724474#action_12724474 ]

    Hudson commented on HADOOP-6106:
    --------------------------------

    Integrated in Hadoop-Common-trunk #8 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/8/])
    . Provides an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time. Contributed by Sreekanth Ramakrishnan.

    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Fix For: 0.21.0

    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 30, 2009 at 6:51 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725499#action_12725499 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    I had a chat with Owen and Giri about how to get this dependency jar into the HDFS and MapReduce sub projects. Basically the current school of thought (until IVY is fixed to automate this) is to take the latest built binary from Hudson (http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Common-trunk/) and commit it to the HDFS and MapReduce sub projects - making an entry in changes.txt referencing this JIRA.

    We are running HDFS and MapReduce unit tests with the latest jar to make sure tests work fine. Once that's done, we'll commit it.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Fix For: 0.21.0

    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 30, 2009 at 10:30 am
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725565#action_12725565 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    HDFS tests passed with the new jars.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Fix For: 0.21.0

    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Jun 30, 2009 at 1:03 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725616#action_12725616 ]

    Hemanth Yamijala commented on HADOOP-6106:
    ------------------------------------------

    Mapreduce tests also ran, except for some test case failures that are already logged. The jars can be committed to HDFS and Map/Reduce subprojects now.
    Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time.
    -------------------------------------------------------------------------------------------------------------------

    Key: HADOOP-6106
    URL: https://issues.apache.org/jira/browse/HADOOP-6106
    Project: Hadoop Common
    Issue Type: Improvement
    Components: util
    Reporter: Hemanth Yamijala
    Assignee: Sreekanth Ramakrishnan
    Fix For: 0.21.0

    Attachments: HADOOP-6106-1.patch, HADOOP-6106-2.patch, HADOOP-6106.patch, mapred-211-common-3.patch


    In MAPREDUCE-211 we came across a need to provide an option to timeout commands launched via the ShellCommandExecutor. The use case is for the health check script being developed in MAPREDUCE-211. We would like the TaskTracker thread to not be blocked by a problematic script or in instances where fork()+exec() has hung (which apparently has been observed in large clusters).
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 25, '09 at 10:59a
activeJun 30, '09 at 1:03p
posts16
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hemanth Yamijala (JIRA): 16 posts

People

Translate

site design / logo © 2022 Grokbase