FAQ
Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
--------------------------------------------------------------------------------------------------

Key: HADOOP-4854
URL: https://issues.apache.org/jira/browse/HADOOP-4854
Project: Hadoop Core
Issue Type: Bug
Components: contrib/capacity-sched
Reporter: Vivek Ratan
Priority: Minor


The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
{code}
RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
{code}

This is incorrect for the following reasons:
* ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
* "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
* The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Sreekanth Ramakrishnan (JIRA) at Dec 12, 2008 at 8:35 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan reassigned HADOOP-4854:
    ----------------------------------------------

    Assignee: Sreekanth Ramakrishnan
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor

    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 12, 2008 at 9:46 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Attachment: HADOOP-4854-1.patch

    Attaching a patch to fix this issue. I have made changes in the forrest documentation also alongwith this patch.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 12, 2008 at 10:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655949#action_12655949 ]

    Sreekanth Ramakrishnan commented on HADOOP-4854:
    ------------------------------------------------

    Attachign output of ant test-patch which I ran on the local box.

    {noformat}
    [exec]
    [exec]
    [exec]
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec]
    {noformat}
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vivek Ratan (JIRA) at Dec 12, 2008 at 1:02 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656010#action_12656010 ]

    Vivek Ratan commented on HADOOP-4854:
    -------------------------------------

    Make sure you also update TestCapacityScheduler.setup(int, int, int), which sets the value of "mapred.capacity-scheduler.reclaimCapacity.interval" in conf, so that the reclaim thread does not interfere with the tests.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 12, 2008 at 4:47 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Attachment: HADOOP-4854-2.patch

    Attaching latest patch incorporating Vivek's comment.

    I am returning a large value from FakeResourceManager class, so the reclaim capacity interval would be large enough not to interfere for all the tests.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 15, 2008 at 4:28 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656543#action_12656543 ]

    Sreekanth Ramakrishnan commented on HADOOP-4854:
    ------------------------------------------------

    The TestCapacityScheduler is failing with the current patch, will upload a patch once i fix those issues.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 15, 2008 at 5:32 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Attachment: HADOOP-4854-3.patch

    Attaching the patch, by correcting the test case. The test case was failing because, of the hard-coded reclaim capacity interval. Made the same configurable and set it to 500 from reclaim capacity interval tests.

    Attaching output of ant test-patch :
    {noformat}
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 6 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    {noformat}

    All capacity-scheduler test cases passed when running test-contrib, chuwka test cases failed and it is reported at and worked upon here [HADOOP-4860|https://issues.apache.org/jira/browse/HADOOP-4860]
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Vivek Ratan (JIRA) at Dec 16, 2008 at 4:57 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656882#action_12656882 ]

    Vivek Ratan commented on HADOOP-4854:
    -------------------------------------

    Looks good. Some minor comments.

    # In _TestCapacityScheduler.java_:
    #* _FakeResourceManagerConf.reclaimCapcityInterval_, the new member variable you've introduced, is misspelt (missing an 'a' in 'Capacity'). It should be _reclaimCapacityInterval_.
    #* As I commented earlier, remove the code in setup(int, int, int) which explicitly sets "mapred.capacity-scheduler.reclaimCapacity.interval" in the configuration. This call is unnecessary. Remove the followling lines:
    {code}
    // set interval to a large number so thread doesn't interfere with us
    conf.setLong("mapred.capacity-scheduler.reclaimCapacity.interval", 500);
    {code}
    # In _xdocs/capacity_scheduler.xml_, use the following writeup:
    {noformat}
    <section>
    <title>Configuring the capacity scheduler</title>
    <p>The capacity scheduler's behavior can be controlled through the
    following properties.
    </p>
    <table>
    <tr>
    <th>Name</th><th>Description</th>
    </tr>
    <tr>
    <td>mapred.capacity-scheduler.reclaimCapacity.interval</td>
    <td>The time interval, in seconds, between which the scheduler
    periodically determines whether capacity needs to be reclaimed
    for any queue. The default value is 5 seconds.
    </td>
    </tr>
    </table>
    </section>
    {noformat}
    # In _conf/capacity-scheduler.xml.template_, use the following writeup:
    {noformat}
    <property>
    <name>mapred.capacity-scheduler.reclaimCapacity.interval</name>
    <value>5</value>
    <description>The time interval, in seconds, between which the scheduler
    periodically determines whether capacity needs to be reclaimed for any
    queue.
    </description>
    </property>
    {noformat}



    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 16, 2008 at 5:23 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Attachment: HADOOP-4854-4.patch

    Attaching latest patch incorporating vivek's comments.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 17, 2008 at 5:27 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Attachment: HADOOP-4854-5.patch

    Attaching new patch incorporating Vivek's offline comment,

    1. Corrected the forrest documentation for capacity scheduler. Previously had copied only the description of the property, now changed the section title and description appropriately.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch, HADOOP-4854-5.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 17, 2008 at 6:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sreekanth Ramakrishnan updated HADOOP-4854:
    -------------------------------------------

    Status: Patch Available (was: Open)
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch, HADOOP-4854-5.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Dec 18, 2008 at 6:35 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657696#action_12657696 ]

    Hemanth Yamijala commented on HADOOP-4854:
    ------------------------------------------

    This is looking good.

    Can you please upload test patch results on the last patch so I can commit ?

    Thanks !
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch, HADOOP-4854-5.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sreekanth Ramakrishnan (JIRA) at Dec 18, 2008 at 2:58 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657794#action_12657794 ]

    Sreekanth Ramakrishnan commented on HADOOP-4854:
    ------------------------------------------------

    attaching output of ant test-patch
    {noformat}
    [exec]
    [exec]
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    {noformat}

    ant test-contrib passed locally.
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch, HADOOP-4854-5.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hemanth Yamijala (JIRA) at Dec 23, 2008 at 8:18 am
    [ https://issues.apache.org/jira/browse/HADOOP-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hemanth Yamijala updated HADOOP-4854:
    -------------------------------------

    Resolution: Fixed
    Fix Version/s: 0.20.0
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I just committed this to trunk and the Hadoop 0.20 branch. Thanks, Sreekanth !

    I have only committed the documentation source (the xml files), and not the generated Forrest documentation following the new practice for committing documentation
    Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file
    --------------------------------------------------------------------------------------------------

    Key: HADOOP-4854
    URL: https://issues.apache.org/jira/browse/HADOOP-4854
    Project: Hadoop Core
    Issue Type: Bug
    Components: contrib/capacity-sched
    Reporter: Vivek Ratan
    Assignee: Sreekanth Ramakrishnan
    Priority: Minor
    Fix For: 0.20.0

    Attachments: HADOOP-4854-1.patch, HADOOP-4854-2.patch, HADOOP-4854-3.patch, HADOOP-4854-4.patch, HADOOP-4854-5.patch


    The Capacity Scheduler needs to know the time interval for checking whether capacity needs to be reclaimed. This is represented by the static variable RECLAIM_CAPACITY_INTERVAL. This value is currently read as follows:
    {code}
    RECLAIM_CAPACITY_INTERVAL = conf.getLong("mapred.capacity-scheduler.reclaimCapacity.interval", 5);
    {code}
    This is incorrect for the following reasons:
    * ""mapred.capacity-scheduler.reclaimCapacity.interval" is not present in any of the config files: hadoop-site.xml or capacity-scheduler.xml.
    * "mapred.capacity-scheduler.reclaimCapacity.interval" should be specified in the scheduler's config file, capacity-scheduler.xml. It should be read through org.apache.hadoop.mapred.CapacitySchedulerConf.
    * The Forrest documentation for the Capacity Scheduler should describe this setting. We should probably create a new sub-section under 'Configuration', titled 'Configuring properties for the Scheduler'.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedDec 12, '08 at 6:41a
activeDec 23, '08 at 8:18a
posts15
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hemanth Yamijala (JIRA): 15 posts

People

Translate

site design / logo © 2021 Grokbase