FAQ
[ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1650:
----------------------------------

Attachment: 1650-1_core.patch
Upgrade Jetty to 6.x
--------------------

Key: HADOOP-1650
URL: https://issues.apache.org/jira/browse/HADOOP-1650
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Devaraj Das
Assignee: Devaraj Das
Attachments: 1650-0.patch, 1650-1_core.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch


This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Chris Douglas (JIRA) at Nov 12, 2008 at 2:00 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Attachment: 1650-1_proxy_delta.patch
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 12, 2008 at 2:00 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Attachment: 1650-1_merge.patch
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 20, 2008 at 3:47 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Attachment: 1650-2.patch

    Updated patch. I tested this using gridmix2 (HADOOP-3770) and found the performance to be comparable to trunk with the same configuration (387 nodes, 39m0s; trunk is about 36m with 400 nodes) using Jetty 6.1.14. It passes all unit tests, but generated findbugs warnings for non-serializable/transient fields ([SE_BAD_FIELD|http://findbugs.sourceforge.net/bugDescriptions.html#SE_BAD_FIELD]) in the jsps. I made an attempt to update the eclipse classpath; if someone could verify that this works for them, that would be helpful.
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-2.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 20, 2008 at 3:49 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Status: Patch Available (was: Open)

    Submitting this for review. Hudson will fail, since it doesn't have the Jetty6 jars.
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-2.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 21, 2008 at 3:21 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Attachment: shuffleData.zip

    To test performance against Jetty5, I ran a 400 node shuffle test. 40 maps wrote 5GB each, feeding 400 reducers running on 385 nodes. Jetty6 finished in 684s, Jetty5 in 711; a negligible difference. In the attached graphs, each of the two runs is visible (two nodes in the cluster (tt1, tt2) that ran map jobs from both runs). The run on the left is Jetty6, the run on the right is Jetty5.
    Filename || Tracking || Notes ||
    \*net | Network | Nearly identical; included to show when the map task ended and when the shuffle began |
    \*mem | Memory | Neither Jetty5 nor Jetty6 had substantial footprints during the shuffle. |
    \*conn | TCP connections | The rate of failed connections in Jetty5, contrasted with Jetty6, is encouraging |
    \*cpu | CPU | Comparable profiles from both Jetty5 and Jetty6 on both nodes. |
    \*load | Load | Slightly higher load running Jetty5. The spike on tt1 running Jetty5 is worth mentioning. |
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Devaraj Das
    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-2.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch, shuffleData.zip


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 22, 2008 at 1:43 am
    [ https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-1650:
    ----------------------------------

    Resolution: Fixed
    Fix Version/s: 0.20.0
    Assignee: Chris Douglas (was: Devaraj Das)
    Release Note: Upgrade all core servers to use Jetty 6
    Hadoop Flags: [Incompatible change, Reviewed]
    Status: Resolved (was: Patch Available)

    I just committed this.
    Upgrade Jetty to 6.x
    --------------------

    Key: HADOOP-1650
    URL: https://issues.apache.org/jira/browse/HADOOP-1650
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Reporter: Devaraj Das
    Assignee: Chris Douglas
    Fix For: 0.20.0

    Attachments: 1650-0.patch, 1650-1_core.patch, 1650-1_merge.patch, 1650-1_proxy_delta.patch, 1650-2.patch, 1650-20080120.patch, 1650-jetty6.1.9.patch, 1650-jetty6.1.9.v1.patch, hadoop-1650-jetty6.1.5.patch, hadoop-jetty6.1.4-lib.tar.gz, hadoop-jetty6.1.6-lib.tar.gz, hadoop-jetty6.1.9-lib.tar.gz, jetty-hadoop-6.1.6.patch, jetty-hbase.patch, jetty6.1.4.patch, jetty6.1.6.patch, shuffleData.zip


    This is the third attempt at moving to jetty6. Apparently, the jetty-6.1.4 has fixed some of the issues we discovered in jetty during HADOOP-736 and HADOOP-1273. I'd like to keep this issue open for sometime so that we have enough time to test out things.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedNov 12, '08 at 1:58a
activeNov 22, '08 at 1:43a
posts7
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Chris Douglas (JIRA): 7 posts

People

Translate

site design / logo © 2022 Grokbase