FAQ
HADOOP-6949 introduced a very important optimization to the RPC layer. Based
on the benchmarks presented in HDFS-1583 this provides an order of magnitude
improvement of current RPC implementation.
RPC is a common component of Hadoop projects. Many of them should benefit
from this change. But since this is an incompatible change it requires a
vote to be included into a previous branch.
Please vote for inclusion of this change into branch 0.22.

+1 from me.

Thanks,
--Konstantin

Search Discussions

  • Dhruba Borthakur at Mar 29, 2011 at 5:20 am
    This is a very effective optimization, +1 on pulling it to 0.22.

    -dhruba


    On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko
    wrote:
    HADOOP-6949 introduced a very important optimization to the RPC layer.
    Based
    on the benchmarks presented in HDFS-1583 this provides an order of
    magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should benefit
    from this change. But since this is an incompatible change it requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin


    --
    Connect to me at http://www.facebook.com/dhruba
  • Nigel Daley at Mar 29, 2011 at 3:03 pm
    [bcc'ing hdfs-dev and mapreduce-dev]

    Konstantin, can you outline the effects of this incompatibility?

    Thx,
    Nige
    On Mar 28, 2011, at 10:19 PM, Dhruba Borthakur wrote:

    This is a very effective optimization, +1 on pulling it to 0.22.

    -dhruba


    On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko
    wrote:
    HADOOP-6949 introduced a very important optimization to the RPC layer.
    Based
    on the benchmarks presented in HDFS-1583 this provides an order of
    magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should benefit
    from this change. But since this is an incompatible change it requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin


    --
    Connect to me at http://www.facebook.com/dhruba
  • He Chen at Mar 29, 2011 at 4:03 pm
    good news. +1.
    On Tue, Mar 29, 2011 at 10:02 AM, Nigel Daley wrote:

    [bcc'ing hdfs-dev and mapreduce-dev]

    Konstantin, can you outline the effects of this incompatibility?

    Thx,
    Nige
    On Mar 28, 2011, at 10:19 PM, Dhruba Borthakur wrote:

    This is a very effective optimization, +1 on pulling it to 0.22.

    -dhruba


    On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko
    wrote:
    HADOOP-6949 introduced a very important optimization to the RPC layer.
    Based
    on the benchmarks presented in HDFS-1583 this provides an order of
    magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should
    benefit
    from this change. But since this is an incompatible change it requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin


    --
    Connect to me at http://www.facebook.com/dhruba
  • Eli Collins at Mar 29, 2011 at 5:26 am
    +1

    On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko
    wrote:
    HADOOP-6949 introduced a very important optimization to the RPC layer. Based
    on the benchmarks presented in HDFS-1583 this provides an order of magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should benefit
    from this change. But since this is an incompatible change it requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin
  • Jakob Homan at Mar 29, 2011 at 6:06 pm
    +1
    n.b. that the vote lost hdfs and common dev at some point. I've added
    them back.
    On Tue, Mar 29, 2011 at 9:18 AM, Amit Sangroya wrote:
    +1
    On Tue, Mar 29, 2011 at 6:04 PM, Stephen Boesch wrote:
    +1

    2011/3/29 Doug Cutting <[email protected]>
    +1

    I don't think this creates an incompatibility.  It changes the RPC wire
    format, but we already require that clients and servers run identical
    builds.  No application that ran with a prior version of Hadoop would be
    broken by this change when it upgrades to this version of Hadoop.

    Doug
    On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
    HADOOP-6949 introduced a very important optimization to the RPC layer. Based
    on the benchmarks presented in HDFS-1583 this provides an order of magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should benefit
    from this change. But since this is an incompatible change it requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin
  • Konstantin Shvachko at Mar 30, 2011 at 1:08 am
    Nigel,
    The nature of incompatibility is that the RPC version is changing, which
    means
    VersionedProtocol-s become incompatible all at once. As opposed to say
    only DatanodeProtocol or mr.ClientProtocol.

    Doug is right because of our strict requirements for protocol compatibility
    this
    will not affect users upgrading to 0.22.

    Matt, thanks for checking and testing.

    --Konstantin
    On Tue, Mar 29, 2011 at 11:04 AM, Jakob Homan wrote:

    +1
    n.b. that the vote lost hdfs and common dev at some point. I've added
    them back.
    On Tue, Mar 29, 2011 at 9:18 AM, Amit Sangroya wrote:
    +1
    On Tue, Mar 29, 2011 at 6:04 PM, Stephen Boesch wrote:
    +1

    2011/3/29 Doug Cutting <[email protected]>
    +1

    I don't think this creates an incompatibility. It changes the RPC wire
    format, but we already require that clients and servers run identical
    builds. No application that ran with a prior version of Hadoop would
    be
    broken by this change when it upgrades to this version of Hadoop.

    Doug
    On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
    HADOOP-6949 introduced a very important optimization to the RPC
    layer.
    Based
    on the benchmarks presented in HDFS-1583 this provides an order of magnitude
    improvement of current RPC implementation.
    RPC is a common component of Hadoop projects. Many of them should
    benefit
    from this change. But since this is an incompatible change it
    requires a
    vote to be included into a previous branch.
    Please vote for inclusion of this change into branch 0.22.

    +1 from me.

    Thanks,
    --Konstantin

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 29, '11 at 4:40a
activeMar 30, '11 at 1:08a
posts7
users6
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2023 Grokbase