Grokbase › Groups › Hadoop › common-user › November 2011
FAQ
Hello

We are currently using hadoop 0.20.203 on a 10 node cluster. We are considering upgrading to a newer version and I have two questions in this regard.

1) It seems 0.21 is unlikely to become a stable release anytime soon and we are weary of moving to an unstable release. Our primary concern is the data we have on our hdfs. I want to know if anyone has been using 0.21 in production and would like to hear about your experiences? Any advice on this front is appreciated.

2) Do we know when 0.23 is likely to become stable? There has been some discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it going to happen by the end of this year?

Thanks
~ Niranjan.

* - http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+

Search Discussions

  • Konstantin Boudnik at Nov 22, 2011 at 8:24 pm ⇧
    We are expecting to release 0.22 very shortly. 0.22 is suppose to be
    considered stable because it has been heavily tested at scale by eBay team
    (as far as I know). However, I will let 0.22's RM to comment on that.

    Cos
    On Tue, Nov 22, 2011 at 12:05PM, Niranjan Balasubramanian wrote:
    Hello

    We are currently using hadoop 0.20.203 on a 10 node cluster. We are
    considering upgrading to a newer version and I have two questions in this
    regard.

    1) It seems 0.21 is unlikely to become a stable release anytime soon and we
    are weary of moving to an unstable release. Our primary concern is the data
    we have on our hdfs. I want to know if anyone has been using 0.21 in
    production and would like to hear about your experiences? Any advice on this
    front is appreciated.

    2) Do we know when 0.23 is likely to become stable? There has been some
    discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
    going to happen by the end of this year?

    Thanks
    ~ Niranjan.

    * - http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+
  • Arun C Murthy at Nov 22, 2011 at 10:06 pm ⇧

    On Nov 22, 2011, at 12:05 PM, Niranjan Balasubramanian wrote:

    Hello

    We are currently using hadoop 0.20.203 on a 10 node cluster. We are considering upgrading to a newer version and I have two questions in this regard.

    1) It seems 0.21 is unlikely to become a stable release anytime soon and we are weary of moving to an unstable release. Our primary concern is the data we have on our hdfs. I want to know if anyone has been using 0.21 in production and would like to hear about your experiences? Any advice on this front is appreciated.

    2) Do we know when 0.23 is likely to become stable? There has been some discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it going to happen by the end of this year?
    Currently hadoop-0.23 has undergone significant feature, scale and performance testing at ~500 nodes, but there is some more to go.

    We anticipate deploying to a few alpha clusters of sizes between 1000-2000 nodes in Jan in secure environments (is security important to you?).

    I'd say for your scale late Q1, 2012 should a good time frame.

    Arun
  • Sridhar basam at Nov 22, 2011 at 10:19 pm ⇧

    On Tue, Nov 22, 2011 at 3:05 PM, Niranjan Balasubramanian wrote:

    Hello

    We are currently using hadoop 0.20.203 on a 10 node cluster. We are
    considering upgrading to a newer version and I have two questions in this
    regard.

    1) It seems 0.21 is unlikely to become a stable release anytime soon and
    we are weary of moving to an unstable release. Our primary concern is the
    data we have on our hdfs. I want to know if anyone has been using 0.21 in
    production and would like to hear about your experiences? Any advice on
    this front is appreciated.
    What new features or what deficiencies in the existing release are your
    trying to overcome. If primary concern is stability, wouldn't it be better
    to wait till 0.23 is GA'd. I don't think there are too many people running
    0.21 at scale.

    Sridhar


    2) Do we know when 0.23 is likely to become stable? There has been some
    discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
    going to happen by the end of this year?

    Thanks
    ~ Niranjan.

    * -
    http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+
  • Niranjan Balasubramanian at Nov 22, 2011 at 10:33 pm ⇧
    Cos, Arun and Sridhar,

    Thanks for the fast responses. Our two main requirements are:

    1. Multiple output -- Write multiple output files in each reducer.

    2. Max Reducers per user -- Use Fairscheduler 0.21 or greater which would let us limit the maximum number of concurrent reducers per pool/user.


    ~ Niranjan.
    On Nov 22, 2011, at 2:19 PM, sridhar basam wrote:

    On Tue, Nov 22, 2011 at 3:05 PM, Niranjan Balasubramanian <
    [email protected]> wrote:
    Hello

    We are currently using hadoop 0.20.203 on a 10 node cluster. We are
    considering upgrading to a newer version and I have two questions in this
    regard.

    1) It seems 0.21 is unlikely to become a stable release anytime soon and
    we are weary of moving to an unstable release. Our primary concern is the
    data we have on our hdfs. I want to know if anyone has been using 0.21 in
    production and would like to hear about your experiences? Any advice on
    this front is appreciated.
    What new features or what deficiencies in the existing release are your
    trying to overcome. If primary concern is stability, wouldn't it be better
    to wait till 0.23 is GA'd. I don't think there are too many people running
    0.21 at scale.

    Sridhar


    2) Do we know when 0.23 is likely to become stable? There has been some
    discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
    going to happen by the end of this year?

    Thanks
    ~ Niranjan.

    * -
    http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+
  • Arun C Murthy at Nov 22, 2011 at 10:37 pm ⇧
    Sridhar,
    On Nov 22, 2011, at 2:33 PM, Niranjan Balasubramanian wrote:

    Cos, Arun and Sridhar,

    Thanks for the fast responses. Our two main requirements are:

    1. Multiple output -- Write multiple output files in each reducer.

    2. Max Reducers per user -- Use Fairscheduler 0.21 or greater which would let us limit the maximum number of concurrent reducers per pool/user.
    Those are fairly small reasons to upgrade... :)

    You can easily use the MultipleOutput stuff with 0.20.2xx (it's just user-land library code) and FairScheduler changes are already in 0.20.205. You might want to consider that.

    Arun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedNov 22, '11 at 8:05p
activeNov 22, '11 at 10:37p
posts6
users4
websitehadoop.apache.org...
irc#hadoop

4 users in discussion

Arun C Murthy: 2 posts Niranjan Balasubramanian: 2 posts Sridhar basam: 1 post Konstantin Boudnik: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase