FAQ
I've created a release candidate for 0.20.204.0 that I would like to release.

It is available at: http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/

0.20.204.0 has many fixes including disk fail in place and the new rpm and deb packages. Fail in place allows the DataNode and TaskTracker to continue after a hard drive fails.

-- Owen

Search Discussions

  • Allen Wittenauer at Jul 28, 2011 at 7:04 pm

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to release.

    It is available at: http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/

    0.20.204.0 has many fixes including disk fail in place and the new rpm and deb packages. Fail in place allows the DataNode and TaskTracker to continue after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Giridharan Kesavan at Jul 28, 2011 at 8:57 pm
    Myself and Eric Yang are looking into this.
    -Giri
    On 7/28/11 12:04 PM, "Allen Wittenauer" wrote:

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to release.

    It is available at: http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/

    0.20.204.0 has many fixes including disk fail in place and the new rpm and
    deb packages. Fail in place allows the DataNode and TaskTracker to continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Giridharan Kesavan at Jul 29, 2011 at 12:03 am
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri
    On 7/28/11 12:04 PM, "Allen Wittenauer" wrote:

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at: http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/

    0.20.204.0 has many fixes including disk fail in place and the new rpm and
    deb packages. Fail in place allows the DataNode and TaskTracker to continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Aaron T. Myers at Jul 29, 2011 at 12:12 am
    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri
    On 7/28/11 12:04 PM, "Allen Wittenauer" wrote:

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Arun C Murthy at Jul 29, 2011 at 12:27 am
    Nope. general@ is only for announcements.

    AFAIK Votes are developer activities.

    Arun
    On Jul 28, 2011, at 5:11 PM, Aaron T. Myers wrote:

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri

    On 7/28/11 12:04 PM, "Allen Wittenauer" <awittenauer@linkedin.com>
    wrote:
    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Milind Bhandarkar at Jul 29, 2011 at 12:37 am
    Somehow I remember that the 0.20.203.0 vote was carried out on general@

    Here is a message from the archive:

    http://mail-archives.apache.org/mod_mbox/hadoop-general/201105.mbox/browser

    - milind

    ---
    Milind Bhandarkar


    -----Original Message-----
    From: Arun C Murthy
    Sent: Thursday, July 28, 2011 5:27 PM
    To: common-dev@hadoop.apache.org
    Subject: Re: [VOTE] Release 0.20.204.0-rc0

    Nope. general@ is only for announcements.

    AFAIK Votes are developer activities.

    Arun
    On Jul 28, 2011, at 5:11 PM, Aaron T. Myers wrote:

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri

    On 7/28/11 12:04 PM, "Allen Wittenauer" <awittenauer@linkedin.com>
    wrote:
    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Arun C Murthy at Jul 29, 2011 at 12:40 am
    In the past we've carried it out on common-dev:
    http://hadoop-common.472056.n3.nabble.com/VOTE-Release-Hadoop-0-21-0-candidate-2-td1181981.html

    Arun
    On Jul 28, 2011, at 5:33 PM, wrote:

    Somehow I remember that the 0.20.203.0 vote was carried out on general@

    Here is a message from the archive:

    http://mail-archives.apache.org/mod_mbox/hadoop-general/201105.mbox/browser

    - milind

    ---
    Milind Bhandarkar


    -----Original Message-----
    From: Arun C Murthy
    Sent: Thursday, July 28, 2011 5:27 PM
    To: common-dev@hadoop.apache.org
    Subject: Re: [VOTE] Release 0.20.204.0-rc0

    Nope. general@ is only for announcements.

    AFAIK Votes are developer activities.

    Arun
    On Jul 28, 2011, at 5:11 PM, Aaron T. Myers wrote:

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri

    On 7/28/11 12:04 PM, "Allen Wittenauer" <awittenauer@linkedin.com>
    wrote:
    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Eli Collins at Jul 29, 2011 at 1:40 am
    We've done both, even within a branch (0.20.0 was voted on core-dev@
    and 0.20.2 on general@). The bylaws suggest general@ should be used,
    which seems to make sense since we're releasing common, hdfs and mr. I
    think either works as long as people know where to check.

    http://hadoop.apache.org/bylaws.html

    "Voting
    Decisions regarding the project are made by votes on the primary
    project development mailing list (general@hadoop.apache.org)"

    On Thu, Jul 28, 2011 at 5:40 PM, Arun C Murthy wrote:
    In the past we've carried it out on common-dev:
    http://hadoop-common.472056.n3.nabble.com/VOTE-Release-Hadoop-0-21-0-candidate-2-td1181981.html

    Arun
    On Jul 28, 2011, at 5:33 PM, wrote:

    Somehow I remember that the 0.20.203.0 vote was carried out on general@

    Here is a message from the archive:

    http://mail-archives.apache.org/mod_mbox/hadoop-general/201105.mbox/browser

    - milind

    ---
    Milind Bhandarkar


    -----Original Message-----
    From: Arun C Murthy
    Sent: Thursday, July 28, 2011 5:27 PM
    To: common-dev@hadoop.apache.org
    Subject: Re: [VOTE] Release 0.20.204.0-rc0

    Nope. general@ is only for announcements.

    AFAIK Votes are developer activities.

    Arun
    On Jul 28, 2011, at 5:11 PM, Aaron T. Myers wrote:

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri

    On 7/28/11 12:04 PM, "Allen Wittenauer" <awittenauer@linkedin.com>
    wrote:
    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Arun C Murthy at Jul 29, 2011 at 2:09 am
    Thanks for the link Eli.
    On Jul 28, 2011, at 6:40 PM, Eli Collins wrote:

    We've done both, even within a branch (0.20.0 was voted on core-dev@
    and 0.20.2 on general@). The bylaws suggest general@ should be used,
    which seems to make sense since we're releasing common, hdfs and mr. I
    think either works as long as people know where to check.

    http://hadoop.apache.org/bylaws.html

    "Voting
    Decisions regarding the project are made by votes on the primary
    project development mailing list (general@hadoop.apache.org)"
    IMO general@ isn't the primary 'development' mailing list - it's just an 'announcement' list.

    But, it doesn't really matter... do folks feel strongly we should restart the vote on general?

    Arun
    On Thu, Jul 28, 2011 at 5:40 PM, Arun C Murthy wrote:
    In the past we've carried it out on common-dev:
    http://hadoop-common.472056.n3.nabble.com/VOTE-Release-Hadoop-0-21-0-candidate-2-td1181981.html

    Arun
    On Jul 28, 2011, at 5:33 PM, wrote:

    Somehow I remember that the 0.20.203.0 vote was carried out on general@

    Here is a message from the archive:

    http://mail-archives.apache.org/mod_mbox/hadoop-general/201105.mbox/browser

    - milind

    ---
    Milind Bhandarkar


    -----Original Message-----
    From: Arun C Murthy
    Sent: Thursday, July 28, 2011 5:27 PM
    To: common-dev@hadoop.apache.org
    Subject: Re: [VOTE] Release 0.20.204.0-rc0

    Nope. general@ is only for announcements.

    AFAIK Votes are developer activities.

    Arun
    On Jul 28, 2011, at 5:11 PM, Aaron T. Myers wrote:

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri

    On 7/28/11 12:04 PM, "Allen Wittenauer" <awittenauer@linkedin.com>
    wrote:
    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
  • Aaron T. Myers at Jul 29, 2011 at 2:14 am

    On Thu, Jul 28, 2011 at 7:08 PM, Arun C Murthy wrote:

    But, it doesn't really matter... do folks feel strongly we should restart
    the vote on general?
    I don't think there's need to restart the vote. I only brought this up
    because I've heard from a few people that they didn't know a release vote
    was going on. Let's just send an email to general@ saying there's a vote
    going on on common-dev@, and from now on be sure to send votes to general@.
    As Eli already said, it doesn't matter which it is, as long as people know
    where to look.

    --
    Aaron T. Myers
    Software Engineer, Cloudera
  • Arun C Murthy at Jul 29, 2011 at 4:55 am
    Done. Thanks.
    On Jul 28, 2011, at 7:13 PM, Aaron T. Myers wrote:
    On Thu, Jul 28, 2011 at 7:08 PM, Arun C Murthy wrote:

    But, it doesn't really matter... do folks feel strongly we should restart
    the vote on general?
    I don't think there's need to restart the vote. I only brought this up
    because I've heard from a few people that they didn't know a release vote
    was going on. Let's just send an email to general@ saying there's a vote
    going on on common-dev@, and from now on be sure to send votes to general@.
    As Eli already said, it doesn't matter which it is, as long as people know
    where to look.

    --
    Aaron T. Myers
    Software Engineer, Cloudera
  • Steve Loughran at Jul 29, 2011 at 11:03 am

    On 29/07/11 03:13, Aaron T. Myers wrote:
    On Thu, Jul 28, 2011 at 7:08 PM, Arun C Murthywrote:
    But, it doesn't really matter... do folks feel strongly we should restart
    the vote on general?
    I don't think there's need to restart the vote. I only brought this up
    because I've heard from a few people that they didn't know a release vote
    was going on. Let's just send an email to general@ saying there's a vote
    going on on common-dev@, and from now on be sure to send votes to general@.
    As Eli already said, it doesn't matter which it is, as long as people know
    where to look.

    I'd like to rm the log4j.properties file; I'll apply the patch for this
    to the branch and build and test locally. Yes: test.
  • Steve Loughran at Jul 29, 2011 at 1:54 pm

    On 29/07/11 12:01, Steve Loughran wrote:
    On 29/07/11 03:13, Aaron T. Myers wrote:
    On Thu, Jul 28, 2011 at 7:08 PM, Arun C Murthy<acm@hortonworks.com>
    wrote:
    But, it doesn't really matter... do folks feel strongly we should
    restart
    the vote on general?
    I don't think there's need to restart the vote. I only brought this up
    because I've heard from a few people that they didn't know a release vote
    was going on. Let's just send an email to general@ saying there's a vote
    going on on common-dev@, and from now on be sure to send votes to
    general@.
    As Eli already said, it doesn't matter which it is, as long as people
    know
    where to look.

    I'd like to rm the log4j.properties file; I'll apply the patch for this
    to the branch and build and test locally. Yes: test.

    I've committed the HADOOP-7468 patch to build.xml to stop
    conf/log4.properties going into the JAR to trunk.

    There's a separate patch for the 0.20.204 release, which I'd argue for
    inclusion
    1. It doesn't impact the server code
    2. It breaks any app downstream that wants its own log4j. To be
    precise, it causes inconsistent behaviour depending on the classpath
    ordering.

    This change isn't going to impact any of the hadoop scripts -including
    client side ones- that have a the conf/ dir on the classpath.

    -Steve
  • Allen Wittenauer at Jul 29, 2011 at 3:59 pm
    I can't believe we're holding a vote on a release that isn't passing the nightly build. If my vote was binding, I'd -1 it based upon that alone.
  • Mahadev Konar at Jul 29, 2011 at 4:24 pm
    Allen,
    I think Giri already sent out an email for that. Below is the
    response from him. There'll be a new rc candidate soon.

    Hope that helps.

    thanks
    mahadev


    ===================================

    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri
    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri
    On 7/28/11 12:04 PM, "Allen Wittenauer" wrote:

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at: http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/

    0.20.204.0 has many fixes including disk fail in place and the new rpm and
    deb packages. Fail in place allows the DataNode and TaskTracker to continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?
    ==============================
    On Fri, Jul 29, 2011 at 8:59 AM, Allen Wittenauer wrote:


    I can't believe we're holding a vote on a release that isn't passing the nightly build.  If my vote was binding, I'd -1 it based upon that alone.
  • Steve Loughran at Jul 29, 2011 at 11:06 am
  • Suresh Srinivas at Jul 29, 2011 at 4:51 pm
    I'd like to get HADOOP-7468 in, which deletes log4j.properties from the
    JAR. Someone did a patch for it yesterday
    This bug is not marked as a blocker. In that case I think we should pick
    this up in 205 release that is going to out soon.

    On Fri, Jul 29, 2011 at 4:04 AM, Steve Loughran wrote:

    Incidentally, has anyone tested on Java7.

    The Lucene team are unhappy:
    http://www.lucidimagination.**com/search/document/**
    1a0d3986e48a9348/warning_**index_corruption_and_crashes_**
    in_apache_lucene_core_apache_**solr_with_java_7<http://www.lucidimagination.com/search/document/1a0d3986e48a9348/warning_index_corruption_and_crashes_in_apache_lucene_core_apache_solr_with_java_7>
    I do not think this is blocker either.

    New release candidate will be out soon that fixes the Jenkins failures. Vote
    will resume on that RC.

    Regards,
    Suresh
  • Matt Foley at Aug 2, 2011 at 10:02 am
    Hi,
    Four critical patches have been applied to 0.20-security-204, and release
    candidate 0.20.204-rc1 is now ready for evaluation.
    The signed release is available at
    http://people.apache.org/~mattf/hadoop-0.20.204-rc1/
    A successful build and test under Jenkins may be examined at
    https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-0.20.204-Build/24/

    The four changes since rc0 are:

    ------------------------------------------------------------------------
    r1152887 | mattf | 2011-08-01 11:32:12 -0700 (Mon, 01 Aug 2011) | 1 line
    HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
    suite for 0.20-security-204 release.
    ------------------------------------------------------------------------
    r1152037 | gkesavan | 2011-07-28 23:39:42 +0000 (Thu, 28 Jul 2011) | 1 line
    HADOOP-7356. Fix bin scripts to support dev build environment
    ------------------------------------------------------------------------
    r1150862 | ddas | 2011-07-25 19:32:59 +0000 (Mon, 25 Jul 2011) | 1 line
    Merge -r 1150859:1150860 from branch-0.20-security onto
    branch-0.20-security-204
    ------------------------------------------------------------------------
    r1150857 | ddas | 2011-07-25 19:28:14 +0000 (Mon, 25 Jul 2011) | 1 line
    MAPREDUCE-2621. Merge -r 1150527:1150528 from branch-0.20-security onto
    branch-0.20-security-204
    ------------------------------------------------------------------------


    Although Owen is the RM for this release, he requested assistance in
    progressing the release while he is traveling.
    Please resume the 0.20.204 release vote today, using this rc1 candidate.

    Thank you,
    --Matt

    On Fri, Jul 29, 2011 at 9:51 AM, Suresh Srinivas wrote:

    ....
    New release candidate will be out soon that fixes the Jenkins failures.
    Vote
    will resume on that RC.

    Regards,
    Suresh
  • Steve Loughran at Aug 2, 2011 at 11:34 am

    On 02/08/11 11:00, Matt Foley wrote:
    Hi,
    Four critical patches have been applied to 0.20-security-204, and release
    candidate 0.20.204-rc1 is now ready for evaluation.
    The signed release is available at
    http://people.apache.org/~mattf/hadoop-0.20.204-rc1/
    A successful build and test under Jenkins may be examined at
    https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-0.20.204-Build/24/
    I'm getting confused about release roadmaps right now


    Is there somewhere that lists the (proposed) timetable for the 0.20.204,
    0.20.205, 0.22, 0.23 releases?

    It sounds like any non-critical problem in 20.204rc0 is intended to go
    into to the 0.20.205 release. Correct? If so is there a release manager
    or can I go ahead and patch in the normal RTC process?
  • Allen Wittenauer at Aug 2, 2011 at 6:08 pm

    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:

    I'm getting confused about release roadmaps right now
    branch-20 is the new trunk, given that features keep popping up in it rather than bug fixes.
  • Eli Collins at Aug 2, 2011 at 7:24 pm

    On Tue, Aug 2, 2011 at 11:08 AM, Allen Wittenauer wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:

    I'm getting confused about release roadmaps right now
    branch-20 is the new trunk, given that features keep popping up in it rather than bug fixes.
    When we voted to adopt 203.rc0 we also voted to adopt the new 4 part
    version scheme which was intended to allow feature development on the
    203.x series. See http://search-hadoop.com/m/iegPf7aDvh1

    branch-20-security is not the new trunk, trunk and branches from trunk
    are where most of the action is happening. However it is disappointing
    to see some of the features being developed on branch-20-security,
    rather being developed first on trunk and then ported to
    branch-20-security.

    Thanks,
    Eli
  • Allen Wittenauer at Aug 2, 2011 at 8:33 pm

    On Aug 2, 2011, at 12:23 PM, Eli Collins wrote:

    However it is disappointing
    to see some of the features being developed on branch-20-security,
    rather being developed first on trunk and then ported to
    branch-20-security.
    ... which was exactly my point.
  • Steve Loughran at Aug 3, 2011 at 10:06 am

    On 02/08/11 21:33, Allen Wittenauer wrote:
    On Aug 2, 2011, at 12:23 PM, Eli Collins wrote:

    However it is disappointing
    to see some of the features being developed on branch-20-security,
    rather being developed first on trunk and then ported to
    branch-20-security.
    ... which was exactly my point.
    Yes, I think this is wrong

    Every feature should go into trunk then be backported to the branch, so
    that trunk is a proper superset of the branch
  • Eric Yang at Aug 3, 2011 at 4:35 pm
    RPM packaging and Ganglia plugin for metrics v2 features committed to trunk before in branch-0.20-security. There is no rule that declares the new feature landed in trunk needs to materialized into a release then back port is allowed to happen. Those are the only two new minor features that were back ported from trunk to 0.20 security branch. No one is using branch-0.20-security as their new trunk. Hence, the 4 part version number should not apply.

    Majority of users are not affected by the utility features introduced in 0.20.204. Change Log shows what is in this release. 0.20.204 is incremental improvement of existing 0.20.x release.

    regards,
    Eric
    On Aug 2, 2011, at 12:23 PM, Eli Collins wrote:
    On Tue, Aug 2, 2011 at 11:08 AM, Allen Wittenauer wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:

    I'm getting confused about release roadmaps right now
    branch-20 is the new trunk, given that features keep popping up in it rather than bug fixes.
    When we voted to adopt 203.rc0 we also voted to adopt the new 4 part
    version scheme which was intended to allow feature development on the
    203.x series. See http://search-hadoop.com/m/iegPf7aDvh1

    branch-20-security is not the new trunk, trunk and branches from trunk
    are where most of the action is happening. However it is disappointing
    to see some of the features being developed on branch-20-security,
    rather being developed first on trunk and then ported to
    branch-20-security.

    Thanks,
    Eli
  • Eli Collins at Aug 3, 2011 at 4:46 pm

    On Wed, Aug 3, 2011 at 9:35 AM, Eric Yang wrote:
    RPM packaging and Ganglia plugin for metrics v2 features committed to trunk before in branch-0.20-security.  There is no rule that declares the new feature landed in trunk needs to materialized into a release then back port is allowed to happen.
    No one is suggesting that. We're saying features should be developed
    on trunk first, before being backported to a release branch. Yes,
    there is no current rule that says trunk needs to be a superset of
    branch-0.20, but it would help us not get stuck on a given release for
    too long.
    Those are the only two new minor features that were back ported from trunk to 0.20 security branch.
    There are others, eg MR disk failure handling, MR security, etc. that
    are not in trunk but are in branch-0.20-security.
    No one is using branch-0.20-security as their new trunk.
    Hence, the 4 part version number should not apply.
    If that's so then why use a 4 part version (0.20.204.0)?

    Thanks,
    Eli
  • Steve Loughran at Aug 3, 2011 at 5:28 pm

    On 02/08/11 20:23, Eli Collins wrote:
    On Tue, Aug 2, 2011 at 11:08 AM, Allen Wittenauerwrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:

    I'm getting confused about release roadmaps right now
    branch-20 is the new trunk, given that features keep popping up in it rather than bug fixes.
    When we voted to adopt 203.rc0 we also voted to adopt the new 4 part
    version scheme which was intended to allow feature development on the
    203.x series. See http://search-hadoop.com/m/iegPf7aDvh1
    This makes things clearer.

    Now, if I were to propagate a change from trunk to the 0.20.20x branch
    by committing it to
    https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-204/
    , that's going to get into .204? Yet my proposal to do what was
    rejected, saying "hold for .205". Should someone create a .205 branch so
    that we can backport all non-critical patches for the 20.20x line into
    there, while the 0.20.204 becomes frozen for the release except for
    absolutely critical patches?
  • Matt Foley at Aug 3, 2011 at 9:15 pm

    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    Here's the analysis of changes in 204 vs trunk. We believe that ALL the
    changes in 204 are either already in trunk or do not need to be in trunk.
    Here is the list of 204 items not already in trunk, and their
    categorization.

    Please, if anyone thinks we've missed something, bring it to my attention
    and if it isn't already in trunk we will get it into trunk as expeditiously
    as possible.
    Thanks,
    --Matt

    Not needed for trunk:
    HDFS-1758. Make Web UI JSP pages thread safe. (Tanping via suresh) (Not a
    problem in trunk)
    HDFS-2057. Wait time to terminate the threads causes unit tests to take
    longer time. (Bharath Mundlapudi via suresh) (Not a problem in trunk)
    HDFS-1878. TestHDFSServerPorts unit test failure - race condition in
    FSNamesystem.close() causes NullPointerException without serious
    consequence. (mattf) (not a problem in trunk)
    HDFS-1842. Handle editlog opcode conflict with 0.20.203 during upgrade, by
    throwing an error to indicate the editlog needs to be empty. (suresh)
    (Handled by HDFS-1824)
    HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
    suite for 0.20-security-204 release. (Matt Foley) (Not a problem in trunk)
    HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
    timing issues. (mattf) (not a problem in trunk)
    HADOOP-7459. Remove jdk-1.6.0 dependency check from rpm. (omalley) (replaced
    by HDFS-2156)
    HADOOP-7398. Suppress warnings about use of HADOOP_HOME. (omalley) (not a
    problem in trunk)
    HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
    (not a problem in trunk)
    HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from trying
    to use the wrong bin directory. (omalley) (not a problem in trunk)
    HADOOP-7475. Fix hadoop-setup-single-node.sh to reflect new layout. (eyang
    via omalley) (not a problem in trunk)

    Back port from trunk:
    MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to fail
    faulty maps more aggressively. (Thomas Graves via cdouglas)
    MAPREDUCE-2479. Move distributed cache cleanup to a background task,
    backporting MAPREDUCE-1568. (Robert Joseph Evans via cdouglas)
    HDFS-2023. Backport of NPE for File.list and File.listFiles. Merged ports
    of HADOOP-7322, HDFS-1934, HADOOP-7342, and HDFS-2019. (Bharath Mundlapudi
    via mattf)
    HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
    (Thomas Graves and Tom White via cdouglas) (from HADOOP-6407)
    HADOOP-7274. Fix typos in IOUtils. (Jonathan Eagles via cdouglas) (from
    HADOOP-7057)
    HADOOP-7277. Add generation of run configurations to eclipse target.
    (Jeffrey Naisbitt and Philip Zeyliger via cdouglas) (from HADOOP-5911)
    HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
    something other than build/test. (Thomas Graves via mahadev) (from
    MAPREDUCE-2575)

    Already in trunk for MRV2:
    MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
    (Jeffrey Naisbitt via mahadev)
    MAPREDUCE-2418. Show job errors in JobHistory page. (Siddharth Seth via
    acmurthy) (Already in MR279)
    MAPREDUCE-2411. Force an exception when the queue has an invalid name or its
    ACLs are misconfigured. (Dick King via cdouglas) (Already in MR279)

    Waiting for MR279 merge:
    MAPREDUCE-2429. Validate JVM in TaskUmbilicalProtocol. (Siddharth Seth via
    acmurthy)
    MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid corner
    cases in error handling. (Siddharth Seth via acmurthy)

    MapReduce v1 exceptions:
    MAPREDUCE-2415. Distribute the user task logs on to multiple disks.
    (Bharath Mundlapudi via omalley)
    MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
    itself. (Ravi Gummadi and Jagane Sundar via omalley)
    MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
    exist". (Sherry Chen via mahadev)
    MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
    Evans via cdouglas)
    MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
    via cdouglas)
    MAPREDUCE-2443. Fix TaskAspect for TaskUmbilicalProtocol.ping(..).
    (Siddharth Seth via szetszwo)





    On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204,
    0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I
    apologize for the lack of clarity on the roadmap and timelines.

    Eric did send out a note on the motivation for sustaining releases (
    http://s.apache.org/hadoop-sustenance-releases) - which I believe is
    important to keep Hadoop installations going. However, I accept that there
    has been very poor clarity on the process for inclusion of content to the
    sustaining releases and the timelines.

    Here is my proposal for the community process for sustaining releases
    moving forward:
    # The Release Manager should clearly communicate, upfront, the expected
    timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to
    trunk, if applicable*, and to the branch-0.20-security branch. Then talk to
    the Release Manager for inclusion via the normal process.

    The RM is responsible for release content, timelines and for enforcing that
    each change should have an equivalent for trunk, as applicable, before
    inclusion.

    If this makes sense, I'll add this to the wiki to record it.

    ----

    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    thanks,
    Arun

    *Please note the exception that we have agreed that MRv2 is the way forward
    (http://s.apache.org/mr1) , and thus, MR1 patches might not be ported
    as-is to trunk in some cases such as fixes to JobTracker/TaskTracker.
    However, this doesn't mean changes to the runtime (i.e. map task, reduce
    task, sort, shuffle etc.) are exempt from the rules proposed above.

  • Eli Collins at Aug 3, 2011 at 10:34 pm

    On Wed, Aug 3, 2011 at 2:14 PM, Matt Foley wrote:

    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    Here's the analysis of changes in 204 vs trunk.  We believe that ALL the
    changes in 204 are either already in trunk or do not need to be in trunk.
    Here is the list of 204 items not already in trunk, and their
    categorization.

    Please, if anyone thinks we've missed something, bring it to my attention
    and if it isn't already in trunk we will get it into trunk as expeditiously
    as possible.
    Thanks,
    --Matt
    Looks good to me Matt. Thanks for doing the analysis. I think the
    regression wrt trunk are from the original cut of the branch and don't
    need to block the 204 release, ie the trunk first discussion is IMO an
    orthogonal thread.

    Thanks,
    Eli
  • Koji Noguchi at Aug 3, 2011 at 10:59 pm
    Can we add
    * HADOOP-6942 Ability for having user's classes take precedence over the
    system classes for tasks' classpath

    It's pretty bad in a sense that it's not on trunk but 0.20.204 and CDH3 both
    have this *using different parameters*.

    Koji
    On 8/3/11 2:14 PM, "Matt Foley" wrote:


    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    Here's the analysis of changes in 204 vs trunk. We believe that ALL the
    changes in 204 are either already in trunk or do not need to be in trunk.
    Here is the list of 204 items not already in trunk, and their
    categorization.

    Please, if anyone thinks we've missed something, bring it to my attention
    and if it isn't already in trunk we will get it into trunk as expeditiously
    as possible.
    Thanks,
    --Matt

    Not needed for trunk:
    HDFS-1758. Make Web UI JSP pages thread safe. (Tanping via suresh) (Not a
    problem in trunk)
    HDFS-2057. Wait time to terminate the threads causes unit tests to take
    longer time. (Bharath Mundlapudi via suresh) (Not a problem in trunk)
    HDFS-1878. TestHDFSServerPorts unit test failure - race condition in
    FSNamesystem.close() causes NullPointerException without serious
    consequence. (mattf) (not a problem in trunk)
    HDFS-1842. Handle editlog opcode conflict with 0.20.203 during upgrade, by
    throwing an error to indicate the editlog needs to be empty. (suresh)
    (Handled by HDFS-1824)
    HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
    suite for 0.20-security-204 release. (Matt Foley) (Not a problem in trunk)
    HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
    timing issues. (mattf) (not a problem in trunk)
    HADOOP-7459. Remove jdk-1.6.0 dependency check from rpm. (omalley) (replaced
    by HDFS-2156)
    HADOOP-7398. Suppress warnings about use of HADOOP_HOME. (omalley) (not a
    problem in trunk)
    HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
    (not a problem in trunk)
    HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from trying
    to use the wrong bin directory. (omalley) (not a problem in trunk)
    HADOOP-7475. Fix hadoop-setup-single-node.sh to reflect new layout. (eyang
    via omalley) (not a problem in trunk)

    Back port from trunk:
    MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to fail
    faulty maps more aggressively. (Thomas Graves via cdouglas)
    MAPREDUCE-2479. Move distributed cache cleanup to a background task,
    backporting MAPREDUCE-1568. (Robert Joseph Evans via cdouglas)
    HDFS-2023. Backport of NPE for File.list and File.listFiles. Merged ports
    of HADOOP-7322, HDFS-1934, HADOOP-7342, and HDFS-2019. (Bharath Mundlapudi
    via mattf)
    HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
    (Thomas Graves and Tom White via cdouglas) (from HADOOP-6407)
    HADOOP-7274. Fix typos in IOUtils. (Jonathan Eagles via cdouglas) (from
    HADOOP-7057)
    HADOOP-7277. Add generation of run configurations to eclipse target.
    (Jeffrey Naisbitt and Philip Zeyliger via cdouglas) (from HADOOP-5911)
    HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
    something other than build/test. (Thomas Graves via mahadev) (from
    MAPREDUCE-2575)

    Already in trunk for MRV2:
    MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
    (Jeffrey Naisbitt via mahadev)
    MAPREDUCE-2418. Show job errors in JobHistory page. (Siddharth Seth via
    acmurthy) (Already in MR279)
    MAPREDUCE-2411. Force an exception when the queue has an invalid name or its
    ACLs are misconfigured. (Dick King via cdouglas) (Already in MR279)

    Waiting for MR279 merge:
    MAPREDUCE-2429. Validate JVM in TaskUmbilicalProtocol. (Siddharth Seth via
    acmurthy)
    MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid corner
    cases in error handling. (Siddharth Seth via acmurthy)

    MapReduce v1 exceptions:
    MAPREDUCE-2415. Distribute the user task logs on to multiple disks.
    (Bharath Mundlapudi via omalley)
    MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
    itself. (Ravi Gummadi and Jagane Sundar via omalley)
    MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
    exist". (Sherry Chen via mahadev)
    MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
    Evans via cdouglas)
    MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
    via cdouglas)
    MAPREDUCE-2443. Fix TaskAspect for TaskUmbilicalProtocol.ping(..).
    (Siddharth Seth via szetszwo)





    On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204,
    0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I
    apologize for the lack of clarity on the roadmap and timelines.

    Eric did send out a note on the motivation for sustaining releases (
    http://s.apache.org/hadoop-sustenance-releases) - which I believe is
    important to keep Hadoop installations going. However, I accept that there
    has been very poor clarity on the process for inclusion of content to the
    sustaining releases and the timelines.

    Here is my proposal for the community process for sustaining releases
    moving forward:
    # The Release Manager should clearly communicate, upfront, the expected
    timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to
    trunk, if applicable*, and to the branch-0.20-security branch. Then talk to
    the Release Manager for inclusion via the normal process.

    The RM is responsible for release content, timelines and for enforcing that
    each change should have an equivalent for trunk, as applicable, before
    inclusion.

    If this makes sense, I'll add this to the wiki to record it.

    ----

    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    thanks,
    Arun

    *Please note the exception that we have agreed that MRv2 is the way forward
    (http://s.apache.org/mr1) , and thus, MR1 patches might not be ported
    as-is to trunk in some cases such as fixes to JobTracker/TaskTracker.
    However, this doesn't mean changes to the runtime (i.e. map task, reduce
    task, sort, shuffle etc.) are exempt from the rules proposed above.

  • Arun C Murthy at Aug 3, 2011 at 11:58 pm

    On Aug 3, 2011, at 3:58 PM, Koji Noguchi wrote:

    Can we add
    * HADOOP-6942 Ability for having user's classes take precedence over the
    system classes for tasks' classpath
    Thanks Koji.

    HADOOP-6942 is in the 'MRv1 exceptions' category since MRv2 doesn't have this issue at all.
    It's pretty bad in a sense that it's not on trunk but 0.20.204 and CDH3 both
    have this *using different parameters*.
    This was originally part of the 0.20.203 release, but what/how Cloudera chooses to put in CDH3 is beyond the scope of this discussion... :)

    Arun
    Koji
    On 8/3/11 2:14 PM, "Matt Foley" wrote:


    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    Here's the analysis of changes in 204 vs trunk. We believe that ALL the
    changes in 204 are either already in trunk or do not need to be in trunk.
    Here is the list of 204 items not already in trunk, and their
    categorization.

    Please, if anyone thinks we've missed something, bring it to my attention
    and if it isn't already in trunk we will get it into trunk as expeditiously
    as possible.
    Thanks,
    --Matt

    Not needed for trunk:
    HDFS-1758. Make Web UI JSP pages thread safe. (Tanping via suresh) (Not a
    problem in trunk)
    HDFS-2057. Wait time to terminate the threads causes unit tests to take
    longer time. (Bharath Mundlapudi via suresh) (Not a problem in trunk)
    HDFS-1878. TestHDFSServerPorts unit test failure - race condition in
    FSNamesystem.close() causes NullPointerException without serious
    consequence. (mattf) (not a problem in trunk)
    HDFS-1842. Handle editlog opcode conflict with 0.20.203 during upgrade, by
    throwing an error to indicate the editlog needs to be empty. (suresh)
    (Handled by HDFS-1824)
    HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
    suite for 0.20-security-204 release. (Matt Foley) (Not a problem in trunk)
    HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
    timing issues. (mattf) (not a problem in trunk)
    HADOOP-7459. Remove jdk-1.6.0 dependency check from rpm. (omalley) (replaced
    by HDFS-2156)
    HADOOP-7398. Suppress warnings about use of HADOOP_HOME. (omalley) (not a
    problem in trunk)
    HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
    (not a problem in trunk)
    HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from trying
    to use the wrong bin directory. (omalley) (not a problem in trunk)
    HADOOP-7475. Fix hadoop-setup-single-node.sh to reflect new layout. (eyang
    via omalley) (not a problem in trunk)

    Back port from trunk:
    MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to fail
    faulty maps more aggressively. (Thomas Graves via cdouglas)
    MAPREDUCE-2479. Move distributed cache cleanup to a background task,
    backporting MAPREDUCE-1568. (Robert Joseph Evans via cdouglas)
    HDFS-2023. Backport of NPE for File.list and File.listFiles. Merged ports
    of HADOOP-7322, HDFS-1934, HADOOP-7342, and HDFS-2019. (Bharath Mundlapudi
    via mattf)
    HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
    (Thomas Graves and Tom White via cdouglas) (from HADOOP-6407)
    HADOOP-7274. Fix typos in IOUtils. (Jonathan Eagles via cdouglas) (from
    HADOOP-7057)
    HADOOP-7277. Add generation of run configurations to eclipse target.
    (Jeffrey Naisbitt and Philip Zeyliger via cdouglas) (from HADOOP-5911)
    HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
    something other than build/test. (Thomas Graves via mahadev) (from
    MAPREDUCE-2575)

    Already in trunk for MRV2:
    MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
    (Jeffrey Naisbitt via mahadev)
    MAPREDUCE-2418. Show job errors in JobHistory page. (Siddharth Seth via
    acmurthy) (Already in MR279)
    MAPREDUCE-2411. Force an exception when the queue has an invalid name or its
    ACLs are misconfigured. (Dick King via cdouglas) (Already in MR279)

    Waiting for MR279 merge:
    MAPREDUCE-2429. Validate JVM in TaskUmbilicalProtocol. (Siddharth Seth via
    acmurthy)
    MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid corner
    cases in error handling. (Siddharth Seth via acmurthy)

    MapReduce v1 exceptions:
    MAPREDUCE-2415. Distribute the user task logs on to multiple disks.
    (Bharath Mundlapudi via omalley)
    MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
    itself. (Ravi Gummadi and Jagane Sundar via omalley)
    MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
    exist". (Sherry Chen via mahadev)
    MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
    Evans via cdouglas)
    MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
    via cdouglas)
    MAPREDUCE-2443. Fix TaskAspect for TaskUmbilicalProtocol.ping(..).
    (Siddharth Seth via szetszwo)





    On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204,
    0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I
    apologize for the lack of clarity on the roadmap and timelines.

    Eric did send out a note on the motivation for sustaining releases (
    http://s.apache.org/hadoop-sustenance-releases) - which I believe is
    important to keep Hadoop installations going. However, I accept that there
    has been very poor clarity on the process for inclusion of content to the
    sustaining releases and the timelines.

    Here is my proposal for the community process for sustaining releases
    moving forward:
    # The Release Manager should clearly communicate, upfront, the expected
    timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to
    trunk, if applicable*, and to the branch-0.20-security branch. Then talk to
    the Release Manager for inclusion via the normal process.

    The RM is responsible for release content, timelines and for enforcing that
    each change should have an equivalent for trunk, as applicable, before
    inclusion.

    If this makes sense, I'll add this to the wiki to record it.

    ----

    So, how are we doing with 0.20.204 content and trunk given the above
    proposal? Very well, in fact. Matt, Suresh and I have done a detailed
    analysis (separate email), please take a look.

    thanks,
    Arun

    *Please note the exception that we have agreed that MRv2 is the way forward
    (http://s.apache.org/mr1) , and thus, MR1 patches might not be ported
    as-is to trunk in some cases such as fixes to JobTracker/TaskTracker.
    However, this doesn't mean changes to the runtime (i.e. map task, reduce
    task, sort, shuffle etc.) are exempt from the rules proposed above.

  • Arun C Murthy at Aug 3, 2011 at 9:03 pm

    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204, 0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I apologize for the lack of clarity on the roadmap and timelines.

    Eric did send out a note on the motivation for sustaining releases (http://s.apache.org/hadoop-sustenance-releases) - which I believe is important to keep Hadoop installations going. However, I accept that there has been very poor clarity on the process for inclusion of content to the sustaining releases and the timelines.

    Here is my proposal for the community process for sustaining releases moving forward:
    # The Release Manager should clearly communicate, upfront, the expected timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to trunk, if applicable*, and to the branch-0.20-security branch. Then talk to the Release Manager for inclusion via the normal process.

    The RM is responsible for release content, timelines and for enforcing that each change should have an equivalent for trunk, as applicable, before inclusion.

    If this makes sense, I'll add this to the wiki to record it.

    ----

    So, how are we doing with 0.20.204 content and trunk given the above proposal? Very well, in fact. Matt, Suresh and I have done a detailed analysis (separate email), please take a look.

    thanks,
    Arun

    *Please note the exception that we have agreed that MRv2 is the way forward (http://s.apache.org/mr1) , and thus, MR1 patches might not be ported as-is to trunk in some cases such as fixes to JobTracker/TaskTracker. However, this doesn't mean changes to the runtime (i.e. map task, reduce task, sort, shuffle etc.) are exempt from the rules proposed above.
  • Eli Collins at Aug 3, 2011 at 10:39 pm

    On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204, 0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I apologize for the lack of clarity on the roadmap and timelines.

    Eric did send out a note on the motivation for sustaining releases (http://s.apache.org/hadoop-sustenance-releases) - which I believe is important to keep Hadoop installations going. However, I accept that there has been very poor clarity on the process for inclusion of content to the sustaining releases and the timelines.

    Here is my proposal for the community process for sustaining releases moving forward:
    # The Release Manager should clearly communicate, upfront, the expected timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to trunk, if applicable*, and to the branch-0.20-security branch. Then talk to the Release Manager for inclusion via the normal process.

    The RM is responsible for release content, timelines and for enforcing that each change should have an equivalent for trunk, as applicable, before inclusion.

    If this makes sense, I'll add this to the wiki to record it.
    +1 sounds great.

    Thanks,
    Eli
  • Matt Foley at Aug 24, 2011 at 11:30 pm
    Per Arun's message below, I edited the "Sustaining Release" section of
    http://wiki.apache.org/hadoop/Roadmap
    to incorporate these items. Feedback welcome.
    --Matt
    On Wed, Aug 3, 2011 at 3:39 PM, Eli Collins wrote:
    On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy wrote:
    On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
    I'm getting confused about release roadmaps right now

    Is there somewhere that lists the (proposed) timetable for the 0.20.204,
    0.20.205, 0.22, 0.23 releases?
    Since I was among the people who started the 'security on 0.20' thread, I
    apologize for the lack of clarity on the roadmap and timelines.
    Eric did send out a note on the motivation for sustaining releases (
    http://s.apache.org/hadoop-sustenance-releases) - which I believe is
    important to keep Hadoop installations going. However, I accept that there
    has been very poor clarity on the process for inclusion of content to the
    sustaining releases and the timelines.
    Here is my proposal for the community process for sustaining releases
    moving forward:
    # The Release Manager should clearly communicate, upfront, the expected
    timeline for each upcoming release.
    # Anyone interested in contributing to the release submits a patch to
    trunk, if applicable*, and to the branch-0.20-security branch. Then talk to
    the Release Manager for inclusion via the normal process.
    The RM is responsible for release content, timelines and for enforcing
    that each change should have an equivalent for trunk, as applicable, before
    inclusion.
    If this makes sense, I'll add this to the wiki to record it.
    +1 sounds great.

    Thanks,
    Eli
  • Steve Loughran at Aug 2, 2011 at 11:28 am
    On 29/07/11 17:51, Suresh Srinivas wrote:
    I'd like to get HADOOP-7468 in, which deletes log4j.properties from the
    JAR. Someone did a patch for it yesterday
    I patched trunk but not the 20.20x branch
    This bug is not marked as a blocker. In that case I think we should pick
    this up in 205 release that is going to out soon.
  • Yu Li at Aug 8, 2011 at 3:36 pm
    Hi all,

    I've just run unit test of 0.20.204 using SUN jdk1.6.0_21, on a 64bit RHEL
    machine, and find the following cases failed:

    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestart FAILED (timeout)
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestartWithLostTracker
    FAILED
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerSafeMode FAILED
    [junit] Test org.apache.hadoop.filecache.TestMRWithDistributedCache FAILED
    [junit] Test org.apache.hadoop.filecache.TestTrackerDistributedCacheManager
    FAILED
    [junit] Test org.apache.hadoop.mapred.TestMiniMRMapRedDebugScript FAILED
    [junit] Test org.apache.hadoop.mapred.TestRecoveryManager FAILED
    [junit] Test org.apache.hadoop.mapred.TestTaskTrackerLocalization FAILED
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestartWithCS FAILED
    (timeout)
    [junit] Test org.apache.hadoop.mapred.TestRackAwareTaskPlacement FAILED

    I run each failed test case for 4 times, the above 9 cases failed for each
    round, while the last case succeeded twice and failed twice.

    I also tested 0.20.203 rc1 with the same testing environment some time
    earlier, and got almost the same result(with the failed cases attached
    below)

    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestart FAILED
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestartWithLostTracker
    FAILED
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerSafeMode FAILED
    [junit] Test org.apache.hadoop.filecache.TestMRWithDistributedCache FAILED
    [junit] Test org.apache.hadoop.filecache.TestTrackerDistributedCacheManager
    FAILED
    [junit] Test org.apache.hadoop.mapred.TestMiniMRMapRedDebugScript FAILED
    [junit] Test org.apache.hadoop.mapred.TestRecoveryManager FAILED
    [junit] Test org.apache.hadoop.mapred.TestTaskTrackerLocalization FAILED
    [junit] Test org.apache.hadoop.hdfsproxy.TestHdfsProxy FAILED

    Could somebody have a look on these failed test cases? Or tell me if I set
    anything wrong in my environment? Many thanks.

    PS: I referred to http://wiki.apache.org/hadoop/HadoopJavaVersions about
    setting the testing environment.

    --
    Best Regards,
    Li Yu
    On 2 August 2011 19:22, Steve Loughran wrote:
    On 29/07/11 17:51, Suresh Srinivas wrote:

    I'd like to get HADOOP-7468 in, which deletes log4j.properties from the
    JAR. Someone did a patch for it yesterday
    I patched trunk but not the 20.20x branch

    This bug is not marked as a blocker. In that case I think we should pick
    this up in 205 release that is going to out soon.
  • Owen O'Malley at Aug 8, 2011 at 4:27 pm

    On Aug 8, 2011, at 8:36 AM, Yu Li wrote:

    Hi all,

    I've just run unit test of 0.20.204 using SUN jdk1.6.0_21, on a 64bit RHEL
    machine, and find the following cases failed:

    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestart FAILED (timeout)
    I hit this one too. If you look at that test case, you'll see it has an @Ignore on it. For some unknown reason, when you use ant 1.8.2 junit does the wrong thing. Use ant 1.7.1 and the test cases will be properly ignored.

    -- Owen
  • Yu Li at Aug 8, 2011 at 5:04 pm
    Thanks Owen, for the quick and helpful response!

    By removing those test cases marked as "@Ignored", there're still 3 failed
    test cases for 0.20.203:

    [junit] Test org.apache.hadoop.filecache.TestMRWithDistributedCache FAILED
    [junit] Test org.apache.hadoop.filecache.TestTrackerDistributedCacheManager
    FAILED
    [junit] Test org.apache.hadoop.hdfsproxy.TestHdfsProxy FAILED

    and 3 failed, 1 unstable cases for 0.20.204:

    [junit] Test org.apache.hadoop.filecache.TestMRWithDistributedCache FAILED
    [junit] Test org.apache.hadoop.filecache.TestTrackerDistributedCacheManager
    FAILED
    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestartWithCS FAILED
    (timeout)
    [junit] Test org.apache.hadoop.mapred.TestRackAwareTaskPlacement --> FAILED
    twice, SUCCEED twice

    Has anybody met with these case failures? Thanks.


    --
    Best Regards,
    Li Yu
    On 9 August 2011 00:26, Owen O'Malley wrote:

    On Aug 8, 2011, at 8:36 AM, Yu Li wrote:

    Hi all,

    I've just run unit test of 0.20.204 using SUN jdk1.6.0_21, on a 64bit RHEL
    machine, and find the following cases failed:

    [junit] Test org.apache.hadoop.mapred.TestJobTrackerRestart FAILED
    (timeout)

    I hit this one too. If you look at that test case, you'll see it has an
    @Ignore on it. For some unknown reason, when you use ant 1.8.2 junit does
    the wrong thing. Use ant 1.7.1 and the test cases will be properly ignored.

    -- Owen
  • Milind Bhandarkar at Jul 29, 2011 at 12:30 am
    I saw @jeric14's tweet about 0.20.204 vote last night, and was about to reply, because I could not see any vote on the general@ mailing list.

    Yes, toy are right @atm, the vote should be on general@.

    - milind

    ---
    Milind Bhandarkar

    -----Original Message-----
    From: Aaron T. Myers
    Sent: Thursday, July 28, 2011 5:12 PM
    To: common-dev@hadoop.apache.org
    Subject: Re: [VOTE] Release 0.20.204.0-rc0

    Shouldn't this vote be taking place on general@, instead of common-dev@? I'm
    under the impression that that is where all votes are supposed to take
    place. Please correct me if I am wrong about that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera



    On Thu, Jul 28, 2011 at 5:02 PM, Giridharan Kesavan
    wrote:
    This issue is fixed with Eric's patch for HADOOP-7356. Since Owen is out on
    vacation Iam working on getting the release tarball.

    -Giri

    On 7/28/11 1:56 PM, "Giridharan Kesavan" wrote:

    Myself and Eric Yang are looking into this.
    -Giri
    On 7/28/11 12:04 PM, "Allen Wittenauer" wrote:

    On Jul 25, 2011, at 7:05 PM, Owen O'Malley wrote:

    I've created a release candidate for 0.20.204.0 that I would like to
    release.

    It is available at:
    http://people.apache.org/~omalley/hadoop-0.20.204.0-rc0/
    0.20.204.0 has many fixes including disk fail in place and the new rpm
    and
    deb packages. Fail in place allows the DataNode and TaskTracker to
    continue
    after a hard drive fails.

    Is it still failing to build according to Jenkins?

Related Discussions

People

Translate

site design / logo © 2022 Grokbase