[Hadoop-common-dev] Fwd: [RESULT] - [VOTE] Rename hadoop branches post hadoop-1.x

Done, all clear; I've also created jira revisions. Please let me know if you find any issues.

thanks,
Arun

Hey Arun,
Thanks a lot for making these changes. Two questions:

- Now that we've renamed branch-0.23 to branch-2, and since there is as yet
no branch-0.23.3, what should be done with JIRAs marked with the 0.23.3
version? Perhaps all of these should be updated to 2.0.0?

- We still have the JIRA version 0.24.0, which is presumably still
representative of trunk. Given that we will likely never release an 0.24.0,
should we rename this version in JIRA as well?

I've created a 3.0.0 version in jira too. I'll update all jiras to point to that instead of 0.24.0.

thanks,
Arun

Is trunk/24 definitely 3.0.0? Given that we don't have any major new
features targeted at it (but not targeted at 2.x) yet, I've been
thinking of it more like 2.1. I also think, given we have PB in place
in both branches, it's likely we can maintain client compatibility
between 23 and 24 in the absence of anything major coming down the
pipe.

Proposal: is it possible to call the JIRA fixVersion "trunk", and then
when we branch off trunk to make a release, rename it at that point to
"2.1" or "3.0" or whatever it gets called?

Todd

I like this idea. Just to be clear, I think the exact workflow would be:

1. Set the version fields to "trunk" if you're not committing the JIRA to
any current versioned branch.
2. When a new release branch is made off of trunk, rename the "trunk" JIRA
version to whatever the appropriate version number is.
3. At the same time as (2), create a new JIRA version also called "trunk".
4. Go to 1.

Is this what you were thinking, Todd?

Yep, that's right.

-Todd

I disagree. Trunk should become branch-3 once someone wants to start
stabilizing it. Arun is going to need the minor versions for when he adds
features.

X.Y.Z

Z = bug fixes
Y = minor release (compatible, adds features)
X = major release (incompatible)

So from branch-2 will come branch-2.0 with tags for 2.0.0, 2.0.1. New
features will go into branch-2, which will become branch-2.1, branch-2.2,
and so on.

-- Owen

But new features also go to trunk. And if none of our new features are
incompatible, why do we anticipate that trunk is 3.0?

Basically trunk is almost identical to branch-2 right now... I guess
my proposal is basically to not bother having a trunk separate from a
branch-2, and use a branch-2.0 for continued stabilization of what we
currently call branch-23. When someone has something incompatible to
propose, then we can bother splitting it off..

Todd

On Wed, Mar 28, 2012 at 12:32 PM, Todd Lipcon wrote:

But new features also go to trunk. And if none of our new features are
Let's imagine that we already had a 2.0.0 release. Now we want to add
features like HA. The only place to put that is in 2.1.0. On the other
hand, you don't want to pull *ALL* of the changes from trunk. That is way
too much scope. So the RM of the 2 branch needs to make the call of what
should be 2.1 vs 3.0.

-- Owen

Hi Owen,
I don't think anyone disagrees with this line of reasoning. It's certainly
up to the RM what gets included in branch-2 and hence what gets put up for
release votes under the "2.y.z" version numbers. I don't think Todd was
suggesting we rename the JIRA version "0.24.0" to "2.1.0".

But, the question still remains of how to refer to the branch "trunk" in
JIRA. I don't think it should be called 3.0.0, as that's not necessarily
the next release that will come off of it, and using a version number for
trunk that changes from time to time has other downsides as I described in
my response to Arun. Given this, do you object to renaming the JIRA fix
version that refers to the branch trunk to "trunk" ?

I thought these were community decisions, not RM decisions, no?

http://s.apache.org/rm

Doug

What to release is the RM's decision and then voted on by the community.
We tried voting on which features to include and it led to no releases for
two years. I think our users are better served by having good usable
releases.

-- Owen

Roy was pointing out that there is an RM for a given release, but
there is not a RM for a branch. The RM does not decide what goes into
branch-2, that's happening on a patch by patch basis, and the RM
decides what to release based on what branch they pick and what they
merge to it. So far it has been working well. Kudos to Arun.

Thanks,
Eli

Essentially 'trunk' is where incompatible changes *may* be committed (in future). We should allow for that.

I'm ambivalent about the 'version string' for trunk being either trunk-SNAPSHOT or 3.0.0-SNAPSHOT.

On on hand, we've historically bumped the version number for trunk (i.e. 3.0.0-SNAPSHOT). This has the nice property that when we do cut a release branch off trunk we don't have to scramble to change fix-versions on all the jiras committed to trunk from 'trunk' to X.0.0. Since I've done my fair share of jira manipulation for releases as the RM, as recently as last night, it's nice to not force the burden on the RM for branch-3.

OTOH, having a constant trunk-SNAPSHOT version string helps devs working on trunk since they don't have to deal with version bumps on trunk (albeit, once in a while). (Credit to Scott Carey for this idea.)

Given the above I'd stick with 3.0.0 since it means lesser confusion and lesser work for the RM on future major releases.

On a related note: as I noted last night, I'd again urge committers to not set the 'fix-version' to trunk's version if it's committed to other branches (branch-1, branch-2 etc.)

thanks,
Arun

I don't think you'd have to change all the JIRAs. You'd just have to change
the name of the "trunk" JIRA version to whatever the right number is, and
then create a new version in JIRA also named "trunk." This would serve the
same purpose, without having to update any individual JIRAs.

This is a nice benefit of Todd's (and Scott's?) proposal. Whenever we do
change the trunk version number, I have to update a bunch of scripts,
environment files, and configuration XML files. Not the end of the world,
but annoying nonetheless.

I'd also add to this benefit that users who are new to the project will
more easily be able to determine what version to put for a JIRA they want
to get committed to trunk. I've seen plenty of users who are confused by
having to set "0.24.0" as the version indicating trunk.

There's also a nice symmetry in having the branch in svn/git (named trunk)
have the same name as the JIRA version indicator.

I honestly believe that this scheme is more confusing for devs and users,
and almost no different for RMs given what I described above with JIRA
version renaming. But, I don't feel super strongly about it. If this makes
sense to you, then I'll stop pushing.

Ah, I didn't know that, thanks for the tip. That alleviates my concerns.

Arun

My proposal was from a few months back related to the naming of versions
in maven.
It boils down to 'laziness is a virtue' + 'the maven version should match
the branch'.
Pick a name for a branch when you actually branch, not months before.
'trunk' in svn == 'trunk-SNAPSHOT' in maven == 'trunk' in JIRA.

The creation of a branch should avoid impacting the place branched from
(i.e. cause work for people on trunk because of a branch, or cause work
for a branch due to a tag, etc).
If the version in 'branch/2.0.x' has maven version '2.0.x-SNAPSHOT' and
JIRA tag '2.0.0' then tagging 2.0.0 has the following steps:

$> svn cp branch/2.0.x tags/2.0.0
$> cd tags/2.0.0
$> mvn versoins:set -DnewVersion=2.0.0
$> svn diff (spot check pom changes that versions:set did)
$> mvn versions:commit
$> svn commit

The result is a new tag with the version set, and no changes to the branch
that was tagged from.
When the decision is made to have a 2.0.1, a jira tag can be made (or a
2.0.x tag renamed to 2.0.1, and a new 2.0.x tag made as Todd suggests).
Being lazy here helps because maybe instead after some development on the
2.0.x branch it is decided to branch 2.1.x from it.

When the decision to branch 2.1 occurs, a new branch 'branch/2.1.x' is
made in svn, its version becomes '2.1.x-SNAPSHOT' and a JIRA tag for
'2.1.x' is made or renamed from an old one. The old 2.0.x branch is
unchanged (and available for more minor bugfix releases).

Once trunk or a branch is set up for build scripts or hudson, etc, it
works without modification no matter how many times a branch or tag occurs
off of it.

Thanks a lot, Scott, for bringing the discussion back to what to call
"trunk" in JIRA and elsewhere. The proposal you describe to me makes a lot
of sense.

Owen, does this (and the JIRA proposal) make sense to you?

Arun,

I am even more confused now than I was before:

Here you say:


Which one is it ?

Do you expect that "new features" will always remain compatible ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

Hi Milind,
What I believe Arun is alluding to here is that we expect for compatibility
to be maintained for the lifetime of a major release branch.

These two statements aren't mutually exclusive. We require that all new
features go to trunk first so as to ensure that future releases are
supersets of the functionality of previous releases, except in the case of
explicit deprecation. Only once it's committed to trunk may it be
back-ported to an earlier branch.

Not necessarily, but only if a feature is compatible may it be back-ported
to major release branches.

Thanks ATM.

I guess the "*may*" emphasis confused me.

Just to get some more clarity:

What would be guideline for a new feature, such as
https://issues.apache.org/jira/browse/MAPREDUCE-4049, which maintains
compatibility for 1.x, but is not relevant to trunk, because the codebases
have completely diverged, so cannot be committed to trunk ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

Are you sure this isn't relevant to trunk? The "target versions" field of
that JIRA lists both 1.0.x and 0.24.0 (trunk.) In the latest comment on
that JIRA, the author appears to intend to do this work for both trunk and
1.0:

"I want to have the described plugin-ability (desired with same interface)
for all future versions of Hadoop (as mentioned in the Target Version/s
field). <snip> On the first phase, I am focusing on the existing 1.0 branch
as I know it. In parallel, I'll try to learn what exists in 0.23"

To my knowledge, shuffle is already pluggable in 0.23 onwards, as long as
it is used only by mapreduce framework.

That's why Avner says : "In parallel, I'll try to *learn what exists* in
0.23". (Emphasize my own.)

That's why I was wondering about the insistence of committing to trunk
first.

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

If that's the case then there doesn't seem to be any question here. The
feature is in trunk, and an implementation could be done for an older
release branch that would be compatible with that branch. Sure, the code to
implement the feature is quite different between the two branches, but
trunk will remain a superset of the functionality of the past release, so
no issue.

Great !

Thanks @atm,

- milind