On Mar 1, 2013, at 11:33 AM, Bill Graham wrote:
+1 to releasing Pig 0.11.1 when this is addressed. I should be able to help
with the release again.
+1 to releasing Pig 0.11.1 when this is addressed. I should be able to help
with the release again.
On Fri, Mar 1, 2013 at 11:25 AM, Prashant Kommireddi wrote:
Hey Guys,
I wanted to start a conversation on this again. If Kai is not looking at
PIG-3194 I can start working on it to get 0.11 compatible with 20.2. If
everyone agrees, we should roll out 0.11.1 sooner than usual and I
volunteer to help with it in anyway possible.
Any objections to getting 0.11.1 out soon after 3194 is fixed?
-Prashant
On Wed, Feb 20, 2013 at 3:34 PM, Russell Jurney <russell.jurney@gmail.com
20.2?)
environment.
branch
The
right,
of
billgraham@gmail.com>
vote
rolled
versions
at
why
Clusters.
forever.
older
in
the
support, but
really
version
the
Oozie)
question is
supports
month to
compatibility
around
time we
situation
1.0.
0.20.2
selfish
Pig
possible.
On Tue, Feb 19, 2013 at 9:51 AM, Bill Graham <billgraham@gmail.com>
w.r.t.
rc.
This
without
then
then
)
me.
give
errors'
to
0.20.2
trying
tickets
think,
through
Please
russell.jurney@gmail.com
--
*Note that I'm no longer using my Yahoo! email address. Pleaseemail me
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
datasyndrome.com
Hey Guys,
I wanted to start a conversation on this again. If Kai is not looking at
PIG-3194 I can start working on it to get 0.11 compatible with 20.2. If
everyone agrees, we should roll out 0.11.1 sooner than usual and I
volunteer to help with it in anyway possible.
Any objections to getting 0.11.1 out soon after 3194 is fixed?
-Prashant
On Wed, Feb 20, 2013 at 3:34 PM, Russell Jurney <russell.jurney@gmail.com
wrote:
I stand corrected. Cool, 0.11 is good!
On Wed, Feb 20, 2013 at 1:15 PM, Jarek Jarcec Cecho <jarcec@apache.org
committed,I stand corrected. Cool, 0.11 is good!
On Wed, Feb 20, 2013 at 1:15 PM, Jarek Jarcec Cecho <jarcec@apache.org
wrote:
Just a unrelated note: The CDH3 is more closer to Hadoop 1.x than to 0.20.
Jarcec
Just a unrelated note: The CDH3 is more closer to Hadoop 1.x than to 0.20.
Jarcec
On Wed, Feb 20, 2013 at 12:04:51PM -0800, Dmitriy Ryaboy wrote:
I agree -- this is a good release. The bugs Kai pointed out should be
fixed, but as they are not critical regressions, we can fix them in 0.11.1
(if someone wants to roll 0.11.1 the minute these fixes are
I agree -- this is a good release. The bugs Kai pointed out should be
fixed, but as they are not critical regressions, we can fix them in 0.11.1
(if someone wants to roll 0.11.1 the minute these fixes are
I
iswon't mind and will dutifully vote for the release).
I think the Hadoop 20.2 incompatibility is unfortunate but iirc this
I think the Hadoop 20.2 incompatibility is unfortunate but iirc this
fixable by setting HADOOP_USER_CLASSPATH_FIRST=true (was that in
FWIW Twitter's running CDH3 and this release works in our
At this point things that block a release are critical regressions in
performance or correctness.
D
On Wed, Feb 20, 2013 at 11:52 AM, Alan Gates <gates@hortonworks.com>
wrote:performance or correctness.
D
On Wed, Feb 20, 2013 at 11:52 AM, Alan Gates <gates@hortonworks.com>
No. Bugs like these are supposed to be found and fixed after we
from trunk (which happened several months ago in the case of 0.11).
point of RCs are to check that it's a good build, licenses are
etc.
Any bugs found this late in the game have to be seen as failures
earlier testing.
Alan.
Alan.
On Feb 20, 2013, at 11:33 AM, Russell Jurney wrote:
Isn't the point of an RC to find and fix bugs like these>
On Wed, Feb 20, 2013 at 11:31 AM, Bill Graham <
Isn't the point of an RC to find and fix bugs like these>
On Wed, Feb 20, 2013 at 11:31 AM, Bill Graham <
wrote:
Regarding Pig 11 rc2, I propose we continue with the current
as is
(which closes today EOD). Patches for 0.20.2 issues can be
into a
onatkovich@yahoo.com
issue is
issues
Pig 0.11.1 release whenever they're available and tested.
On Wed, Feb 20, 2013 at 9:24 AM, Olga Natkovich <
On Wed, Feb 20, 2013 at 9:24 AM, Olga Natkovich <
wrote:
I agree that supporting as much as we can is a good goal. The
I agree that supporting as much as we can is a good goal. The
who
is going to be testing against all these versions? We found the
under discussion because of a customer report, not because we
consistently
test against all versions. Perhaps when we decide which
consistently
test against all versions. Perhaps when we decide which
to
testingsupport
for next release we need also to agree who is going to be
and
maintaining compatibility with a particular version.
For instance since Hadoop 23 compatibility is important for us
For instance since Hadoop 23 compatibility is important for us
Yahoo
0.10 and
needwe
have been maintaining compatibility with this version for 0.9,
will do the same for 0.11 and going forward. I think we would
others
to step in and claim the versions of their interest.
Olga
________________________________
From: Kai Londenberg <kai.londenberg@googlemail.com>
To: dev@pig.apache.org
Sent: Wednesday, February 20, 2013 1:51 AM
Subject: Re: pig 0.11 candidate 2 feedback: Several problems
Hi,
I stronly agree with Jonathan here. If there are good reasons
Olga
________________________________
From: Kai Londenberg <kai.londenberg@googlemail.com>
To: dev@pig.apache.org
Sent: Wednesday, February 20, 2013 1:51 AM
Subject: Re: pig 0.11 candidate 2 feedback: Several problems
Hi,
I stronly agree with Jonathan here. If there are good reasons
you
thing.
such in
it,can't support an older version of Hadoop any more, that's one
But having to change 2 lines of code doesn't really qualify as
my point of view ;)
At least for me, pig support for 0.20.2 is essential - without
At least for me, pig support for 0.20.2 is essential - without
I
andcan't use it. If it doesn't support it, I'll have to branch pig
hack it myself, or stop using it.
I guess, there are a lot of people still running 0.20.2
I guess, there are a lot of people still running 0.20.2
If
busyyou really have lots of data stored on HDFS and a continuously
cluster, an upgrade is nothing you do "just because".
2013/2/20 Jonathan Coveney <jcoveney@gmail.com>:
2013/2/20 Jonathan Coveney <jcoveney@gmail.com>:
I agree that we shouldn't have to support old versions
That
said,
I also don't think we should be too blase about supporting
versions
where it is not odious to do so. We have a lot of competition
the
language space and the broader the versions we can support,
better
think
in-line(assuming it isn't too odious to do so). In this case, I don't
it
commons-codec
should be too hard to change ObjectSerializer so that the
code used is compatible with both versions...we could just
some
of
the Base64 code, and comment accordingly.
That said, we also should be clear about what versions we
That said, we also should be clear about what versions we
6-12
months seems short. The upgrade cycles on Hadoop are really,
long.
2013/2/20 Prashant Kommireddi <prash1784@gmail.com>
2013/2/20 Prashant Kommireddi <prash1784@gmail.com>
Agreed, that makes sense. Probably supporting older hadoop
for
a 1
or 2 pig releases before moving to a newer/stable version?
Having said that, should we use 0.11 period to communicate
Having said that, should we use 0.11 period to communicate
same
past
toto
the
community and start moving on 0.12 onwards? I know we are way
6-12
months (1-2 release) time frame with 0.20.2, but we also need
make
sure
users are aware and plan accordingly.
I'd also be interested to hear how other projects (Hive,
I'd also be interested to hear how other projects (Hive,
are
onatkovich@yahoo.com
clearlyhandling this.
-Prashant
On Tue, Feb 19, 2013 at 3:22 PM, Olga Natkovich <
-Prashant
On Tue, Feb 19, 2013 at 3:22 PM, Olga Natkovich <
wrote:
It seems that for each Pig release we need to agree and
It seems that for each Pig release we need to agree and
state
which Hadoop versions it will support. I guess the main
how
we
decide on this. Perhaps we should say that Pig no longer
older
Hadoop versions once the newer one is out for at least 6-12
make
sure it is stable. I don't think we can support old versions
indefinitely.
It is in everybody's interest to keep moving forward.
Olga
________________________________
From: Prashant Kommireddi <prash1784@gmail.com>
To: dev@pig.apache.org
Sent: Tuesday, February 19, 2013 10:57 AM
Subject: Re: pig 0.11 candidate 2 feedback: Several problems
What do you guys feel about the JIRA to do with 0.20.2
indefinitely.
It is in everybody's interest to keep moving forward.
Olga
________________________________
From: Prashant Kommireddi <prash1784@gmail.com>
To: dev@pig.apache.org
Sent: Tuesday, February 19, 2013 10:57 AM
Subject: Re: pig 0.11 candidate 2 feedback: Several problems
What do you guys feel about the JIRA to do with 0.20.2
(PIG-3194)? I am interested in discussing the strategy
backward
compatibility as this is something that would haunt us each
move
to
the next hadoop version. For eg, we might be in a similar
while
moving to Hadoop 2.0, when some of the stuff might break for
I feel it would be good to get this JIRA fix in for 0.11, as
users
might be caught unaware. Of course, I must admit there is
interest
here and it's probably easier for us to have a workaround on
rather
than upgrade hadoop in all our production DCs.
-Prashant
On Tue, Feb 19, 2013 at 9:54 AM, Russell Jurney <
russell.jurney@gmail.com-Prashant
On Tue, Feb 19, 2013 at 9:54 AM, Russell Jurney <
wrote:
I think someone should step up and fix the easy ones, if
I think someone should step up and fix the easy ones, if
On Tue, Feb 19, 2013 at 9:51 AM, Bill Graham <
wrote:
Thanks Kai for reporting these.
What do people think about the severity of these issues
What do people think about the severity of these issues
Pig
11?
I
see a few possible options:
1. We include some or all of these patches in a new Pig 11
1. We include some or all of these patches in a new Pig 11
We'd
want
to
make sure that they don't destabilize the current branch.
approach
makes sense if we think Pig 11 wouldn't be a good release
one
or
more of these included.
2. We continue with the Pig 11 release without these, but
2. We continue with the Pig 11 release without these, but
include
one
or more in a 0.11.1 release.
3. We continue with the Pig 11 release without these, but
3. We continue with the Pig 11 release without these, but
include
them
in a 0.12 release.
Jon has a patch for the MAP issue
(PIG-3144<https://issues.apache.org/jira/browse/PIG-3144
Jon has a patch for the MAP issue
(PIG-3144<https://issues.apache.org/jira/browse/PIG-3144
ready, which seems like the most pressing of the three to
thanks,
Bill
On Mon, Feb 18, 2013 at 2:27 AM, Kai Londenberg <
kai.londenberg@googlemail.com> wrote:
Bill
On Mon, Feb 18, 2013 at 2:27 AM, Kai Londenberg <
kai.londenberg@googlemail.com> wrote:
Hi,
I just subscribed to the dev mailing list in order to
I just subscribed to the dev mailing list in order to
you
'Erroneoussome
candidate
feedback on pig 0.11 candidate 2.
The following three issues are currently present in 0.11
The following three issues are currently present in 0.11
map
entry
alias resolution leading to "Duplicate schema alias"
to
ObjectSerializer.java break compatibility with Hadoop
Condition in
PhysicalOperator leads to ExecException "Error while
to
get
next result in POStream"
The last two of these are easily solveable (see the
The last two of these are easily solveable (see the
for
details on that). The first one is a bit trickier I
but
at
least there is a workaround for it (pass Map fields
an
STREAMUDF)
severe,In my personal opinion, each of these problems is pretty
but
opinions about the importance of the MAP Datatype and
Operator,
--
*Note that I'm no longer using my Yahoo! email address.
as well as Hadoop 0.20.2 compatibility might differ.
so far ..
Kai Londenberg
so far ..
Kai Londenberg
--
*Note that I'm no longer using my Yahoo! email address.
email
me
at
--
Russell Jurney twitter.com/rjurney
billgraham@gmail.com going forward.*
--
Russell Jurney twitter.com/rjurney
datasyndrome.com
--
*Note that I'm no longer using my Yahoo! email address. Please
at
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.comdatasyndrome.com
billgraham@gmail.com going forward.*
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
datasyndrome.com
