[HBase-dev] [poll] Does anyone run or test against hadoop 0.21, 0.22, 0.23 under HBase 0.92.0+/0.94.0?

We use hadoop 0.22

Cheers

We work on hadoop 2.0.

Regards
Ram

Works for me, 1.0 and 2.0 only. Not 3.0.

Jon, what versions of HBase are you asking about? A while back we
said that HBase 0.96.0 would be the first release that could assume
for a minimum a 1.0.x API [1]. Previous releases we postulated
(though probably haven't tested much) could be run on versions as old
as branch-0.20-append. We might want to reexamine this "decision" if
we we are talking HBase 0.94 (or even HBase 0.92)?

+1 on hadoop 1.0 and 2.0 versions/APIs only. I'd be fine w/ a 3.0 snapshot.

Any more versions than this and we'll be spending all dev effort on a
shim layer that looks like the Pompidou Center to make the various
HBase versions run (slowly) over all 57 varieties of Hadoop.

St.Ack

1. http://search-hadoop.com/m/5qFYe24EsUb2/hbase+0.96+hadoop+1.0&subj=Re+DISCUSS+Have+hbase+require+at+least+hadoop+1+0+0+in+hbase+0+96+0+

inline
0.92.0+/0.94.0+. I should have also included 0.20-append in the list of
hadoops.

I don't think this affects the decision you refer to. I'm not trying to age
off older versions -- I'm trying find out from this discussion which
"newer" versions we should have HBase work/compile against, and if we can
just say 1.x and 2.x and not include the 0.21, 0.22, and now 0.23 branches.


We've found out along the way that while different Hadoop versions are
"source-compatible" (no source change but requires recompile), they are not
"binary-compatible" (one hbase jar for all hadoop versions).

As examples, an hbase compiled against a hadoop 1.0 hdfs doesn't work ontop
of a hadoop 2.0 hdfs, and that a hbase compiled against hadoop 2.0 hdfs
doesn't work on a hadoop 1.0 cluster. To add to the mix, today we learned
that an hbase compiled against a hadoop 23 doesn't work on a hadoop 2.0
hdfs cluster.

When you say HBase 0.92/0.94 above, do you mean the next releases from
the 0.92 and 0.94 branches (0.92.2 and 0.94.2?) would be tested
against hadoop 1.x and 2.x only (If the result of this discussion is a
decision that hadoop 1.0 and 2.0 are all we support)?

St.Ack

version in 0.96 really means that we could remove shim code currently
present to support the older 0.20-append branch then (hflush vs hsync,
etc). The fact that our pom doesn't contain the 0.20-append branch on
0.92+ basically says to me that we aren't really testing against that at
all these days.

Today we only really run unit tests on hadoop 1.0 -- we have a hadoop 23
compile check that is done and if that build fails on jenkins no alarms are
raised (that said, the builds against 1.0.0 are better but still in pretty
rough shape). Once we fix the last of the hadoop 2.0 unit tests
uptream I'd like to turn on the hudson builds to build and test trunk
against hadoop 2.0. As it stands today, trunk needs HBASE-5876 -- which
needs to be renamed to be hadoop 2.0; 0.94 would needs
+HBASE-5966,HBASE-5985 which are already on trunk, and 0.92 need 5966, 5985
and two I believe 2 others but does not need HBASE-5876.

Jon.

Let that be our answer going forward; customization's go into the
hadoop compatibility modules in 0.96 forward.
Agreed. But if someone showed up w/ a patch to fix a bug in
branch-0.20-append, we'd take it on.
Thanks for keeping an account Jon. Lets see how this discussion
finishes up then write up how we go forward.

St.Ack