[Hadoop-common-user] Does Hadoop 0.20.2 and HBase 0.90.3 compatible ??

From: praveenesh kumar <[email protected]>
Subject: Does Hadoop 0.20.2 and HBase 0.90.3 compatible ??
To: [email protected], [email protected]
Date: Friday, June 3, 2011, 3:37 AM
Guys,

I am in a very big big confusion. Please.. I really need
your feedbacks and
suggestions..

The scenario is like this...

I set up *Hadoop 0.20.2 cluster* of *12 nodes*..

Now I set up* Hbase 0.90.3*  *12 node cluster* on top
of it.

But after all that experimenting and struggling.. I read
the following SHOCKING line on my Hbase web UI

---* " You are currently running the HMaster without HDFS append support
enabled. This may result in data loss. Please see the HBase wiki for
details. "*

And when I searched more about it.. I found Michael G. Nolls article..
saying that *Hadoop 0.20.2 and Hbase 0.90.2 are not compatible*.

Is *Hadoop 0.20.2  also not compatible with Hbase 0.90.3 ???*

So does  it means I have to re-install hadoop 0.20.
append if I want to use Hbase.

I did a lot of struggle to reach upto this stage.. do I
have to do all of it again.. ??

Is there any other work around solution.. of not
re-installing everything again ??

Please help..!!!  :-(

Thanks,
Praveenesh

In a strict sense they are, but without append support HBase cannot guarantee that the last block of write ahead logs are synced to disk, so in some failure cases edits will be lost. With append support then the "hole" of these failure cases is closed. Also append branch adds a change to lease recovery that allows the HBase Master to take ownership of regionserver logs in order to split them quickly. (Without this patch you may be waiting 10 minutes for lease recovery...) So these differences are clearly important for durability and fault recovery in a realistic time frame.

A full reinstallation is not necessary.

You can take the Hadoop core jar packaged in lib/ of HBase 0.90.3 and replace every Hadoop core jar on your cluster with them.

OR

You can compile 0.20-append and just replace the Hadoop core jar everywhere with the result, including the one in the HBase lib/.

- Andy

--- On Fri, 6/3/11, praveenesh kumar wrote:

On Jun 3, 2011, at 3:23 PM, Brian Bockelman wrote:

A meta-comment here (triggered by Praveenesh's comments about not wanting to re-install):

If you have to install the same software on more than one piece of software, you really ought to automate it. In the long run, you'll save more time if you can instantly roll out changes than manually doing it. In all likelihood, if you're going to be working with a piece of software (Hadoop-based or not!), you'll re-install it a few times.

The install of HDFS should take roughly the same amount of time on 2, 20, or 200 nodes.

Brian