FAQ
Guys,

I am in a very big big confusion. Please.. I really need your feedbacks and
suggestions..

The scenario is like this...

I set up *Hadoop 0.20.2 cluster* of *12 nodes*..

Now I set up* Hbase 0.90.3* *12 node cluster* on top of it.

But after all that experimenting and struggling.. I read the following
SHOCKING line on my Hbase web UI

---* " You are currently running the HMaster without HDFS append support
enabled. This may result in data loss. Please see the HBase wiki for
details. "*

And when I searched more about it.. I found Michael G. Nolls article..
saying that *Hadoop 0.20.2 and Hbase 0.90.2 are not compatible*.

Is *Hadoop 0.20.2 also not compatible with Hbase 0.90.3 ???*

So does it means I have to re-install hadoop 0.20. append if I want to use
Hbase.

I did a lot of struggle to reach upto this stage.. do I have to do all of it
again.. ??

Is there any other work around solution.. of not re-installing everything
again ??

Please help..!!! :-(

Thanks,
Praveenesh

Search Discussions

  • Andrew Purtell at Jun 3, 2011 at 11:47 am
    Is *Hadoop 0.20.2  also not compatible with Hbase 0.90.3 ???*
    In a strict sense they are, but without append support HBase cannot guarantee that the last block of write ahead logs are synced to disk, so in some failure cases edits will be lost. With append support then the "hole" of these failure cases is closed. Also append branch adds a change to lease recovery that allows the HBase Master to take ownership of regionserver logs in order to split them quickly. (Without this patch you may be waiting 10 minutes for lease recovery...) So these differences are clearly important for durability and fault recovery in a realistic time frame.

    A full reinstallation is not necessary.

    You can take the Hadoop core jar packaged in lib/ of HBase 0.90.3 and replace every Hadoop core jar on your cluster with them.

    OR

    You can compile 0.20-append and just replace the Hadoop core jar everywhere with the result, including the one in the HBase lib/.

    - Andy

    --- On Fri, 6/3/11, praveenesh kumar wrote:
    From: praveenesh kumar <praveenesh@gmail.com>
    Subject: Does Hadoop 0.20.2 and HBase 0.90.3 compatible ??
    To: common-user@hadoop.apache.org, user@hbase.apache.org
    Date: Friday, June 3, 2011, 3:37 AM
    Guys,

    I am in a very big big confusion. Please.. I really need
    your feedbacks and
    suggestions..

    The scenario is like this...

    I set up *Hadoop 0.20.2 cluster* of *12 nodes*..

    Now I set up* Hbase 0.90.3*  *12 node cluster* on top
    of it.

    But after all that experimenting and struggling.. I read
    the following SHOCKING line on my Hbase web UI

    ---* " You are currently running the HMaster without HDFS append support
    enabled. This may result in data loss. Please see the HBase wiki for
    details. "*

    And when I searched more about it.. I found Michael G. Nolls article..
    saying that *Hadoop 0.20.2 and Hbase 0.90.2 are not compatible*.

    Is *Hadoop 0.20.2  also not compatible with Hbase 0.90.3 ???*

    So does  it means I have to re-install hadoop 0.20.
    append if I want to use Hbase.

    I did a lot of struggle to reach upto this stage.. do I
    have to do all of it again.. ??

    Is there any other work around solution.. of not
    re-installing everything again ??

    Please help..!!!  :-(

    Thanks,
    Praveenesh
  • Brian Bockelman at Jun 3, 2011 at 12:23 pm
    A meta-comment here (triggered by Praveenesh's comments about not wanting to re-install):

    If you have to install the same software on more than one piece of software, you really ought to automate it. In the long run, you'll save more time if you can instantly roll out changes than manually doing it. In all likelihood, if you're going to be working with a piece of software (Hadoop-based or not!), you'll re-install it a few times.

    The install of HDFS should take roughly the same amount of time on 2, 20, or 200 nodes.

    Brian
    On Jun 3, 2011, at 6:47 AM, Andrew Purtell wrote:

    Is *Hadoop 0.20.2 also not compatible with Hbase 0.90.3 ???*
    In a strict sense they are, but without append support HBase cannot guarantee that the last block of write ahead logs are synced to disk, so in some failure cases edits will be lost. With append support then the "hole" of these failure cases is closed. Also append branch adds a change to lease recovery that allows the HBase Master to take ownership of regionserver logs in order to split them quickly. (Without this patch you may be waiting 10 minutes for lease recovery...) So these differences are clearly important for durability and fault recovery in a realistic time frame.

    A full reinstallation is not necessary.

    You can take the Hadoop core jar packaged in lib/ of HBase 0.90.3 and replace every Hadoop core jar on your cluster with them.

    OR

    You can compile 0.20-append and just replace the Hadoop core jar everywhere with the result, including the one in the HBase lib/.

    - Andy

    --- On Fri, 6/3/11, praveenesh kumar wrote:
    From: praveenesh kumar <praveenesh@gmail.com>
    Subject: Does Hadoop 0.20.2 and HBase 0.90.3 compatible ??
    To: common-user@hadoop.apache.org, user@hbase.apache.org
    Date: Friday, June 3, 2011, 3:37 AM
    Guys,

    I am in a very big big confusion. Please.. I really need
    your feedbacks and
    suggestions..

    The scenario is like this...

    I set up *Hadoop 0.20.2 cluster* of *12 nodes*..

    Now I set up* Hbase 0.90.3* *12 node cluster* on top
    of it.

    But after all that experimenting and struggling.. I read
    the following SHOCKING line on my Hbase web UI

    ---* " You are currently running the HMaster without HDFS append support
    enabled. This may result in data loss. Please see the HBase wiki for
    details. "*

    And when I searched more about it.. I found Michael G. Nolls article..
    saying that *Hadoop 0.20.2 and Hbase 0.90.2 are not compatible*.

    Is *Hadoop 0.20.2 also not compatible with Hbase 0.90.3 ???*

    So does it means I have to re-install hadoop 0.20.
    append if I want to use Hbase.

    I did a lot of struggle to reach upto this stage.. do I
    have to do all of it again.. ??

    Is there any other work around solution.. of not
    re-installing everything again ??

    Please help..!!! :-(

    Thanks,
    Praveenesh
  • Cosmin Lehene at Jun 3, 2011 at 2:48 pm
    Also I think we should make it more clear (on the front page) that HBase requires the append branch and provide a link to that as well.

    Cosmin
    On Jun 3, 2011, at 3:23 PM, Brian Bockelman wrote:

    A meta-comment here (triggered by Praveenesh's comments about not wanting to re-install):

    If you have to install the same software on more than one piece of software, you really ought to automate it. In the long run, you'll save more time if you can instantly roll out changes than manually doing it. In all likelihood, if you're going to be working with a piece of software (Hadoop-based or not!), you'll re-install it a few times.

    The install of HDFS should take roughly the same amount of time on 2, 20, or 200 nodes.

    Brian
    On Jun 3, 2011, at 6:47 AM, Andrew Purtell wrote:

    Is *Hadoop 0.20.2 also not compatible with Hbase 0.90.3 ???*
    In a strict sense they are, but without append support HBase cannot guarantee that the last block of write ahead logs are synced to disk, so in some failure cases edits will be lost. With append support then the "hole" of these failure cases is closed. Also append branch adds a change to lease recovery that allows the HBase Master to take ownership of regionserver logs in order to split them quickly. (Without this patch you may be waiting 10 minutes for lease recovery...) So these differences are clearly important for durability and fault recovery in a realistic time frame.

    A full reinstallation is not necessary.

    You can take the Hadoop core jar packaged in lib/ of HBase 0.90.3 and replace every Hadoop core jar on your cluster with them.

    OR

    You can compile 0.20-append and just replace the Hadoop core jar everywhere with the result, including the one in the HBase lib/.

    - Andy

    --- On Fri, 6/3/11, praveenesh kumar wrote:
    From: praveenesh kumar <praveenesh@gmail.com>
    Subject: Does Hadoop 0.20.2 and HBase 0.90.3 compatible ??
    To: common-user@hadoop.apache.org, user@hbase.apache.org
    Date: Friday, June 3, 2011, 3:37 AM
    Guys,

    I am in a very big big confusion. Please.. I really need
    your feedbacks and
    suggestions..

    The scenario is like this...

    I set up *Hadoop 0.20.2 cluster* of *12 nodes*..

    Now I set up* Hbase 0.90.3* *12 node cluster* on top
    of it.

    But after all that experimenting and struggling.. I read
    the following SHOCKING line on my Hbase web UI

    ---* " You are currently running the HMaster without HDFS append support
    enabled. This may result in data loss. Please see the HBase wiki for
    details. "*

    And when I searched more about it.. I found Michael G. Nolls article..
    saying that *Hadoop 0.20.2 and Hbase 0.90.2 are not compatible*.

    Is *Hadoop 0.20.2 also not compatible with Hbase 0.90.3 ???*

    So does it means I have to re-install hadoop 0.20.
    append if I want to use Hbase.

    I did a lot of struggle to reach upto this stage.. do I
    have to do all of it again.. ??

    Is there any other work around solution.. of not
    re-installing everything again ??

    Please help..!!! :-(

    Thanks,
    Praveenesh

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 3, '11 at 10:38a
activeJun 3, '11 at 2:48p
posts4
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase