FAQ
Hi All,

I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
I need more help regarding this Avatar namenode configuration.


1. Which IP will be given to the Datanoe fs.default.name ?
2. Is there any good method other than avatar available for Backup namenode ?



Regards,

Shanmuganathan

Search Discussions

  • Shanmuganathan.r at Aug 11, 2011 at 6:46 pm
    Hi All,

    I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
    I need more help regarding this Avatar namenode configuration.


    1. Which IP will be given to the Datanoe fs.default.name ?
    2. Is there any good method other than avatar available for Backup namenode ?



    Regards,

    Shanmuganathan
  • Prashant at Aug 16, 2011 at 4:45 am
    Namenode HA is still not available in open source. Please see
    https://issues.apache.org/jira/browse/HDFS-1623
    <https://issues.apache.org/jira/browse/HDFS-1623>

    -prashant
    On 08/11/2011 11:49 PM, shanmuganathan.r wrote:
    Hi All,

    I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
    I need more help regarding this Avatar namenode configuration.


    1. Which IP will be given to the Datanoe fs.default.name ?
    2. Is there any good method other than avatar available for Backup namenode ?



    Regards,

    Shanmuganathan

  • Jack Levin at Aug 19, 2011 at 10:31 am
    I don't think there is anyone except Facebook actually uses it. Their
    case is special, as they have millions and millions of files in HDFS.

    -Jack

    On Thu, Aug 11, 2011 at 11:19 AM, shanmuganathan.r
    wrote:
    Hi All,

    I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
    I need more help regarding this Avatar namenode configuration.


    1. Which IP will be given to the Datanoe fs.default.name ?
    2. Is there any good method other than avatar available for Backup namenode ?



    Regards,

    Shanmuganathan

  • Ryan Rawson at Aug 19, 2011 at 4:58 am
    There are a few problems for Avatar node which would prevent me from
    ever using it:

    - assumption of highly available NFS, this would typically mean
    specialized hardware
    - failover time is potentially lengthy (article says 60 seconds), and
    HBase regionservers might fail

    It's an interesting hack, I hear it was only 800 loc, but I'm not sure
    if you are not Dhruba you can run it.
    On Thu, Aug 18, 2011 at 9:49 PM, Jack Levin wrote:
    I don't think there is anyone except Facebook actually uses it.  Their
    case is special, as they have millions and millions of files in HDFS.

    -Jack

    On Thu, Aug 11, 2011 at 11:19 AM, shanmuganathan.r
    wrote:
    Hi All,

    I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
    I need more help regarding this Avatar namenode configuration.


    1. Which IP will be given to the Datanoe fs.default.name ?
    2. Is there any good method other than avatar available for Backup namenode ?



    Regards,

    Shanmuganathan

  • Lars hofhansl at Aug 20, 2011 at 1:36 pm
    I was thinking the same.

    In an ideal world, I guess, the namenodes would be quorum based.
    Clients would be aware of all the namenodes and fire updates in parallel to all namenodes, and updates not return until N namenodes confirmed the update.


    To make it easier one could initially require that all namenodes need to receive the update and only one namenode would serve lookups.

    When namenodes come up they somehow need to synchronize with each other. (waving hands here)


    When a namenode fails there would still be some manual intervention to remove it from the quorum, or to switch to another one if this was the main namenode.


    This would put twice the traffic on the network (w.r.t. to namenode traffic), slow the clients to the speed to the slowest namenode, etc. etc.

    And nothing is ever simple in distributed computing; there will be a 1000 corner cases to consider.
    The NFS approach will at least work, and it is easier to reason about the state the system is in.


    -- Lars



    ________________________________
    From: Ryan Rawson <ryanobjc@gmail.com>
    To: user@hbase.apache.org
    Cc: hadoop-user@lucene.apache.org
    Sent: Thursday, August 18, 2011 9:57 PM
    Subject: Re: Avatar namenode?

    There are a few problems for Avatar node which would prevent me from
    ever using it:

    - assumption of highly available NFS, this would typically mean
    specialized hardware
    - failover time is potentially lengthy (article says 60 seconds), and
    HBase regionservers might fail

    It's an interesting hack, I hear it was only 800 loc, but I'm not sure
    if you are not Dhruba you can run it.
    On Thu, Aug 18, 2011 at 9:49 PM, Jack Levin wrote:
    I don't think there is anyone except Facebook actually uses it.  Their
    case is special, as they have millions and millions of files in HDFS.

    -Jack

    On Thu, Aug 11, 2011 at 11:19 AM, shanmuganathan.r
    wrote:
    Hi All,

    I am running the HBase distributed mode in seven node cluster with backup master. The HBase is running properly in the backup master environment. I want to run this HBase on top of the High Availability Hadoop. I saw about Avatar node in the following link http://hadoopblog.blogspot.com/2010/02/hadoop-namenode-high-availability.html .
    I need more help regarding this Avatar namenode configuration.


    1. Which IP will be given to the Datanoe fs.default.name ?
    2. Is there any good method other than avatar available for Backup namenode ?



    Regards,

    Shanmuganathan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 11, '11 at 6:19p
activeAug 20, '11 at 1:36p
posts6
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase