FAQ
Dears,

Does anyone know HBase 0.90.6(CDH3U4) has limitation about the total
regions per region server? I cannot find any setting in HBse's
configuration file!? or I missed something?

And when I try in my (CDH3U4) 6 nodes small cluster, the average
number of region is 6500 per region server, when one node crash yesterday,
there were 2 of the alive region servers's region become about 9000, then
these two nodes become very slow then dead. Afer all, whole cluster down
and the cluster cannot restart.

So, my questions are:
1. What's the maximum number of regions per region server ?
2. In this incident, what's the best practice to recover the dead cluster?
(except add more nodes, because this need more time)

Best Regards.
James Chang

--

Search Discussions

  • Harsh J at Jan 28, 2013 at 5:24 am
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD
    On Mon, Jan 28, 2013 at 10:32 AM, James Chang wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about the total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the average
    number of region is 6500 per region server, when one node crash yesterday,
    there were 2 of the alive region servers's region become about 9000, then
    these two nodes become very slow then dead. Afer all, whole cluster down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --
  • James Chang at Jan 28, 2013 at 6:25 am
    Hi Hash,

    Thanks your information, the thread you mentioned only answer the
    first question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD
    On Mon, Jan 28, 2013 at 10:32 AM, James Chang wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about the total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about 9000, then
    these two nodes become very slow then dead. Afer all, whole cluster down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --


    --
  • Harsh J at Jan 28, 2013 at 6:32 am
    Hi,

    It does also talk of practical limitations - the more regions you
    grow, the more the pressure on the RS increases (especially in
    memory). May I ask why you have over 9k regions in a single
    region-server? Sounds rather inefficient to me if it is all for a few
    tables, cause a fatter region would have much less overhead while
    giving an equivalent performance in return.
    On Mon, Jan 28, 2013 at 11:54 AM, James Chang wrote:
    Hi Hash,

    Thanks your information, the thread you mentioned only answer the first
    question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD

    On Mon, Jan 28, 2013 at 10:32 AM, James Chang <strategist922@gmail.com>
    wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about the
    total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about 9000,
    then
    these two nodes become very slow then dead. Afer all, whole cluster
    down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead
    cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --
  • James Chang at Jan 28, 2013 at 7:01 am
    Hi Harsh,

    May I know more about "a fatter region would have much less overhead
    while
    giving an equivalent performance in return."

    Did you mean increase the value in the following setting?
    <property>
    <name>hbase.hregion.max.filesize</name>
    <value>1073741824</value>
    </property>

    By the way, there are total 50 tables in this cluster, about 3 tables'
    size more than 1TB,
    others are between 200MB ~ 700MB

    Best Regards.
    James Chang



    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi,

    It does also talk of practical limitations - the more regions you
    grow, the more the pressure on the RS increases (especially in
    memory). May I ask why you have over 9k regions in a single
    region-server? Sounds rather inefficient to me if it is all for a few
    tables, cause a fatter region would have much less overhead while
    giving an equivalent performance in return.
    On Mon, Jan 28, 2013 at 11:54 AM, James Chang wrote:
    Hi Hash,

    Thanks your information, the thread you mentioned only answer the first
    question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD

    On Mon, Jan 28, 2013 at 10:32 AM, James Chang <strategist922@gmail.com>
    wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about the
    total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the
    average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about 9000,
    then
    these two nodes become very slow then dead. Afer all, whole cluster
    down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead
    cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --


    --
  • Harsh J at Jan 28, 2013 at 7:31 am
    Yes I meant to say about increasing the per-region filesize, such that
    there is less metadata overhead per table, across the cluster. Also,
    using CDH4 would yield much better performance for fatter regions
    thanks to its HFileV2 format.
    On Mon, Jan 28, 2013 at 12:30 PM, James Chang wrote:
    Hi Harsh,

    May I know more about "a fatter region would have much less overhead
    while

    giving an equivalent performance in return."

    Did you mean increase the value in the following setting?
    <property>
    <name>hbase.hregion.max.filesize</name>
    <value>1073741824</value>
    </property>

    By the way, there are total 50 tables in this cluster, about 3 tables'
    size more than 1TB,
    others are between 200MB ~ 700MB

    Best Regards.
    James Chang



    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi,

    It does also talk of practical limitations - the more regions you
    grow, the more the pressure on the RS increases (especially in
    memory). May I ask why you have over 9k regions in a single
    region-server? Sounds rather inefficient to me if it is all for a few
    tables, cause a fatter region would have much less overhead while
    giving an equivalent performance in return.

    On Mon, Jan 28, 2013 at 11:54 AM, James Chang <strategist922@gmail.com>
    wrote:
    Hi Hash,

    Thanks your information, the thread you mentioned only answer the
    first
    question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD

    On Mon, Jan 28, 2013 at 10:32 AM, James Chang <strategist922@gmail.com>
    wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about the
    total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the
    average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about 9000,
    then
    these two nodes become very slow then dead. Afer all, whole cluster
    down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead
    cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --
  • James Chang at Jan 28, 2013 at 10:05 am
    Hi Harsh,

    We are interested in upgrading to CDH4, but we are waitting Cloudra
    release a SOP for upgrade without using Cloudera Manager.

    Best Regards.
    James Chang


    2013/1/28 Harsh J <harsh@cloudera.com>
    Yes I meant to say about increasing the per-region filesize, such that
    there is less metadata overhead per table, across the cluster. Also,
    using CDH4 would yield much better performance for fatter regions
    thanks to its HFileV2 format.
    On Mon, Jan 28, 2013 at 12:30 PM, James Chang wrote:
    Hi Harsh,

    May I know more about "a fatter region would have much less overhead
    while

    giving an equivalent performance in return."

    Did you mean increase the value in the following setting?
    <property>
    <name>hbase.hregion.max.filesize</name>
    <value>1073741824</value>
    </property>

    By the way, there are total 50 tables in this cluster, about 3 tables'
    size more than 1TB,
    others are between 200MB ~ 700MB

    Best Regards.
    James Chang



    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi,

    It does also talk of practical limitations - the more regions you
    grow, the more the pressure on the RS increases (especially in
    memory). May I ask why you have over 9k regions in a single
    region-server? Sounds rather inefficient to me if it is all for a few
    tables, cause a fatter region would have much less overhead while
    giving an equivalent performance in return.

    On Mon, Jan 28, 2013 at 11:54 AM, James Chang <strategist922@gmail.com>
    wrote:
    Hi Hash,

    Thanks your information, the thread you mentioned only answer the
    first
    question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from user@hbase.apache.org
    :
    http://search-hadoop.com/m/cyFfl1SHnbD

    On Mon, Jan 28, 2013 at 10:32 AM, James Chang <
    strategist922@gmail.com>
    wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation about
    the
    total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the
    average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about
    9000,
    then
    these two nodes become very slow then dead. Afer all, whole cluster
    down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the dead
    cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --


    --
  • James Chang at Jan 29, 2013 at 4:32 am
    Hi Harsh,

    What I mwan SOP is "Standard Operating Procedure Guide for Upgrade
    From CDH3(especially CDH3U4) to CDH4".
    My currently cluster is CDH3U4 with RedHat Enterprise Linux 6.3 Release 64
    bit version.

    We are heavily use HBase and Pig. So we need a step by step guide for
    upgrading, especially how to reduce down time and
    prevent data lose.

    As we know, the component of CDH3 and CDH4 are quite different, so we
    are seriously considering upgrade or not.

    So, in my opinion, in this SOP, it should cover the following:
    1. Step by step upgrading procedure.
    2. Rollback procedures if upgrade failed.
    3. Sanity check and verify procedure.

    Best Regards.
    James Chang


    Best Regarsd.



    2013/1/28 Harsh J <harsh@cloudera.com>
    I'm not sure what SOP means, but CDH upgrades do not require CM. In
    fact, as of 4.1, CM does not handle CDH upgrades yet (though it has
    arrived in 4.5 beta's parcels). Let us know if you need help with any
    upgrades.
    On Mon, Jan 28, 2013 at 3:34 PM, James Chang wrote:
    Hi Harsh,

    We are interested in upgrading to CDH4, but we are waitting Cloudra
    release a SOP for upgrade without using Cloudera Manager.

    Best Regards.
    James Chang


    2013/1/28 Harsh J <harsh@cloudera.com>
    Yes I meant to say about increasing the per-region filesize, such that
    there is less metadata overhead per table, across the cluster. Also,
    using CDH4 would yield much better performance for fatter regions
    thanks to its HFileV2 format.

    On Mon, Jan 28, 2013 at 12:30 PM, James Chang <strategist922@gmail.com>
    wrote:
    Hi Harsh,

    May I know more about "a fatter region would have much less
    overhead
    while

    giving an equivalent performance in return."

    Did you mean increase the value in the following setting?
    <property>
    <name>hbase.hregion.max.filesize</name>
    <value>1073741824</value>
    </property>

    By the way, there are total 50 tables in this cluster, about 3
    tables'
    size more than 1TB,
    others are between 200MB ~ 700MB

    Best Regards.
    James Chang



    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi,

    It does also talk of practical limitations - the more regions you
    grow, the more the pressure on the RS increases (especially in
    memory). May I ask why you have over 9k regions in a single
    region-server? Sounds rather inefficient to me if it is all for a few
    tables, cause a fatter region would have much less overhead while
    giving an equivalent performance in return.

    On Mon, Jan 28, 2013 at 11:54 AM, James Chang <
    strategist922@gmail.com>
    wrote:
    Hi Hash,

    Thanks your information, the thread you mentioned only answer
    the
    first
    question and the answer is "NO maximum number" isn't it?
    Could you kindly give me some suggestion about my second question?

    Thanks in advance!


    2013/1/28 Harsh J <harsh@cloudera.com>
    Hi James,

    Please take a look at this earlier thread from
    user@hbase.apache.org:
    http://search-hadoop.com/m/cyFfl1SHnbD

    On Mon, Jan 28, 2013 at 10:32 AM, James Chang
    <strategist922@gmail.com>
    wrote:
    Dears,

    Does anyone know HBase 0.90.6(CDH3U4) has limitation
    about
    the
    total
    regions per region server? I cannot find any setting in HBse's
    configuration file!? or I missed something?

    And when I try in my (CDH3U4) 6 nodes small cluster, the
    average
    number of region is 6500 per region server, when one node crash
    yesterday,
    there were 2 of the alive region servers's region become about
    9000,
    then
    these two nodes become very slow then dead. Afer all, whole
    cluster
    down
    and the cluster cannot restart.

    So, my questions are:
    1. What's the maximum number of regions per region server ?
    2. In this incident, what's the best practice to recover the
    dead
    cluster?
    (except add more nodes, because this need more time)

    Best Regards.
    James Chang





    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --

    --



    --
    Harsh J

    --

    --



    --
    Harsh J
    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcdh-user @
categorieshadoop
postedJan 28, '13 at 5:03a
activeJan 29, '13 at 4:32a
posts8
users2
websitecloudera.com
irc#hadoop

2 users in discussion

James Chang: 5 posts Harsh J: 3 posts

People

Translate

site design / logo © 2022 Grokbase