 |
James Chang |
at Jan 29, 2013 at 4:32 am
|
⇧ | 
| |
Hi Harsh,
What I mwan SOP is "Standard Operating Procedure Guide for Upgrade
From CDH3(especially CDH3U4) to CDH4".
My currently cluster is CDH3U4 with RedHat Enterprise Linux 6.3 Release 64
bit version.
We are heavily use HBase and Pig. So we need a step by step guide for
upgrading, especially how to reduce down time and
prevent data lose.
As we know, the component of CDH3 and CDH4 are quite different, so we
are seriously considering upgrade or not.
So, in my opinion, in this SOP, it should cover the following:
1. Step by step upgrading procedure.
2. Rollback procedures if upgrade failed.
3. Sanity check and verify procedure.
Best Regards.
James Chang
Best Regarsd.
2013/1/28 Harsh J <
harsh@cloudera.com>
I'm not sure what SOP means, but CDH upgrades do not require CM. In
fact, as of 4.1, CM does not handle CDH upgrades yet (though it has
arrived in 4.5 beta's parcels). Let us know if you need help with any
upgrades.
On Mon, Jan 28, 2013 at 3:34 PM, James Chang wrote:Hi Harsh,
We are interested in upgrading to CDH4, but we are waitting Cloudra
release a SOP for upgrade without using Cloudera Manager.
Best Regards.
James Chang
2013/1/28 Harsh J <
harsh@cloudera.com>
Yes I meant to say about increasing the per-region filesize, such that
there is less metadata overhead per table, across the cluster. Also,
using CDH4 would yield much better performance for fatter regions
thanks to its HFileV2 format.
On Mon, Jan 28, 2013 at 12:30 PM, James Chang <
strategist922@gmail.com>
wrote:
Hi Harsh,
May I know more about "a fatter region would have much less
overhead
while
giving an equivalent performance in return."
Did you mean increase the value in the following setting?
<property>
<name>hbase.hregion.max.filesize</name>
<value>1073741824</value>
</property>
By the way, there are total 50 tables in this cluster, about 3
tables'
size more than 1TB,
others are between 200MB ~ 700MB
Best Regards.
James Chang
2013/1/28 Harsh J <
harsh@cloudera.com>
Hi,
It does also talk of practical limitations - the more regions you
grow, the more the pressure on the RS increases (especially in
memory). May I ask why you have over 9k regions in a single
region-server? Sounds rather inefficient to me if it is all for a few
tables, cause a fatter region would have much less overhead while
giving an equivalent performance in return.
On Mon, Jan 28, 2013 at 11:54 AM, James Chang <
strategist922@gmail.com>
wrote:
Hi Hash,
Thanks your information, the thread you mentioned only answer
the
first
question and the answer is "NO maximum number" isn't it?
Could you kindly give me some suggestion about my second question?
Thanks in advance!
2013/1/28 Harsh J <
harsh@cloudera.com>
Hi James,
Please take a look at this earlier thread from
user@hbase.apache.org:
http://search-hadoop.com/m/cyFfl1SHnbDOn Mon, Jan 28, 2013 at 10:32 AM, James Chang
<
strategist922@gmail.com>
wrote:
Dears,
Does anyone know HBase 0.90.6(CDH3U4) has limitation
about
the
total
regions per region server? I cannot find any setting in HBse's
configuration file!? or I missed something?
And when I try in my (CDH3U4) 6 nodes small cluster, the
average
number of region is 6500 per region server, when one node crash
yesterday,
there were 2 of the alive region servers's region become about
9000,
then
these two nodes become very slow then dead. Afer all, whole
cluster
down
and the cluster cannot restart.
So, my questions are:
1. What's the maximum number of regions per region server ?
2. In this incident, what's the best practice to recover the
dead
cluster?
(except add more nodes, because this need more time)
Best Regards.
James Chang
--
--
Harsh J
--
--
--
Harsh J
--
--
--
Harsh J
--
--
--
Harsh J
--
