|| at May 28, 2012 at 6:22 pm
Overlapping regions (https://issues.apache.org/jira/browse/HBASE-4238) do not show up very often. We know that it is fixed after 0.90.5, but instead of upgrading the production hbase cluster, we have an internal tool (call Hbase APIs) to fix it.
Regions out of sync (between META and HDFS) and Inactive regions appear more often --- we can see them every week; again, our internal tool handles these cases as well.
From: Kevin O'dell
Sent: Monday, May 28, 2012 1:45 PM
Subject: Re: How often should we reboot hbase cluster---looking for best practice
+1 what Harsh said. It sounds to me like you are putting a bandaid on a
flesh wound. We should do further analysis and get your cluster to a
stable state rather than repairing it weekly. Can you also describe in
more detail everything you are running to do the repair in 90.4 fixing an
overlapping region is not an easy task by any means.
On Mon, May 28, 2012 at 9:14 AM, Harsh J wrote:
If you're talking of "hbck -fix", then no you don't need to restart
HBase after it resolves your issues.
Would be good to investigate/know what causes such frequent
inconsistencies in your cluster though. Its not normal for
inconsistencies to appear regularly every week. Do your region servers
often crash weekly, for instance?
On Mon, May 28, 2012 at 9:35 PM, Xu, Richard wrote:
It is more like an operation question.
Hbase version is 0.90.4, we have a weekly job to fix known issues such
as META table out of sync, inactive/overlapping/dangling regions while
hbase is online.
Should we restart the hbase cluster right after the fix? What is the
best practice here?
Thanks in advance!
Customer Operations Engineer, Cloudera