FAQ
Hi,
I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
cluster which runs 0.90

Can someone go over the attached file and see if any parameter is
incorrectly set ?

Thanks

Search Discussions

  • Stack at Jan 22, 2011 at 5:56 am
    Remove:

    hbase.master
    hbase.regions.slop
    hbase.regionserver.restart.on.zk.expire
    hbase.regions.percheckin
    hbase.zookeeper.property.tickTime

    None of the above are pertinent any more.

    Your 'hbase.hregion.memstore.flush.size' is very high. Why so high?
    Same for your hfile blocksize (Though IIRC, you have lots of small
    records so you are probably trying to keep down the size of your
    indices?)

    Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
    and you have 23seconds. Any reason?

    Your zk timeout is very high, higher than default 3 minutes in 0.90.
    Maybe its OK though?

    The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
    abnormal. It shouldn't usually have to be this high if you reuse your
    Configuration making HTables... but leave it as is. Probably does no
    harm.

    Hope that helps Ted,
    St.Ack



    On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu wrote:
    Hi,
    I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
    cluster which runs 0.90

    Can someone go over the attached file and see if any parameter is
    incorrectly set ?

    Thanks
  • Stack at Jan 22, 2011 at 5:57 am
    Oh, makes sure you don't have an hbase-default.xml from an old hbase
    in your conf dir (its in the hbase-0.90.0.jar now and doc'd here
    http://hbase.apache.org/configuration.html#hbase_default_configurations).
    St.Ack
    On Fri, Jan 21, 2011 at 9:56 PM, Stack wrote:
    Remove:

    hbase.master
    hbase.regions.slop
    hbase.regionserver.restart.on.zk.expire
    hbase.regions.percheckin
    hbase.zookeeper.property.tickTime

    None of the above are pertinent any more.

    Your 'hbase.hregion.memstore.flush.size' is very high.  Why so high?
    Same for your hfile blocksize (Though IIRC, you have lots of small
    records so you are probably trying to keep down the size of your
    indices?)

    Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
    and you have 23seconds.  Any reason?

    Your zk timeout is very high, higher than default 3 minutes in 0.90.
    Maybe its OK though?

    The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
    abnormal.  It shouldn't usually have to be this high if you reuse your
    Configuration making HTables... but leave it as is.  Probably does no
    harm.

    Hope that helps Ted,
    St.Ack



    On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu wrote:
    Hi,
    I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
    cluster which runs 0.90

    Can someone go over the attached file and see if any parameter is
    incorrectly set ?

    Thanks
  • Ted Yu at Jan 22, 2011 at 3:45 pm
    I encountered different kinds of failure w.r.t. hbase 0.90 running our flow
    over 200GB input data.
    Some values from my config were derived from processing 500GB data using
    0.20.6 on our staging cluster.

    I couldn't find reference to 'hfile.min.blocksize.size' in 0.90 source code.


    'hbase.zookeeper.property.maxClientCnxns' is high because of zookeeper issue
    I mentioned in another thread.

    I will report back after further experiments.

    Thanks Stack.
    On Fri, Jan 21, 2011 at 9:56 PM, Stack wrote:

    Remove:

    hbase.master
    hbase.regions.slop
    hbase.regionserver.restart.on.zk.expire
    hbase.regions.percheckin
    hbase.zookeeper.property.tickTime

    None of the above are pertinent any more.

    Your 'hbase.hregion.memstore.flush.size' is very high. Why so high?
    Same for your hfile blocksize (Though IIRC, you have lots of small
    records so you are probably trying to keep down the size of your
    indices?)

    Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
    and you have 23seconds. Any reason?

    Your zk timeout is very high, higher than default 3 minutes in 0.90.
    Maybe its OK though?

    The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
    abnormal. It shouldn't usually have to be this high if you reuse your
    Configuration making HTables... but leave it as is. Probably does no
    harm.

    Hope that helps Ted,
    St.Ack



    On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu wrote:
    Hi,
    I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
    cluster which runs 0.90

    Can someone go over the attached file and see if any parameter is
    incorrectly set ?

    Thanks
  • Tatsuya Kawano at Jan 22, 2011 at 6:12 am
    Hi Ted,

    This is not a direct answer to your question, but I found the Configuration chapter of The Apache HBase Book v0.90.0 is helpful. It contains a complete list of the properties and some important/recommended settings. You might want to check it if you haven't.

    http://hbase.apache.org/book.html

    Thanks,
    --
    Tatsuya Kawano
    Tokyo, Japan

    On Jan 22, 2011, at 1:09 PM, Ted Yu wrote:

    Hi,
    I copied hbase-site.xml from our staging cluster running 0.20.6 to dev cluster which runs 0.90

    Can someone go over the attached file and see if any parameter is incorrectly set ?

    Thanks
    <hbase-site.xml>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJan 22, '11 at 4:10a
activeJan 22, '11 at 3:45p
posts5
users3
websitehbase.apache.org

3 users in discussion

Ted Yu: 2 posts Stack: 2 posts Tatsuya Kawano: 1 post

People

Translate

site design / logo © 2022 Grokbase