FAQ
I use Hbase0.94.1 and hadoop-0.20.2-cdh3u5.

The Hbase0.94.1 write checksums in HBase block, so we don't need to read
checksums from metadata file of block file. But I find the BlockSender
still read checksums from metadata file of block file every time, so the
Hbase0.94.1 can't decrease the iops, that is right? If that is right,
should I set the hbase.regionserver.checksum.verify property to false in
order to avoid the Hbase0.94.1 to verify th checksum?


Thanks,
LiuLei

Search Discussions

  • Michael Stack at Sep 15, 2012 at 8:31 pm

    On Sat, Sep 15, 2012 at 10:31 AM, jlei liu wrote:
    I use Hbase0.94.1 and hadoop-0.20.2-cdh3u5.

    The Hbase0.94.1 write checksums in HBase block, so we don't need to read
    checksums from metadata file of block file. But I find the BlockSender
    still read checksums from metadata file of block file every time,
    How are you figuring this? That sounds like a regression.

    In 0.94.x the default is that hbase.regionserver.checksum.verify is
    true which should have hbase doing checksum verify on read rather than
    hdfs.
    so the
    Hbase0.94.1 can't decrease the iops, that is right? If that is right,
    should I set the hbase.regionserver.checksum.verify property to false in
    order to avoid the Hbase0.94.1 to verify th checksum?
    It would be interesting flipping the switch. If things went faster,
    then for sure there is a regression.
    St.Ack
  • Jlei liu at Sep 16, 2012 at 2:07 am
    The HBase support checksums in HBase block cache in
    HBASE-5074<https://issues.apache.org/jira/browse/HBASE-5074>
    jira.
    The HBase support checksums for decrease the iops of HDFS, so that HDFS
    dont't need to read the metadata of block file. But in hadoop-0.20.2-cdh3u5
    version, BlockSender still read the metadata file even if the
    hbase.regionserver.checksum.verify property is ture. Because the HBASE-5074
    jira can't decrease the iops of HDFS, so I want to set
    hbase.regionserver.checksum.verify property to fasle.
    2012/9/16 Stack <stack@duboce.net>
    On Sat, Sep 15, 2012 at 10:31 AM, jlei liu wrote:
    I use Hbase0.94.1 and hadoop-0.20.2-cdh3u5.

    The Hbase0.94.1 write checksums in HBase block, so we don't need to read
    checksums from metadata file of block file. But I find the BlockSender
    still read checksums from metadata file of block file every time,
    How are you figuring this? That sounds like a regression.

    In 0.94.x the default is that hbase.regionserver.checksum.verify is
    true which should have hbase doing checksum verify on read rather than
    hdfs.
    so the
    Hbase0.94.1 can't decrease the iops, that is right? If that is right,
    should I set the hbase.regionserver.checksum.verify property to false in
    order to avoid the Hbase0.94.1 to verify th checksum?
    It would be interesting flipping the switch. If things went faster,
    then for sure there is a regression.
    St.Ack
  • Michael Stack at Sep 16, 2012 at 10:13 pm

    On Sat, Sep 15, 2012 at 7:06 PM, jlei liu wrote:
    The HBase support checksums in HBase block cache in
    HBASE-5074<https://issues.apache.org/jira/browse/HBASE-5074>
    jira.
    The HBase support checksums for decrease the iops of HDFS, so that HDFS
    dont't need to read the metadata of block file. But in hadoop-0.20.2-cdh3u5
    version, BlockSender still read the metadata file even if the
    hbase.regionserver.checksum.verify property is ture. Because the HBASE-5074
    jira can't decrease the iops of HDFS, so I want to set
    hbase.regionserver.checksum.verify property to fasle.
    2012/9/16 Stack <stack@duboce.net>
    Go ahead, but, it seems like something is broke? We should file an
    issue to fix this broken checksuming?
    St.Ack

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedSep 15, '12 at 5:32p
activeSep 16, '12 at 10:13p
posts4
users2
websitehbase.apache.org

2 users in discussion

Michael Stack: 2 posts Jlei liu: 2 posts

People

Translate

site design / logo © 2021 Grokbase