Grokbase Groups HBase dev June 2009
FAQ
BLOCKCACHE always on even when disabled
---------------------------------------

Key: HBASE-1486
URL: https://issues.apache.org/jira/browse/HBASE-1486
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.20.0


Comes from Billy Pearson up on list.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Lars George (JIRA) at Jun 6, 2009 at 10:48 pm
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Lars George reassigned HBASE-1486:
    ----------------------------------

    Assignee: Lars George
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Lars George (JIRA) at Jun 6, 2009 at 10:52 pm
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Lars George updated HBASE-1486:
    -------------------------------

    Attachment: 1486.patch

    I am taking a stab at this. Maybe Billy can verify as well if this fixes the issue.

    The issue was that the isBlockCacheEnabled from the HColumnDefinition was *not* handed to the Store and StoreFiles eventually. I added this to these two classes and when a block cache is asked for during the creation of a Reader it skips it when blockcache is set to false.
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Lars George (JIRA) at Jun 6, 2009 at 11:28 pm
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Lars George updated HBASE-1486:
    -------------------------------

    Attachment: 1486-2.patch

    Oops, sorry, my IDE missed a few spots. The ant build found them though. This second patch fixes all compile issues.

    But please note that now the TestStoreFile.java has the option to test with and without the block cache. I have set it to "false" but maybe the tests need to be run twice? Once with and once without block cache?
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Ryan Rawson at Jun 7, 2009 at 12:00 am
    Lars, can you make sure this bit defaults to true?

    On Jun 6, 2009 4:28 PM, "Lars George (JIRA)" wrote:

    [
    https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issue.
    ..
    Attachment: 1486-2.patch

    Oops, sorry, my IDE missed a few spots. The ant build found them though.
    This second patch fixes all compile issues.

    But please note that now the TestStoreFile.java has the option to test with
    and without the block cache. I have set it to "false" but maybe the tests
    need to be run twice? Once with and once without block cache?
    BLOCKCACHE always on even when disabled >
    --------------------------------------- > > ...
    Attachments: 1486-2.patch, 1486.patch
    Comes from Billy Pearson up on list. -- This message is automatically
    generated by JIRA. - Y...
  • ryan rawson (JIRA) at Jun 7, 2009 at 12:30 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716999#action_12716999 ]

    ryan rawson commented on HBASE-1486:
    ------------------------------------

    the block cache family value should default to 'true' or 'on'. Can you please make sure this is the case.
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Billy Pearson (JIRA) at Jun 7, 2009 at 12:40 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717001#action_12717001 ]

    Billy Pearson commented on HBASE-1486:
    --------------------------------------

    BLOCKCACHE => 'true' for info in meta
    and
    BLOCKCACHE => 'false' for historian
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Billy Pearson (JIRA) at Jun 7, 2009 at 12:40 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717000#action_12717000 ]

    Billy Pearson commented on HBASE-1486:
    --------------------------------------

    when I create a new table and do not define block cache in the column it was getting set to true

    also same in meta

    {code}
    hbase(main):002:0> describe '.META.'
    09/06/06 19:37:36 INFO zookeeper.ZooKeeperWrapper: Quorum servers: server-1:2181
    09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
    09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got 10.0.1.1:60000
    DESCRIPTION ENABLED
    09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
    09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.0.1.5:60020
    09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Found ROOT at 10.0.1.5:60020
    09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: address: 10.0.1.8:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', MEMCACHE_FLUSHSIZE => '16384', IS_META => 'true', FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION => 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
    09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.0.1.5:60020
    09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Found ROOT at 10.0.1.5:60020
    {NAME => '.META.', MEMCACHE_FLUSHSIZE => '16384', IS_META => 'true', F true
    AMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '
    2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false
    ', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VER
    SIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY =>
    'false', BLOCKCACHE => 'true'}]}
    1 row(s) in 0.2430 seconds

    {code}

    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Billy Pearson (JIRA) at Jun 7, 2009 at 1:04 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717002#action_12717002 ]

    Billy Pearson commented on HBASE-1486:
    --------------------------------------

    Seams to be working HBASE-1304 killed my ability to import until its side effects get resolved I put some data and flushed via shell then confirmed that the region server was using 0 blocks of block cache where the region was stored.
    so +1

    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Lars George (JIRA) at Jun 7, 2009 at 8:10 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Lars George updated HBASE-1486:
    -------------------------------

    Attachment: 1486-3.patch

    OK, as per the discussion, I have

    - changed HColumnDefinition to have the block cache set to *true* by default
    - changed the TestStoreFile to test *with* block cache on as it was the case originally (since it was always on)

    Also, I checked the HTableDescriptor and found that for

    - ROOT block cache is *true*
    - .META. block cache is *true*
    - Historian block cache is *false*

    Are these still OK. Sounds to me, not sure about the Historian, but is seems to be set to false on purpose?


    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486-3.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jun 8, 2009 at 1:50 am
    [ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack resolved HBASE-1486.
    --------------------------

    Resolution: Fixed

    Committed. Thanks for the patch Lars.

    In 0.19.x, we had block cache on for catalog tables as default (off for historian). We wanted to have block cache on for all tables by default for same reasons as I see being put forward up on #hbase but in 0.19, block cache could cause us to OOME so we were selective -- the catalog tables only.

    I added note that migration needs to enable block cache on tables or at least recommend that old tables have it manually enabled.
    BLOCKCACHE always on even when disabled
    ---------------------------------------

    Key: HBASE-1486
    URL: https://issues.apache.org/jira/browse/HBASE-1486
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: Lars George
    Fix For: 0.20.0

    Attachments: 1486-2.patch, 1486-3.patch, 1486.patch


    Comes from Billy Pearson up on list.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJun 5, '09 at 7:40p
activeJun 8, '09 at 1:50a
posts11
users2
websitehbase.apache.org

2 users in discussion

stack (JIRA): 10 posts Ryan Rawson: 1 post

People

Translate

site design / logo © 2022 Grokbase