FAQ
If HBase starts/stops often in less than 24 hours, you end up with lots of store files
--------------------------------------------------------------------------------------

Key: HBASE-1947
URL: https://issues.apache.org/jira/browse/HBASE-1947
Project: Hadoop HBase
Issue Type: Improvement
Affects Versions: 0.20.1
Reporter: Jean-Daniel Cryans
Fix For: 0.20.2, 0.21.0


Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Jean-Daniel Cryans (JIRA) at Oct 30, 2009 at 9:34 pm
    [ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jean-Daniel Cryans updated HBASE-1947:
    --------------------------------------

    Attachment: HBASE-1947.patch

    Simple patch that adds new methods to do the checks. I tried it on my machine and it compacted the 20 store files I had in ROOT right away.
    If HBase starts/stops often in less than 24 hours, you end up with lots of store files
    --------------------------------------------------------------------------------------

    Key: HBASE-1947
    URL: https://issues.apache.org/jira/browse/HBASE-1947
    Project: Hadoop HBase
    Issue Type: Improvement
    Affects Versions: 0.20.1
    Reporter: Jean-Daniel Cryans
    Fix For: 0.20.2, 0.21.0

    Attachments: HBASE-1947.patch


    Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Nov 2, 2009 at 11:23 pm
    [ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772747#action_12772747 ]

    Jean-Daniel Cryans commented on HBASE-1947:
    -------------------------------------------

    Ryan reviewed this patch and wants me to clear the compaction message. I will commit the same patch but with:

    {code}
    - // Startup a compaction early if one is needed, if region has references.
    - if (region.hasReferences()) {
    + // Startup a compaction early if one is needed, if region has references
    + // or it a store has too many store files
    + if (region.hasReferences() || region.hasTooManyStoreFiles()) {
    this.compactSplitThread.compactionRequested(region,
    - "Region has references on open");
    + region.hasReferences() ? "Region has references on open" :
    + "Region has too many store files");
    {code}
    If HBase starts/stops often in less than 24 hours, you end up with lots of store files
    --------------------------------------------------------------------------------------

    Key: HBASE-1947
    URL: https://issues.apache.org/jira/browse/HBASE-1947
    Project: Hadoop HBase
    Issue Type: Improvement
    Affects Versions: 0.20.1
    Reporter: Jean-Daniel Cryans
    Fix For: 0.20.2, 0.21.0

    Attachments: HBASE-1947.patch


    Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Nov 2, 2009 at 11:33 pm
    [ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jean-Daniel Cryans resolved HBASE-1947.
    ---------------------------------------

    Resolution: Fixed
    Assignee: Jean-Daniel Cryans
    Hadoop Flags: [Reviewed]

    Committed to branch and trunk.
    If HBase starts/stops often in less than 24 hours, you end up with lots of store files
    --------------------------------------------------------------------------------------

    Key: HBASE-1947
    URL: https://issues.apache.org/jira/browse/HBASE-1947
    Project: Hadoop HBase
    Issue Type: Improvement
    Affects Versions: 0.20.1
    Reporter: Jean-Daniel Cryans
    Assignee: Jean-Daniel Cryans
    Fix For: 0.20.2, 0.21.0

    Attachments: HBASE-1947.patch


    Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedOct 30, '09 at 6:48p
activeNov 2, '09 at 11:33p
posts4
users1
websitehbase.apache.org

1 user in discussion

Jean-Daniel Cryans (JIRA): 4 posts

People

Translate

site design / logo © 2023 Grokbase