FAQ
[ https://issues.apache.org/jira/browse/HBASE-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566252#action_12566252 ]

viper799 edited comment on HBASE-69 at 2/6/08 10:28 AM:
-------------------------------------------------------------

Might check out the compaction queue I had 1 out or 2 regions on one server split and reload the splits on the different server and my job finished with in a few mins after the split
Only two more columns compacted after the split and the job finished. I now have column family's that have more then 3 map files and no compaction happening.

Not sure if the stop of updates coming in or the split mess up the compaction queue or the split cleared it out.
Before the split happened there was several requested for compaction for the region that did not split.




was (Author: viper799):
Might check out the compaction queue I had 1 out or 2 regions on one server split and reload the splits on the same server and my job finished with in a few mins after the split
Only two more columns compacted after the split and the job finished. I now have column family's that have more then 3 map files and no compaction happening.

Not sure if the stop of updates coming in or the split mess up the compaction queue or the split cleared it out.
Before the split happened there was several columns requested compaction for both regions.

And on reload of the split region we should have queue up compaction to so compaction could clear out the references files from the split.


[hbase] Make cache flush triggering less simplistic
---------------------------------------------------

Key: HBASE-69
URL: https://issues.apache.org/jira/browse/HBASE-69
Project: Hadoop HBase
Issue Type: Improvement
Components: regionserver
Reporter: stack
Assignee: Jim Kellerman
Fix For: 0.2.0

Attachments: patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
I would think Stores should only dump their memcache disk if they have some substance.
The problem becomes more acute, the more families you have in a Region.
Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size. Behavior would vary dependent on the prompt that provoked the flush. Would also log why the flush is running: optional or > max size.
This issue comes out of HADOOP-2621.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Billy Pearson (JIRA) at Feb 12, 2008 at 9:59 am
    [ https://issues.apache.org/jira/browse/HBASE-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568027#action_12568027 ]

    viper799 edited comment on HBASE-69 at 2/12/08 1:57 AM:
    -------------------------------------------------------------

    have not tried newest patch yet for this problem.
    I thank we are still missing something when I talk about hlog build up I mean there is never a removal of the hlogs unless there is a split or a shutdown.

    I have over 106 hlog on one server
    hbase.regionserver.maxlogentries = 250000
    The modification time of the oldest to the newest logs files are much greater then the 30 min option flush time set in hbase-default.xml


    was (Author: viper799):
    I thank we are still missing something when I talk about hlog build up I mean there is never a removal of the hlogs unless there is a split or a shutdown.

    I have over 106 hlog on one server
    hbase.regionserver.maxlogentries = 250000
    The modification time of the oldest to the newest logs files are much greater then the 30 min option flush time set in hbase-default.xml

    [hbase] Make cache flush triggering less simplistic
    ---------------------------------------------------

    Key: HBASE-69
    URL: https://issues.apache.org/jira/browse/HBASE-69
    Project: Hadoop HBase
    Issue Type: Improvement
    Components: regionserver
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.2.0

    Attachments: patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
    I would think Stores should only dump their memcache disk if they have some substance.
    The problem becomes more acute, the more families you have in a Region.
    Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size. Behavior would vary dependent on the prompt that provoked the flush. Would also log why the flush is running: optional or > max size.
    This issue comes out of HADOOP-2621.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedFeb 6, '08 at 6:29p
activeFeb 12, '08 at 9:59a
posts2
users1
websitehbase.apache.org

1 user in discussion

Billy Pearson (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase