FAQ
[hbase] regionserver creates hlogs without bound
------------------------------------------------

Key: HADOOP-1820
URL: https://issues.apache.org/jira/browse/HADOOP-1820
Project: Hadoop
Issue Type: Improvement
Components: contrib/hbase
Reporter: stack
Priority: Minor


Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.

Just now, I had a host crash w/ 112 log files each of 30k plus edits each.

We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • stack (JIRA) at Aug 30, 2007 at 9:18 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523938 ]

    stack commented on HADOOP-1820:
    -------------------------------

    Here's some more info. The split process itself gets stuck never making it past file 2, throws an exception, and then starts over splitting from scratch again. I'll attach a log file excerpt.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Improvement
    Components: contrib/hbase
    Reporter: stack
    Priority: Minor

    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Aug 30, 2007 at 9:24 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HADOOP-1820:
    --------------------------

    Attachment: excerpt.log
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Improvement
    Components: contrib/hbase
    Reporter: stack
    Priority: Minor
    Attachments: excerpt.log


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 10, 2007 at 7:11 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman reassigned HADOOP-1820:
    -------------------------------------

    Assignee: Jim Kellerman
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Improvement
    Components: contrib/hbase
    Reporter: stack
    Assignee: Jim Kellerman
    Priority: Minor
    Attachments: excerpt.log


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 10, 2007 at 11:22 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526303 ]

    Jim Kellerman commented on HADOOP-1820:
    ---------------------------------------

    Every time the log is rolled, it is supposed to delete logs which are older than the oldest applied edit.

    It could be that when a region splits, the information about the parent region is not getting removed from the HLog's map of region name -> last commit id. Since the parent region will never receive another update, its sequence number will continue to become older and older and prevent any logs from being deleted since the last sequence number for the oldest parent region. Consequently the only way the logs will get cleaned up is if:

    - the region server shuts down cleanly
    - the region server crashes and the master splits the logs for it.

    it appears that what we need to do is when a region splits, we need to remove it from the map of region names -> last sequence number.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Improvement
    Components: contrib/hbase
    Reporter: stack
    Assignee: Jim Kellerman
    Priority: Minor
    Attachments: excerpt.log


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 15, 2007 at 10:56 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Priority: Major (was: Minor)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Improvement
    Components: contrib/hbase
    Reporter: stack
    Assignee: Jim Kellerman
    Attachments: excerpt.log


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 15, 2007 at 11:08 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Issue Type: Bug (was: Improvement)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Reporter: stack
    Assignee: Jim Kellerman
    Attachments: excerpt.log


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 12:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt

    Works locally. Try Hudson.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Reporter: stack
    Assignee: Jim Kellerman
    Attachments: excerpt.log, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 12:09 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Fix Version/s: 0.15.0
    Affects Version/s: 0.15.0
    Status: Patch Available (was: Open)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 16, 2007 at 1:22 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527806 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12365926/patch.txt
    against trunk revision r575986.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/773/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/773/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/773/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/773/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 1:24 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: In Progress (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 1:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: In Progress)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 1:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt

    Allows the number of log files remaining at end to be <= 2 instead of exactly 2
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 16, 2007 at 3:13 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527817 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12365927/patch.txt
    against trunk revision r575986.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/774/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/774/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/774/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/774/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 3:27 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 3:27 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)

    Each time it runs a different test fails that is unrelated to what this patch addresses
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 16, 2007 at 6:17 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527826 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12365927/patch.txt
    against trunk revision r575986.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/776/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/776/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/776/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/776/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 6:21 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 16, 2007 at 6:22 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)

    Two different tests failed this time. Hudson is a Heisenbug generator.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 16, 2007 at 8:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527834 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12365927/patch.txt
    against trunk revision r575986.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/777/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/777/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/777/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/777/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 20, 2007 at 6:11 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 20, 2007 at 6:15 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)

    Works locally. Try Hudson.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 20, 2007 at 6:15 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt

    new patch based on recently committed changes for HADOOP-1923 and HADOOP-1924
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 20, 2007 at 8:45 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529255 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366305/patch.txt
    against trunk revision r577603.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/801/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/801/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/801/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/801/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 21, 2007 at 12:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 21, 2007 at 12:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 21, 2007 at 12:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt

    Increase lease timeout period, reduce frequency of lease checking for TestLogRolling and TestSplit
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 21, 2007 at 2:44 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529442 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366322/patch.txt
    against trunk revision r577917.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/804/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/804/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/804/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/804/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 5:21 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: In Progress (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 5:41 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529753 ]

    Jim Kellerman commented on HADOOP-1820:
    ---------------------------------------

    There were three HBase tests that failed in build 804:

    testLogRolling failed because a region split took an excessive amount of time (43 seconds), and by the time the split completed, the client had given up and started shutdown of the MiniHBase cluster. This is undoubtedly caused by the aggressive parameters we set for client retries (time between retries: 5 seconds, number of retries: 5) to make tests run faster. Because a region split is highly I/O intensive, the thread will often block on I/O. If the thread cannot get rescheduled because the client is doing a retry, this could cause the splitting thread to be starved and cause the long time for the split to complete. By contrast, the test testBasicSplit which does not have a cluster or a client running, completes a region split in 1 second.

    Similarly a region split in testSplitRegionIsDeleted, which also runs a cluster and a client, took 50 seconds to complete a region split, and once again the client gave up and started shutting down the cluster just as the split was completing.

    in testTableMapReduce, the default settings for region server timeout are too short so the master takes the region server out of rotation just before it completed its task.

    So it appears that in the build environment, we need to increase the parameters for these respective tests so that they can run without timing out.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 8:31 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 8:33 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: In Progress)

    This patch makes the client retry wait longer and the lease timeout longer for the effected tests.
    Hopefully, it is slow enough that it will run on Hudson properly.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 23, 2007 at 9:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529765 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366429/patch.txt
    against trunk revision r578543.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/811/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/811/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/811/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/811/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 10:11 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: In Progress (was: Patch Available)

    Almost from the very beginning TestHBaseCluster had problems with the mini dfs cluster:

    [junit] 07/09/23 21:09:13 WARN fs.FSNamesystem: Not able to place enough replicas, still in need of 1

    This continued until the test timed out. Basically no forward progress could be made. The changes in this patch do not effect this test, so there must be some kind of race going on between the dfs mini cluster's servers and HBase's.

    The next patch will back off some of the more aggressive test settings for this test to see if the real problem is timing related. Since the tests that failed previously succeeded this time, apparently, for some tests, the aggressive parameters used for testing are just a bit too aggressive for a machine which is "resource challenged" as hudson appears to be.

    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 10:48 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: In Progress)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 23, 2007 at 10:48 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 24, 2007 at 12:06 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529770 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366432/patch.txt
    against trunk revision r578605.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/812/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/812/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/812/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/812/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 1:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: In Progress (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 6:14 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 6:16 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: In Progress)

    TestLogRolling: wait after writing until data is flushed to disk.

    TestHBaseCluster: Make lease timeout longer, increase time between client retries.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 24, 2007 at 7:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529789 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366435/patch.txt
    against trunk revision r578605.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/813/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/813/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/813/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/813/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 5:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 5:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)

    Try with Hudson's upgraded JDK.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 24, 2007 at 6:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529928 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366435/patch.txt
    against trunk revision r578879.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests +1. The patch passed core unit tests.

    contrib tests -1. The patch failed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/814/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/814/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/814/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/814/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Open (was: Patch Available)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Status: Patch Available (was: Open)
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Attachment: patch.txt

    Use default client ipc timeout in TestDFSAbort, wait longer in TestLogRolling for cache flushes and log rolls to complete.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 24, 2007 at 10:40 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530006 ]

    Hadoop QA commented on HADOOP-1820:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12366483/patch.txt
    against trunk revision r578964.

    @author +1. The patch does not contain any @author tags.

    javadoc +1. The javadoc tool did not generate any warning messages.

    javac +1. The applied patch does not generate any new compiler warnings.

    findbugs +1. The patch does not introduce any new Findbugs warnings.

    core tests -1. The patch failed core unit tests.

    contrib tests +1. The patch passed contrib unit tests.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/817/testReport/
    Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/817/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/817/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/817/console

    This message is automatically generated.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 24, 2007 at 10:54 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman updated HADOOP-1820:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    Passed Hudson contrib tests (finally!). Committed.
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jim Kellerman (JIRA) at Sep 25, 2007 at 6:18 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jim Kellerman reopened HADOOP-1820:
    -----------------------------------


    This test is failing on Hudson... reopening
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Sep 25, 2007 at 7:37 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530240 ]

    Hudson commented on HADOOP-1820:
    --------------------------------

    Integrated in Hadoop-Nightly #250 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/250/])
    [hbase] regionserver creates hlogs without bound
    ------------------------------------------------

    Key: HADOOP-1820
    URL: https://issues.apache.org/jira/browse/HADOOP-1820
    Project: Hadoop
    Issue Type: Bug
    Components: contrib/hbase
    Affects Versions: 0.15.0
    Reporter: stack
    Assignee: Jim Kellerman
    Fix For: 0.15.0

    Attachments: excerpt.log, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt, patch.txt


    Regionserver keeps log of all edits for all the regions its carrying. Its used recoverying state if a regionserver crashes: edits that have not been persisted to an HStoreFile are rerun to populate memcache which in turn is converted to an on-filesytem HStoreFile. On a period, the log is rotated and a new one is opened. While the region server is up, the logs grow in number without bound. Only the most recent contain unpersisted edits. If the region server goes down clean, then its logs are cleaned up. If a region server crashes, as part of recovery, the logs of edits are sorted and split per region. Recovery would run faster if it did not have to plough through reams of stale edits.
    Just now, I had a host crash w/ 112 log files each of 30k plus edits each.
    We could rename the log rolling thread the log maintainer. As well as rolling logs, it could check for edit logs to clean. When rolled, logs could be marked with the sequence id of their last contained edit. The thread could on a period ask each hosted region for the "lowest highest" sequence id of all regions deployed. Once this number had crossed out that on a particular log, the log could be cleaned up safely.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedAug 30, '07 at 8:18p
activeOct 1, '07 at 6:10p
posts64
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 64 posts

People

Translate

site design / logo © 2022 Grokbase