Grokbase Groups HBase dev August 2009
FAQ

[HBase-dev] [jira] Created: (HBASE-1739) hbase-1683 broke splitting; only split three logs no matter what N was.

stack (JIRA)
Aug 3, 2009 at 11:17 pm
hbase-1683 broke splitting; only split three logs no matter what N was.
-----------------------------------------------------------------------

Key: HBASE-1739
URL: https://issues.apache.org/jira/browse/HBASE-1739
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack


There's a hard-coding in HLog#splitLog that presumes we always read in batches of ten logs:

{code}
Index: src/java/org/apache/hadoop/hbase/regionserver/HLog.java
===================================================================
--- src/java/org/apache/hadoop/hbase/regionserver/HLog.java (revision 799653)
+++ src/java/org/apache/hadoop/hbase/regionserver/HLog.java (working copy)
@@ -860,7 +860,7 @@
// Stop at logfiles.length when it's the last step
int endIndex = step == maxSteps - 1? logfiles.length:
step * concurrentLogReads + concurrentLogReads;
- for (int i = (step * 10); i < endIndex; i++) {
+ for (int i = (step * concurrentLogReads); i < endIndex; i++) {
// Check for possibly empty file. With appends, currently Hadoop
// reports a zero length even if the file has been sync'd. Revisit if
// HADOOP-4751 is committed.
{code}

When I changed it so we default to reading 3 files at a time rather than 10 over in hbase-1683, the hard-coding made it so we didn't read all logs.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
reply

Search Discussions

3 responses

  • stack (JIRA) at Aug 3, 2009 at 11:19 pm
    [ https://issues.apache.org/jira/browse/HBASE-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738671#action_12738671 ]

    stack commented on HBASE-1739:
    ------------------------------

    Committed after testing.
    hbase-1683 broke splitting; only split three logs no matter what N was.
    -----------------------------------------------------------------------

    Key: HBASE-1739
    URL: https://issues.apache.org/jira/browse/HBASE-1739
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack

    There's a hard-coding in HLog#splitLog that presumes we always read in batches of ten logs:
    {code}
    Index: src/java/org/apache/hadoop/hbase/regionserver/HLog.java
    ===================================================================
    --- src/java/org/apache/hadoop/hbase/regionserver/HLog.java (revision 799653)
    +++ src/java/org/apache/hadoop/hbase/regionserver/HLog.java (working copy)
    @@ -860,7 +860,7 @@
    // Stop at logfiles.length when it's the last step
    int endIndex = step == maxSteps - 1? logfiles.length:
    step * concurrentLogReads + concurrentLogReads;
    - for (int i = (step * 10); i < endIndex; i++) {
    + for (int i = (step * concurrentLogReads); i < endIndex; i++) {
    // Check for possibly empty file. With appends, currently Hadoop
    // reports a zero length even if the file has been sync'd. Revisit if
    // HADOOP-4751 is committed.
    {code}
    When I changed it so we default to reading 3 files at a time rather than 10 over in hbase-1683, the hard-coding made it so we didn't read all logs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Aug 3, 2009 at 11:23 pm
    [ https://issues.apache.org/jira/browse/HBASE-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack resolved HBASE-1739.
    --------------------------

    Resolution: Fixed
    Fix Version/s: 0.20.0

    Applied trunk and 0.20 branch.
    hbase-1683 broke splitting; only split three logs no matter what N was.
    -----------------------------------------------------------------------

    Key: HBASE-1739
    URL: https://issues.apache.org/jira/browse/HBASE-1739
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Fix For: 0.20.0


    There's a hard-coding in HLog#splitLog that presumes we always read in batches of ten logs:
    {code}
    Index: src/java/org/apache/hadoop/hbase/regionserver/HLog.java
    ===================================================================
    --- src/java/org/apache/hadoop/hbase/regionserver/HLog.java (revision 799653)
    +++ src/java/org/apache/hadoop/hbase/regionserver/HLog.java (working copy)
    @@ -860,7 +860,7 @@
    // Stop at logfiles.length when it's the last step
    int endIndex = step == maxSteps - 1? logfiles.length:
    step * concurrentLogReads + concurrentLogReads;
    - for (int i = (step * 10); i < endIndex; i++) {
    + for (int i = (step * concurrentLogReads); i < endIndex; i++) {
    // Check for possibly empty file. With appends, currently Hadoop
    // reports a zero length even if the file has been sync'd. Revisit if
    // HADOOP-4751 is committed.
    {code}
    When I changed it so we default to reading 3 files at a time rather than 10 over in hbase-1683, the hard-coding made it so we didn't read all logs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jean-Daniel Cryans (JIRA) at Sep 3, 2009 at 5:29 pm
    [ https://issues.apache.org/jira/browse/HBASE-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jean-Daniel Cryans reassigned HBASE-1739:
    -----------------------------------------

    Assignee: stack
    hbase-1683 broke splitting; only split three logs no matter what N was.
    -----------------------------------------------------------------------

    Key: HBASE-1739
    URL: https://issues.apache.org/jira/browse/HBASE-1739
    Project: Hadoop HBase
    Issue Type: Bug
    Reporter: stack
    Assignee: stack
    Fix For: 0.20.0


    There's a hard-coding in HLog#splitLog that presumes we always read in batches of ten logs:
    {code}
    Index: src/java/org/apache/hadoop/hbase/regionserver/HLog.java
    ===================================================================
    --- src/java/org/apache/hadoop/hbase/regionserver/HLog.java (revision 799653)
    +++ src/java/org/apache/hadoop/hbase/regionserver/HLog.java (working copy)
    @@ -860,7 +860,7 @@
    // Stop at logfiles.length when it's the last step
    int endIndex = step == maxSteps - 1? logfiles.length:
    step * concurrentLogReads + concurrentLogReads;
    - for (int i = (step * 10); i < endIndex; i++) {
    + for (int i = (step * concurrentLogReads); i < endIndex; i++) {
    // Check for possibly empty file. With appends, currently Hadoop
    // reports a zero length even if the file has been sync'd. Revisit if
    // HADOOP-4751 is committed.
    {code}
    When I changed it so we default to reading 3 files at a time rather than 10 over in hbase-1683, the hard-coding made it so we didn't read all logs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post

1 user in discussion

Jean-Daniel Cryans (JIRA): 4 posts