FAQ
I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:

InternalScanner rootScanner =
root.getScanner(scan);

try {
List<KeyValue> results = new ArrayList<KeyValue>();
while(rootScanner.next(results)) {
for(KeyValue kv: results) {
HRegionInfo info = Writables.getHRegionInfoOrNull(kv.getValue());
if (info != null) {
metaRegions.add(info);
}
}
}
} finally {
...
}

That call to InternalScanner.next() in the while condition returns true if there's another result *after* the one it just loaded into the out param. That is, after it reads the last row into the 'results' collection, it returns false and the loop exits with that last row unread. It probably wants to be structured more like this:

final InternalScanner metaScanner = meta.getScanner(scan);
List<KeyValue> results = Lists.newArrayList();

while (true) {

boolean hasMore = metaScanner.next(results);

for (KeyValue kv : results) {
HRegionInfo hri = Writables.getHRegionInfoOrNull(kv.getValue());
if (hri != null) {
regionInfo.add(hri);
}
}

if (!hasMore) {
break;
}
}

I get the impression this class doesn't get used much, but just thought I'd point it out.

Thanks,
Sandy

Search Discussions

  • Stack at Feb 3, 2012 at 5:08 am

    On Thu, Feb 2, 2012 at 3:05 PM, Sandy Pratt wrote:
    I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:
    Agree

    I get the impression this class doesn't get used much, but just thought I'd point it out.
    Any chance of a JIRA and a patch Sandy? (You are right that stuff
    doesn't get exercised much -- we need to finish off the online merge
    scripts -- are you working on somethink like this?)

    St.Ack
  • Sandy Pratt at Feb 3, 2012 at 8:22 pm
    Filed one, but looks like it's already fixed on trunk.

    -----Original Message-----
    From: saint.ack@gmail.com On Behalf Of Stack
    Sent: Thursday, February 02, 2012 9:08 PM
    To: dev@hbase.apache.org
    Subject: Re: OBOB in HMerge?
    On Thu, Feb 2, 2012 at 3:05 PM, Sandy Pratt wrote:
    I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:
    Agree

    I get the impression this class doesn't get used much, but just thought I'd point it out.
    Any chance of a JIRA and a patch Sandy? (You are right that stuff doesn't get exercised much -- we need to finish off the online merge scripts -- are you working on somethink like this?)

    St.Ack

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedFeb 2, '12 at 11:08p
activeFeb 3, '12 at 8:22p
posts3
users2
websitehbase.apache.org

2 users in discussion

Sandy Pratt: 2 posts Stack: 1 post

People

Translate

site design / logo © 2022 Grokbase