Grokbase Groups HBase dev March 2011
FAQ
Hi,
I have a question about TestRegionRebalancing.assertRegionsAreBalanced()
I see hardcoded slop of 0.1
In 0.90 and beyond, sloppiness is gone.
Therefore:
// TODO: Fix this test. Old balancer used to run with 'slop'. New
// balancer does not.
I want to get your opinion on how this test should be rewritten.

Thanks

Search Discussions

  • Stack at Mar 20, 2011 at 2:33 am

    On Sat, Mar 19, 2011 at 2:52 PM, Ted Yu wrote:
    Hi,
    I have a question about TestRegionRebalancing.assertRegionsAreBalanced()
    I see hardcoded slop of 0.1
    In 0.90 and beyond, sloppiness is gone.
    Therefore:
    // TODO: Fix this test.  Old balancer used to run with 'slop'.  New
    // balancer does not.
    I want to get your opinion on how this test should be rewritten.
    Yes slop is gone.

    But that method still uses 'slop'. IIRC -- and the method javadoc
    agrees -- the new balancer should make it so we are +/- 1 either side
    of the 'average'. The 'slop' here is being used to do the +/- 1.

    You could change the method to do +/- 1 explicitly and clean up dead
    code (we set slop == 0.1 and then test for <= 0 which is unnecessary).

    Or just clean dead code and add comments about what 'slop' means in
    this context.

    St.Ack
  • Ted Yu at Mar 20, 2011 at 4:44 am
    This test was failing after I made AssignmentManager.getAssignments() filter
    out regions whose request count was zero.
    The rationale is that table created with multiple regions may hold some
    empty regions for a while because user may not precisely predict the range
    of row keys at the time of table creation.
    TestRegionRebalancing uses the following to obtain average load:
    double avg = cluster.getMaster().getServerManager().getAverageLoad();
    However, considering hbase-3676, ServerManager wouldn't be the best
    candidate to give master such information.

    I am thinking of moving the above method to AssignmentManager:
    public double getAverageLoad()
    so that AssignmentManager becomes the single class that is aware of details
    of region load.

    Please comment.
    On Sat, Mar 19, 2011 at 7:33 PM, Stack wrote:
    On Sat, Mar 19, 2011 at 2:52 PM, Ted Yu wrote:
    Hi,
    I have a question about TestRegionRebalancing.assertRegionsAreBalanced()
    I see hardcoded slop of 0.1
    In 0.90 and beyond, sloppiness is gone.
    Therefore:
    // TODO: Fix this test. Old balancer used to run with 'slop'. New
    // balancer does not.
    I want to get your opinion on how this test should be rewritten.
    Yes slop is gone.

    But that method still uses 'slop'. IIRC -- and the method javadoc
    agrees -- the new balancer should make it so we are +/- 1 either side
    of the 'average'. The 'slop' here is being used to do the +/- 1.

    You could change the method to do +/- 1 explicitly and clean up dead
    code (we set slop == 0.1 and then test for <= 0 which is unnecessary).

    Or just clean dead code and add comments about what 'slop' means in
    this context.

    St.Ack

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedMar 19, '11 at 9:52p
activeMar 20, '11 at 4:44a
posts3
users2
websitehbase.apache.org

2 users in discussion

Ted Yu: 2 posts Stack: 1 post

People

Translate

site design / logo © 2022 Grokbase