Grokbase › Groups › HBase › dev › March 2011
FAQ
Hi,
Please refer to 'One of the regionserver aborted, then the master shut down
itself' on user list for background.

I looked at calls to this method:
public void assignUserRegions(List<HRegionInfo> regions, List<HServerInfo>
servers)
Here is one from AssignmentManager:
assignUserRegions(Arrays.asList(regions.keySet().toArray(new
HRegionInfo[0])), servers);
Here is another from Master:
this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
servers);

I propose changing the first parameter to HRegionInfo[] so that extra
copying can be avoided - considering that the number of regions in
production is high.

A bigger task is to reduce memory footprint of AssignmentManager.

I will present my further findings.

Thanks

Search Discussions

  • Stack at Mar 16, 2011 at 5:28 pm ⇧
    Will that eliminate copying Ted? If so, go for it (Thanks for looking
    into this).
    St.Ack
    On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu wrote:
    Hi,
    Please refer to 'One of the regionserver aborted, then the master shut down
    itself' on user list for background.

    I looked at calls to this method:
    public void assignUserRegions(List<HRegionInfo> regions, List<HServerInfo>
    servers)
    Here is one from AssignmentManager:
    assignUserRegions(Arrays.asList(regions.keySet().toArray(new
    HRegionInfo[0])), servers);
    Here is another from Master:
    this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
    servers);

    I propose changing the first parameter to HRegionInfo[] so that extra
    copying can be avoided - considering that the number of regions in
    production is high.

    A bigger task is to reduce memory footprint of AssignmentManager.

    I will present my further findings.

    Thanks
  • Ted Yu at Mar 16, 2011 at 6:13 pm ⇧
    It would eliminate this copy:

    165 public static <T> List<T> asList(T... array) {
    166 return new ArrayList<T>(array);
    167 }
    168
    On Wed, Mar 16, 2011 at 10:27 AM, Stack wrote:

    Will that eliminate copying Ted? If so, go for it (Thanks for looking
    into this).
    St.Ack
    On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu wrote:
    Hi,
    Please refer to 'One of the regionserver aborted, then the master shut down
    itself' on user list for background.

    I looked at calls to this method:
    public void assignUserRegions(List<HRegionInfo> regions,
    List<HServerInfo>
    servers)
    Here is one from AssignmentManager:
    assignUserRegions(Arrays.asList(regions.keySet().toArray(new
    HRegionInfo[0])), servers);
    Here is another from Master:
    this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
    servers);

    I propose changing the first parameter to HRegionInfo[] so that extra
    copying can be avoided - considering that the number of regions in
    production is high.

    A bigger task is to reduce memory footprint of AssignmentManager.

    I will present my further findings.

    Thanks
  • Ted Yu at Mar 16, 2011 at 6:36 pm ⇧
    Patch has been uploaded to https://issues.apache.org/jira/browse/HBASE-3657
    On Wed, Mar 16, 2011 at 11:12 AM, Ted Yu wrote:

    It would eliminate this copy:

    165 public static <T> List<T> asList(T... array) {
    166 return new ArrayList<T>(array);
    167 }
    168
    On Wed, Mar 16, 2011 at 10:27 AM, Stack wrote:

    Will that eliminate copying Ted? If so, go for it (Thanks for looking
    into this).
    St.Ack
    On Wed, Mar 16, 2011 at 9:30 AM, Ted Yu wrote:
    Hi,
    Please refer to 'One of the regionserver aborted, then the master shut down
    itself' on user list for background.

    I looked at calls to this method:
    public void assignUserRegions(List<HRegionInfo> regions,
    List<HServerInfo>
    servers)
    Here is one from AssignmentManager:
    assignUserRegions(Arrays.asList(regions.keySet().toArray(new
    HRegionInfo[0])), servers);
    Here is another from Master:
    this.assignmentManager.assignUserRegions(Arrays.asList(newRegions),
    servers);

    I propose changing the first parameter to HRegionInfo[] so that extra
    copying can be avoided - considering that the number of regions in
    production is high.

    A bigger task is to reduce memory footprint of AssignmentManager.

    I will present my further findings.

    Thanks

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedMar 16, '11 at 4:33p
activeMar 16, '11 at 6:36p
posts4
users2
websitehbase.apache.org

2 users in discussion

Ted Yu: 3 posts Stack: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase