FAQ

[HBase-dev] Loaded Coprocessor

Js_84
Feb 7, 2011 at 9:48 am
Hi,
is there any way to check which Coprocessors Classes are currently loaded? Preferebly using the shell...
And related: Can one dynamically load Coprocessors for a running hbase system?

Thanks for your support!
JS
reply

Search Discussions

24 responses

  • Sebastian Bauer at Feb 7, 2011 at 3:50 pm
    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and pass
    tests :)

    Regards,
    SB
  • Sebastian Bauer at Feb 7, 2011 at 3:57 pm

    On 07.02.2011 17:43, Sebastian Bauer wrote:
    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and
    pass tests :)

    Regards,
    SB
    Path-2 had small bug that balancer wasn't turn on again, so this is new
    url: http://paste.org/pastebin/view/28530

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Ted Yu at Feb 7, 2011 at 4:01 pm
    In HRegion.java:
    final AtomicInteger requestsCount = new AtomicInteger(0);

    AtomicLong should be used.
    On Mon, Feb 7, 2011 at 8:43 AM, Sebastian Bauer wrote:

    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and pass
    tests :)

    Regards,
    SB
  • Sebastian Bauer at Feb 7, 2011 at 4:17 pm
    then why hbase have this?

    in HRegionServer.java:206
    // Request counter
    private volatile AtomicInteger requestCount = new AtomicInteger();
    On 07.02.2011 17:00, Ted Yu wrote:
    In HRegion.java:
    final AtomicInteger requestsCount = new AtomicInteger(0);

    AtomicLong should be used.

    On Mon, Feb 7, 2011 at 8:43 AM, Sebastian Bauerwrote:
    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and pass
    tests :)

    Regards,
    SB

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Sebastian Bauer at Feb 7, 2011 at 4:31 pm
    path-1 with AtomicLong:
    http://paste.org/pastebin/view/28532
    On 07.02.2011 17:16, Sebastian Bauer wrote:
    then why hbase have this?

    in HRegionServer.java:206
    // Request counter
    private volatile AtomicInteger requestCount = new AtomicInteger();
    On 07.02.2011 17:00, Ted Yu wrote:
    In HRegion.java:
    final AtomicInteger requestsCount = new AtomicInteger(0);

    AtomicLong should be used.

    On Mon, Feb 7, 2011 at 8:43 AM, Sebastian Bauer<admin@ugame.net.pl>
    wrote:
    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and pass
    tests :)

    Regards,
    SB

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Tsuna at Feb 7, 2011 at 4:45 pm
    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Stack at Feb 7, 2011 at 5:07 pm
    Benoît:

    Stick the below up in an issue. Its good stuff.

    St.Ack
    On Mon, Feb 7, 2011 at 8:44 AM, tsuna wrote:
    Hey this is pretty cool, I've been wanting something like that for a
    while.  It's a little bit rough but the idea is there.  Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster.  The region stays down for several seconds (!).  This is
    really disruptive for high-throughput low-latency user-facing
    applications.  So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime.  It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region.  The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID).  The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS.  Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests.  The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS".  From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Sebastian Bauer at Feb 7, 2011 at 5:27 pm
    JIRA created: https://issues.apache.org/jira/browse/HBASE-3507
    On 07.02.2011 18:07, Stack wrote:
    Benoît:

    Stick the below up in an issue. Its good stuff.

    St.Ack

    On Mon, Feb 7, 2011 at 8:44 AM, tsunawrote:
    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Ted Yu at Feb 7, 2011 at 5:12 pm
    This is a good suggestion.
    We also need to consider the situation where source RS is under heavy load,
    facing potential GC pause. In this case we shouldn't expect source RS to
    carry on too much computation.
    On Mon, Feb 7, 2011 at 8:44 AM, tsuna wrote:

    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Tsuna at Feb 7, 2011 at 6:03 pm

    On Mon, Feb 7, 2011 at 9:12 AM, Ted Yu wrote:
    This is a good suggestion.
    We also need to consider the situation where source RS is under heavy load,
    facing potential GC pause. In this case we shouldn't expect source RS to
    carry on too much computation.
    Whenever things fail unexpectedly, we can always fall back to the
    current way of migrating regions (expensive for clients but easier for
    the server).

    Note that the problem is also valid if the destination RS is having
    load / GC issues. But hopefully once we expose proper load metrics,
    the master will be able to make good load balancing decisions. If the
    master has information such as QPS per region, latency per region,
    number of GC cycles per GC type, then it can get a pretty clear
    picture of the load of each RS and try to even it out.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Ted Yu at Feb 7, 2011 at 6:26 pm
    About 'number of GC cycles per GC type', is there API to get this
    information ?

    Regards
    On Mon, Feb 7, 2011 at 10:02 AM, tsuna wrote:
    On Mon, Feb 7, 2011 at 9:12 AM, Ted Yu wrote:
    This is a good suggestion.
    We also need to consider the situation where source RS is under heavy load,
    facing potential GC pause. In this case we shouldn't expect source RS to
    carry on too much computation.
    Whenever things fail unexpectedly, we can always fall back to the
    current way of migrating regions (expensive for clients but easier for
    the server).

    Note that the problem is also valid if the destination RS is having
    load / GC issues. But hopefully once we expose proper load metrics,
    the master will be able to make good load balancing decisions. If the
    master has information such as QPS per region, latency per region,
    number of GC cycles per GC type, then it can get a pretty clear
    picture of the load of each RS and try to even it out.

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Tsuna at Feb 7, 2011 at 6:50 pm

    On Mon, Feb 7, 2011 at 10:26 AM, Ted Yu wrote:
    About 'number of GC cycles per GC type', is there API to get this
    information ?
    As far as I know this is only exposed through JMX. I'm sure there is
    a way to use some horrible APIs in the JDK to "efficiently" access JMX
    stuff from the local JVM. If there's another way than JMX, I'd love
    to know.

    The way I collect HBase metrics for OpenTSDB is to use the Sun private
    API to attach to the JVM and get a JXM URL, even if the VM wasn't
    configured to expose any JMX thing on a port. I think under the hood
    it starts a thread in the remote JVM to handle RMI calls that probably
    go over the loopback interface. This is pretty much how jconsole and
    friends work, BTW. Pretty horrible stuff. :(

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Ted Yu at Feb 7, 2011 at 6:58 pm
    This means we need horrible stuff to write good stuff :-)
    On Mon, Feb 7, 2011 at 10:49 AM, tsuna wrote:
    On Mon, Feb 7, 2011 at 10:26 AM, Ted Yu wrote:
    About 'number of GC cycles per GC type', is there API to get this
    information ?
    As far as I know this is only exposed through JMX. I'm sure there is
    a way to use some horrible APIs in the JDK to "efficiently" access JMX
    stuff from the local JVM. If there's another way than JMX, I'd love
    to know.

    The way I collect HBase metrics for OpenTSDB is to use the Sun private
    API to attach to the JVM and get a JXM URL, even if the VM wasn't
    configured to expose any JMX thing on a port. I think under the hood
    it starts a thread in the remote JVM to handle RMI calls that probably
    go over the loopback interface. This is pretty much how jconsole and
    friends work, BTW. Pretty horrible stuff. :(

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Todd Lipcon at Feb 7, 2011 at 7:00 pm
    http://download.oracle.com/javase/1.5.0/docs/api/java/lang/management/ManagementFactory.html#getGarbageCollectorMXBeans()
    <http://download.oracle.com/javase/1.5.0/docs/api/java/lang/management/ManagementFactory.html#getGarbageCollectorMXBeans()>exposes
    some of the info

    Unfortunately you can't get at any of the really juicy details like CMS FLS
    statistics :(

    -Todd
    On Mon, Feb 7, 2011 at 10:49 AM, tsuna wrote:
    On Mon, Feb 7, 2011 at 10:26 AM, Ted Yu wrote:
    About 'number of GC cycles per GC type', is there API to get this
    information ?
    As far as I know this is only exposed through JMX. I'm sure there is
    a way to use some horrible APIs in the JDK to "efficiently" access JMX
    stuff from the local JVM. If there's another way than JMX, I'd love
    to know.

    The way I collect HBase metrics for OpenTSDB is to use the Sun private
    API to attach to the JVM and get a JXM URL, even if the VM wasn't
    configured to expose any JMX thing on a port. I think under the hood
    it starts a thread in the remote JVM to handle RMI calls that probably
    go over the loopback interface. This is pretty much how jconsole and
    friends work, BTW. Pretty horrible stuff. :(

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com


    --
    Todd Lipcon
    Software Engineer, Cloudera
  • Tsuna at Feb 7, 2011 at 8:03 pm

    On Mon, Feb 7, 2011 at 10:59 AM, Todd Lipcon wrote:
    Unfortunately you can't get at any of the really juicy details like CMS FLS
    statistics :(
    Do you have to resort to reading back your own GC log to figure this
    stuff out? :(

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Ted Yu at Feb 7, 2011 at 9:12 pm
    This approach would introduce dependency on the underlying JVM - vendor,
    version, etc because the log format can change across JVM versions.
    On Mon, Feb 7, 2011 at 12:03 PM, tsuna wrote:
    On Mon, Feb 7, 2011 at 10:59 AM, Todd Lipcon wrote:
    Unfortunately you can't get at any of the really juicy details like CMS FLS
    statistics :(
    Do you have to resort to reading back your own GC log to figure this
    stuff out? :(

    --
    Benoit "tsuna" Sigoure
    Software Engineer @ www.StumbleUpon.com
  • Sebastian Bauer at Feb 7, 2011 at 5:15 pm
    Yeah i know move take a while but we have small cluster(3 machines) and
    when few hot regions land in one machine then we had about
    4000/1000/1000 requests on machines after a rebalancing we have now
    about 1900/2200/1900 so its worked :)

    maybe someone can write tool to take few regions from overloaded
    regionserver and put on the others, but i have no skills to write it in
    ruby ;)
    On 07.02.2011 17:44, tsuna wrote:
    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Ted Yu at Feb 7, 2011 at 5:21 pm
    What version of HBase do you use ?
    Looking at the line numbers, you are using version older than 0.90

    Also, incrementing counter in startRegionOperation() doesn't consider the
    following call (in 0.90 RC3):
    public Integer obtainRowLock(final byte [] row) throws IOException {
    startRegionOperation();

    Regards
    On Mon, Feb 7, 2011 at 9:15 AM, Sebastian Bauer wrote:

    Yeah i know move take a while but we have small cluster(3 machines) and
    when few hot regions land in one machine then we had about 4000/1000/1000
    requests on machines after a rebalancing we have now about 1900/2200/1900 so
    its worked :)

    maybe someone can write tool to take few regions from overloaded
    regionserver and put on the others, but i have no skills to write it in ruby
    ;)

    On 07.02.2011 17:44, tsuna wrote:

    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.
    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Sebastian Bauer at Feb 7, 2011 at 5:26 pm
    its trunk version:

    Path: .
    URL: https://svn.apache.org/repos/asf/hbase/trunk
    Repository Root: https://svn.apache.org/repos/asf
    Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
    Revision: 1067865
    Node Kind: directory
    Schedule: normal
    Last Changed Author: stack
    Last Changed Rev: 1067598
    Last Changed Date: 2011-02-06 06:46:39 +0100 (Sun, 06 Feb 2011)
    On 07.02.2011 18:20, Ted Yu wrote:
    What version of HBase do you use ?
    Looking at the line numbers, you are using version older than 0.90

    Also, incrementing counter in startRegionOperation() doesn't consider the
    following call (in 0.90 RC3):
    public Integer obtainRowLock(final byte [] row) throws IOException {
    startRegionOperation();

    Regards

    On Mon, Feb 7, 2011 at 9:15 AM, Sebastian Bauerwrote:
    Yeah i know move take a while but we have small cluster(3 machines) and
    when few hot regions land in one machine then we had about 4000/1000/1000
    requests on machines after a rebalancing we have now about 1900/2200/1900 so
    its worked :)

    maybe someone can write tool to take few regions from overloaded
    regionserver and put on the others, but i have no skills to write it in ruby
    ;)

    On 07.02.2011 17:44, tsuna wrote:

    Hey this is pretty cool, I've been wanting something like that for a
    while. It's a little bit rough but the idea is there. Ultimately, I
    hope HBase will be able to provide fine-grained metrics on a
    per-region basis and use that to do load balancing.

    The problem right now is that load balancing is very costly for
    clients, because it takes too long to move a region around in a real
    cluster. The region stays down for several seconds (!). This is
    really disruptive for high-throughput low-latency user-facing
    applications. So until we make the region migration process more
    seamless, we can't really have a very aggressive / proactive load
    balancer.

    The other day I suggested an idea to Stack to change the region
    migration process with virtually zero downtime. It basically involves
    telling the source region server where the region is going to land,
    and telling the destination region server to prepare to receive the
    region. The source RS would do a first flush and remember the point
    at which it is (there's a generation ID or something already, whatever
    is needed for ACID). The source RS would send an RPC to the
    destination RS to tell it to start loading whatever was flushed and
    then it would replicate all the edits to the destination RS. Once
    both RS are in sync, the source RS would block requests to the region
    (by locking it), tell the destination RS about it, and after getting
    the final ACK from it, would update META and send a special NSRE to
    all the clients of the blocked requests. The special NSRE would
    basically just be like a normal NSRE but it would also say "hint: I
    think the region is now on that RS". From the clients' point of view,
    the region downtime would be pretty minimal (almost unnoticeable).
    Also, this scheme allows for opportunities such as warming up the
    block cache in the destination RS before it starts serving.
    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Stack at Feb 7, 2011 at 5:37 pm
    Sebastian:

    I was going to say please make an issue with this good stuff but I
    notice you just started in.

    Good on you Sebastian. Thanks,
    St.Ack


    On Mon, Feb 7, 2011 at 8:30 AM, Sebastian Bauer wrote:
    path-1 with AtomicLong:
    http://paste.org/pastebin/view/28532
    On 07.02.2011 17:16, Sebastian Bauer wrote:

    then why hbase have this?

    in HRegionServer.java:206
    // Request counter
    private volatile AtomicInteger requestCount = new AtomicInteger();
    On 07.02.2011 17:00, Ted Yu wrote:

    In HRegion.java:
    final AtomicInteger requestsCount = new AtomicInteger(0);

    AtomicLong should be used.

    On Mon, Feb 7, 2011 at 8:43 AM, Sebastian Bauer<admin@ugame.net.pl>
    wrote:
    Hi,

    Path-1 add another mertic for HRegion to count request made to region.

    Path-2 add another command to hbase shell to grab all regions, sort by
    requests from Path-1 and move in round-robin algorithm to servers


    Path-1: http://paste.org/pastebin/view/28526
    Path-2: http://paste.org/pastebin/view/28528


    PS. I'm not a java nor ruby coder but this code working for us and pass
    tests :)

    Regards,
    SB

    --

    Pozdrawiam
    Sebastian Bauer
    -----------------------------------------------------
    http://tikecik.pl
  • Mingjie Lai at Feb 7, 2011 at 7:33 pm
    Hi JS.
    is there any way to check which Coprocessors Classes are currently
    loaded? Preferebly using the shell...

    No. Not yet. But that's a reasonable requirement. You can open a jira
    for it. Right now you have to check HTableDescriptor (for table
    coprocessors) or configurations (for system-wide cp) to see loaded
    coprocessors.
    And related: Can one dynamically load Coprocessors for a running
    hbase system?

    Yes, and no. You can dynamically load a table-based coprocessor: 1)
    disable a table; 2) set an HTableDecriptor property; 3) enable the
    table. Please read ``Load from table attribute'' section of
    http://hbaseblog.com/2010/11/30/hbase-coprocessors/. But for system cp,
    you cannot do it right now.

    It's absolutely doable to have an easier way to dynamically load/unload
    coprocessor. But it might relate to security issue: you don't want
    anyone from your cluster to easily load/unload a customized cp without
    any limitation.

    There is still rooms to improve coprocessor loading/unloading mechanism.
    It will depend on how users want to use it. Welcome to provide your
    feedback and contribution.

    -Mingjie
    On 02/07/2011 01:48 AM, js_84@email.de wrote:
    Hi,
    is there any way to check which Coprocessors Classes are currently loaded? Preferebly using the shell...
    And related: Can one dynamically load Coprocessors for a running hbase system?

    Thanks for your support!
    JS
  • Andrew Purtell at Feb 7, 2011 at 10:43 pm

    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    Not yet. Good idea, filed: https://issues.apache.org/jira/browse/HBASE-3512
    And related: Can one dynamically load Coprocessors for a
    running hbase system?
    Currently coprocessors are loaded statically via configuration or dynamically upon table open if there are any table attributes that specify it (attribute name begins with "Coprocessor"). IIRC the javadoc on the coprocessors package contains detail on this. In short:

    HTableDescriptor htd;
    // ...
    htd.setValue("Coprocessor$foo", "hdfs://namenode:port/path/to/coprocessor.jar");

    A table descriptor can be updated if such specifications are not made a table create time. See javadoc for HBaseAdmin.

    However there is currently no mechanism for triggering additional loads on tables already open or loading at arbitrary times. There hasn't been a clearly articulated need for this yet.

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back.
    - Piet Hein (via Tom White)


    --- On Mon, 2/7/11, js_84@email.de wrote:
    From: js_84@email.de <js_84@email.de>
    Subject: Loaded Coprocessor
    To: dev@hbase.apache.org
    Date: Monday, February 7, 2011, 1:48 AM
    Hi,
    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    And related: Can one dynamically load Coprocessors for a
    running hbase system?

    Thanks for your support!
    JS
  • Js_84 at Feb 8, 2011 at 6:02 pm
    Hi,
    thanks for your feedback!

    Just a follow up question;
    According to the TableDescriptor
    {NAME => 'test3', Coprocessor$2 => '/Users/hbase/CP.jar:Coprocessor.PreGet:12', Coprocessor$1 => '/Users/hbase/CP.jar:Coprocessor.PrePut:10',
    FAMILIES => [{NAME => 'cf', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}

    the coprocessors is specified, but actually neither of them is executed when executing a put; So I am not sure whether they are actually loaded (what happens if if jar file is not found? -> this should trigger at least a warning).

    Any ideas/pointers why this fails (HBAse Version 0.91.0-SNAPSHOT, r1065918)?

    package Coprocessor;
    public class PrePut extends BaseRegionObserverCoprocessor {
    @Override
    public void prePut( RegionCoprocessorEnvironment e, Map<byte[],
    List<KeyValue>> familyMap, boolean writeToWAL) throws IOException {

    //bypass normal system call
    e.bypass();
    }
    }


    Thanks again!
    JS


    -----Ursprüngliche Nachricht-----
    Von: "Andrew Purtell" <apurtell@apache.org>
    Gesendet: 07.02.2011 23:42:35
    An: dev@hbase.apache.org
    Betreff: Re: Loaded Coprocessor
    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    Not yet. Good idea, filed: https://issues.apache.org/jira/browse/HBASE-3512
    And related: Can one dynamically load Coprocessors for a
    running hbase system?
    Currently coprocessors are loaded statically via configuration or dynamically upon table open if there are any table attributes that specify it (attribute name begins with "Coprocessor"). IIRC the javadoc on the coprocessors package contains detail on this. In short:

    HTableDescriptor htd;
    // ...
    htd.setValue("Coprocessor$foo", "hdfs://namenode:port/path/to/coprocessor.jar");

    A table descriptor can be updated if such specifications are not made a table create time. See javadoc for HBaseAdmin.

    However there is currently no mechanism for triggering additional loads on tables already open or loading at arbitrary times. There hasn't been a clearly articulated need for this yet.

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back.
    - Piet Hein (via Tom White)


    --- On Mon, 2/7/11, js_84@email.de wrote:
    From: js_84@email.de <js_84@email.de>
    Subject: Loaded Coprocessor
    To: dev@hbase.apache.org
    Date: Monday, February 7, 2011, 1:48 AM
    Hi,
    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    And related: Can one dynamically load Coprocessors for a
    running hbase system?

    Thanks for your support!
    JS
  • Mingjie Lai at Feb 8, 2011 at 8:12 pm
    Hi JS.

    Here are some hints to help you debug:
    - make sure you have the jar copied to all the region servers if you
    want it to sit on local fs, or you can put it to hdfs so all RS can
    access it without copying arount.
    - remember to disable, and enable the table to make sure the attribute
    be loaded
    - if you still cannot see it working, please post your RS log after the
    time of table enabled.

    Loading cp classes from file system lacks of testing, so there might be
    some issues. I added a jira:
    https://issues.apache.org/jira/browse/HBASE-3516 to add test cases back
    for class loading.

    Another workaround is to use configuration to test your coprocessor.

    This has been tested a lot among us. The only problem is that it would
    be applied to all your tables. You can use this method if there is an
    issue for cp class loading from jar files.

    Thanks,
    Mingjie

    On 02/08/2011 10:01 AM, js_84@email.de wrote:
    Hi,
    thanks for your feedback!

    Just a follow up question;
    According to the TableDescriptor
    {NAME => 'test3', Coprocessor$2 => '/Users/hbase/CP.jar:Coprocessor.PreGet:12', Coprocessor$1 => '/Users/hbase/CP.jar:Coprocessor.PrePut:10',
    FAMILIES => [{NAME => 'cf', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}

    the coprocessors is specified, but actually neither of them is executed when executing a put; So I am not sure whether they are actually loaded (what happens if if jar file is not found? -> this should trigger at least a warning).

    Any ideas/pointers why this fails (HBAse Version 0.91.0-SNAPSHOT, r1065918)?

    package Coprocessor;
    public class PrePut extends BaseRegionObserverCoprocessor {
    @Override
    public void prePut( RegionCoprocessorEnvironment e, Map<byte[],
    List<KeyValue>> familyMap, boolean writeToWAL) throws IOException {

    //bypass normal system call
    e.bypass();
    }
    }


    Thanks again!
    JS


    -----Ursprüngliche Nachricht-----
    Von: "Andrew Purtell"<apurtell@apache.org>
    Gesendet: 07.02.2011 23:42:35
    An: dev@hbase.apache.org
    Betreff: Re: Loaded Coprocessor
    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    Not yet. Good idea, filed: https://issues.apache.org/jira/browse/HBASE-3512
    And related: Can one dynamically load Coprocessors for a
    running hbase system?
    Currently coprocessors are loaded statically via configuration or dynamically upon table open if there are any table attributes that specify it (attribute name begins with "Coprocessor"). IIRC the javadoc on the coprocessors package contains detail on this. In short:

    HTableDescriptor htd;
    // ...
    htd.setValue("Coprocessor$foo", "hdfs://namenode:port/path/to/coprocessor.jar");

    A table descriptor can be updated if such specifications are not made a table create time. See javadoc for HBaseAdmin.

    However there is currently no mechanism for triggering additional loads on tables already open or loading at arbitrary times. There hasn't been a clearly articulated need for this yet.

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back.
    - Piet Hein (via Tom White)


    --- On Mon, 2/7/11, js_84@email.dewrote:
    From: js_84@email.de<js_84@email.de>
    Subject: Loaded Coprocessor
    To: dev@hbase.apache.org
    Date: Monday, February 7, 2011, 1:48 AM
    Hi,
    is there any way to check which Coprocessors Classes are
    currently loaded? Preferebly using the shell...
    And related: Can one dynamically load Coprocessors for a
    running hbase system?

    Thanks for your support!
    JS

Related Discussions