Grokbase Groups HBase dev July 2009
FAQ
client is broken, it requests ROOT region location from ZK too much
-------------------------------------------------------------------

Key: HBASE-1651
URL: https://issues.apache.org/jira/browse/HBASE-1651
Project: Hadoop HBase
Issue Type: Bug
Affects Versions: 0.20.0
Reporter: ryan rawson
Assignee: stack
Priority: Blocker
Fix For: 0.20.0


something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:

09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Nitay Joffe (JIRA) at Jul 14, 2009 at 3:56 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730674#action_12730674 ]

    Nitay Joffe commented on HBASE-1651:
    ------------------------------------

    Was chatting with Stack about this. This brings up a good point, which is that we don't cache ZK results _anywhere_. The assumption we've had when writing all of this stuff was that we already did caching, so it should still work. For example, with the ROOT region, we used to get it from the master. I assumed the code previously would get it once, and only query the master for it again when it couldn't talk to ROOT anymore. So, by replacing it with ZK it should've still done the same caching. Clearly, either that logic wasn't right, or I broke it when we put the ZK stuff in. We should check all of our ZK queries, I have a feeling this is not the only place we'll find such a thing.

    Good catch Ryan.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 3:58 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730676#action_12730676 ]

    stack commented on HBASE-1651:
    ------------------------------

    So, this log only shows if you enable DEBUG on zookeeper.

    Looking in ZKW (and chatting w/ nitay), yeah, each log is a trip to ZK. Would explain hbase-1614.

    Taking a look...
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 4:18 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730679#action_12730679 ]

    stack commented on HBASE-1651:
    ------------------------------

    Where you seeing this logging come out Ryan? I don't see it in client or server.

    I do see that we seem to clear cache too much. On a NSRE, we relookup root and meta which seems unnecessary:

    {code}
    2009-07-14 04:10:10,282 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Removed .META.,,1 for tableName=.META. from cache because of TestTable,\x01\x00\x01\x06\x00\x03\x01\x02\x06\x06,99999999999999
    2009-07-14 04:10:10,282 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: LOCATEROOTREGION
    2009-07-14 04:10:10,305 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got XX.XX.XX.140:60020
    2009-07-14 04:10:10,309 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Found ROOT at XX.XX.XX.140:60020
    2009-07-14 04:10:10,318 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Cached location address: XX.XX.XX.141:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE => '16384', FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION => 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}
    {code}
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • ryan rawson (JIRA) at Jul 14, 2009 at 4:29 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730683#action_12730683 ]

    ryan rawson commented on HBASE-1651:
    ------------------------------------

    this is on the client,the log4j is:

    log4j.logger.org.apache.hadoop.hbase=DEBUG

    This is new in the last few dozen commits, it didn't use to spew, the only difference is grabbing your new client code.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • ryan rawson (JIRA) at Jul 14, 2009 at 4:31 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730684#action_12730684 ]

    ryan rawson commented on HBASE-1651:
    ------------------------------------

    oh btw another critical aspect is I'm committing to a table with 2k regions.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 5:43 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730701#action_12730701 ]

    stack commented on HBASE-1651:
    ------------------------------

    Without below, we never cache root location -- we just get it every time:
    {code}
    Index: src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
    ===================================================================
    --- src/java/org/apache/hadoop/hbase/client/HConnectionManager.java (revision 793429)
    +++ src/java/org/apache/hadoop/hbase/client/HConnectionManager.java (working copy)
    @@ -512,7 +514,7 @@
    // second waits. The second thread will not do find.

    if (!useCache || rootRegionLocation == null) {
    - return locateRootRegion();
    + rootRegionLocation = locateRootRegion();
    }
    return rootRegionLocation;
    }
    {code}
    Above seems safe enough. Servers still get to set root location if they want.

    The above 'fix' used to be in place. Was removed here with this commit:

    {code}
    ------------------------------------------------------------------------
    r736371 | jimk | 2009-01-21 10:42:31 -0800 (Wed, 21 Jan 2009) | 1 line

    HBASE-1121 Cluster confused about where -ROOT- is. This patch fixes "re-finding" the root region.
    {code}

    Here is patch that removed it:

    {code}
    svn diff -r736371:737213 src/java/org/apache/hadoop/hbase/client/HConnectionManager.java > /tmp/diff.txt
    Index: src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
    ===================================================================
    --- src/java/org/apache/hadoop/hbase/client/HConnectionManager.java (revision 736371)
    +++ src/java/org/apache/hadoop/hbase/client/HConnectionManager.java (revision 737213)
    @@ -134,6 +134,7 @@
    private final Map<String, HRegionInterface> servers =
    new ConcurrentHashMap<String, HRegionInterface>();

    + // Used by master and region servers during safe mode only
    private volatile HRegionLocation rootRegionLocation;

    private final Map<Integer, SoftValueSortedMap<byte [], HRegionLocation>>
    @@ -177,10 +178,12 @@
    return this.pause * HConstants.RETRY_BACKOFF[ntries];
    }

    + // Used by master and region servers during safe mode only
    public void unsetRootRegionLocation() {
    this.rootRegionLocation = null;
    }

    + // Used by master and region servers during safe mode only
    public void setRootRegionLocation(HRegionLocation rootRegion) {
    if (rootRegion == null) {
    throw new IllegalArgumentException(
    @@ -445,7 +448,7 @@
    // second waits. The second thread will not do find.

    if (!useCache || rootRegionLocation == null) {
    - this.rootRegionLocation = locateRootRegion();
    + return locateRootRegion();
    }
    return rootRegionLocation;
    }
    {code}
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 5:53 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HBASE-1651:
    -------------------------

    Attachment: hbase-1651.patch

    Cache root in HConnectionManager. This was undone a while ago. As I read it, it should be fine putting it back. Will test.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • ryan rawson (JIRA) at Jul 14, 2009 at 6:01 am
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730707#action_12730707 ]

    ryan rawson commented on HBASE-1651:
    ------------------------------------

    +1 on this. I get less messages, and i took down the ROOT regionserver and we recovered ok.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Andrew Purtell (JIRA) at Jul 14, 2009 at 6:05 pm
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731004#action_12731004 ]

    Andrew Purtell commented on HBASE-1651:
    ---------------------------------------

    +1 tested this also.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 6:11 pm
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack resolved HBASE-1651.
    --------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]

    Tested too. I killed root on loaded cluster and I then killed server with meta so clients would have to go back to find new root location. I had a running shell with root cached. Eventually it found the new root location.

    Resolving.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 14, 2009 at 6:39 pm
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731031#action_12731031 ]

    stack commented on HBASE-1651:
    ------------------------------

    I think we make too many requests of .META. too. If we can't find an entry in the .META. for a particular region -- it just NSRE'd because it split or something -- then we purge the .META. from cache. That seems wrong. Its not like getting .META. again is going to give you a different response.
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Jul 17, 2009 at 6:25 pm
    [ https://issues.apache.org/jira/browse/HBASE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732665#action_12732665 ]

    stack commented on HBASE-1651:
    ------------------------------

    Backported to branch
    client is broken, it requests ROOT region location from ZK too much
    -------------------------------------------------------------------

    Key: HBASE-1651
    URL: https://issues.apache.org/jira/browse/HBASE-1651
    Project: Hadoop HBase
    Issue Type: Bug
    Affects Versions: 0.20.0
    Reporter: ryan rawson
    Assignee: stack
    Priority: Blocker
    Fix For: 0.20.0

    Attachments: hbase-1651.patch


    something bad happened to the client, now it requests the ROOT region location literally a hundred times a second:
    09/07/13 18:39:59 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:00 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    09/07/13 18:40:01 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.20.20.158:60020
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJul 14, '09 at 1:42a
activeJul 17, '09 at 6:25p
posts13
users1
websitehbase.apache.org

1 user in discussion

stack (JIRA): 13 posts

People

Translate

site design / logo © 2022 Grokbase