FAQ
Hi again,

we have a 1 master, 3 data nodes Hadoop+HBase cluster for a PoC. I ran
into "Too many open files" errors on the region server during load
testing. No problem as such. But now, after shutting down and starting
up again, when trying to count how many rows actually made it,
bin/hadoop jar ../hbase/hbase-0.20.6.jar rowcounter tir_items
I get

Exception in thread "main"
org.apache.hadoop.hbase.client.NoServerForRegionException: No server
address listed in .META. for region
tir_items,customer/0/FB032E3983F42455E0ACCFE61A0C3385A37975A42439430F0993D4D3DC5F76E,1289565977870
at org.apache.hadoop.hbase.client.HConnectionManager
$TableServers.locateRegionInMeta(HConnectionManager.java:726)
at org.apache.hadoop.hbase.client.HConnectionManager
$TableServers.locateRegion(HConnectionManager.java:634)
at org.apache.hadoop.hbase.client.HConnectionManager
$TableServers.locateRegion(HConnectionManager.java:601)
at org.apache.hadoop.hbase.client.HConnectionManager
$TableServers.getRegionLocation(HConnectionManager.java:428)
at
org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:207)
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:289)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at
org.apache.hadoop.hbase.mapreduce.RowCounter.main(RowCounter.java:126)
...

This situation persists even after reboot. Any idea how to fix this?
(All other advice I found says that things should be
fine after restart at latest).

Thanks,
Henning

Search Discussions

  • Hari Sreekumar at Nov 12, 2010 at 2:26 pm
    Hi Henning,

    Try doing flush '.META.' and major_compact '.META.' in the hbase
    shell. It worked once for me. I hope it helps.

    Cheers,
    hari
    On Fri, Nov 12, 2010 at 6:43 PM, Henning Blohm wrote:

    Hi again,

    we have a 1 master, 3 data nodes Hadoop+HBase cluster for a PoC. I ran
    into "Too many open files" errors on the region server during load
    testing. No problem as such. But now, after shutting down and starting
    up again, when trying to count how many rows actually made it,
    bin/hadoop jar ../hbase/hbase-0.20.6.jar rowcounter tir_items
    I get

    Exception in thread "main"
    org.apache.hadoop.hbase.client.NoServerForRegionException: No server
    address listed in .META. for region

    tir_items,customer/0/FB032E3983F42455E0ACCFE61A0C3385A37975A42439430F0993D4D3DC5F76E,1289565977870
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegionInMeta(HConnectionManager.java:726)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegion(HConnectionManager.java:634)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegion(HConnectionManager.java:601)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.getRegionLocation(HConnectionManager.java:428)
    at
    org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:207)
    at

    org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:289)
    at
    org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
    at
    org.apache.hadoop.hbase.mapreduce.RowCounter.main(RowCounter.java:126)
    ...

    This situation persists even after reboot. Any idea how to fix this?
    (All other advice I found says that things should be
    fine after restart at latest).

    Thanks,
    Henning

  • Henning Blohm at Nov 12, 2010 at 3:37 pm
    Hi Hari,

    amazing, I tried that before (just to say that I googled before posting
    the question :-)
    and it didn't seem to help but now the job got scheduled again and
    started counting again....

    Thanks!
    Henning



    Am Freitag, den 12.11.2010, 19:56 +0530 schrieb Hari Sreekumar:
    Hi Henning,

    Try doing flush '.META.' and major_compact '.META.' in the hbase
    shell. It worked once for me. I hope it helps.

    Cheers,
    hari
    On Fri, Nov 12, 2010 at 6:43 PM, Henning Blohm wrote:

    Hi again,

    we have a 1 master, 3 data nodes Hadoop+HBase cluster for a PoC. I ran
    into "Too many open files" errors on the region server during load
    testing. No problem as such. But now, after shutting down and starting
    up again, when trying to count how many rows actually made it,
    bin/hadoop jar ../hbase/hbase-0.20.6.jar rowcounter tir_items
    I get

    Exception in thread "main"
    org.apache.hadoop.hbase.client.NoServerForRegionException: No server
    address listed in .META. for region

    tir_items,customer/0/FB032E3983F42455E0ACCFE61A0C3385A37975A42439430F0993D4D3DC5F76E,1289565977870
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegionInMeta(HConnectionManager.java:726)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegion(HConnectionManager.java:634)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.locateRegion(HConnectionManager.java:601)
    at org.apache.hadoop.hbase.client.HConnectionManager
    $TableServers.getRegionLocation(HConnectionManager.java:428)
    at
    org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:207)
    at

    org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:289)
    at
    org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
    at
    org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
    at
    org.apache.hadoop.hbase.mapreduce.RowCounter.main(RowCounter.java:126)
    ...

    This situation persists even after reboot. Any idea how to fix this?
    (All other advice I found says that things should be
    fine after restart at latest).

    Thanks,
    Henning

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedNov 12, '10 at 1:13p
activeNov 12, '10 at 3:37p
posts3
users2
websitehbase.apache.org

2 users in discussion

Henning Blohm: 2 posts Hari Sreekumar: 1 post

People

Translate

site design / logo © 2022 Grokbase