Grokbase Groups HBase user June 2011
FAQ
Hi folks,

I need to load 1 million queue messages into a hbase table in 30 mins.

As "HBase: The Definitive Guide" suggests, I use Client API, flushCommits().

I launched, say, 20 threads, each thread has its own queue connection and hbase instance, which read msg from queue and insert it into hbase. At the end of thread life, it will call the hbaseTable.flushCommits().

It seems working fine, except for:

1. Each flushCommit action takes quite a long time.
2. Occasionally, flushCommit cause WrongRegionException.

Can someone please share the best practice in this situation? Especially, should each thread call flushCommit() or only the main thread call it?

Thanks,

RX

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 3 | next ›
Discussion Overview
groupuser @
categorieshbase, hadoop
postedJun 1, '11 at 1:35p
activeJun 1, '11 at 6:28p
posts3
users2
websitehbase.apache.org

2 users in discussion

Xu, Richard: 2 posts Jean-Daniel Cryans: 1 post

People

Translate

site design / logo © 2022 Grokbase