Grokbase Groups HBase user June 2011
Hi folks,

I need to load 1 million queue messages into a hbase table in 30 mins.

As "HBase: The Definitive Guide" suggests, I use Client API, flushCommits().

I launched, say, 20 threads, each thread has its own queue connection and hbase instance, which read msg from queue and insert it into hbase. At the end of thread life, it will call the hbaseTable.flushCommits().

It seems working fine, except for:

1. Each flushCommit action takes quite a long time.
2. Occasionally, flushCommit cause WrongRegionException.

Can someone please share the best practice in this situation? Especially, should each thread call flushCommit() or only the main thread call it?



Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 3 | next ›
Discussion Overview
groupuser @
categorieshbase, hadoop
postedJun 1, '11 at 1:35p
activeJun 1, '11 at 6:28p

2 users in discussion

Xu, Richard: 2 posts Jean-Daniel Cryans: 1 post



site design / logo © 2022 Grokbase