Grokbase Groups Camel users May 2009

I have run into a problem that I will *really* appreciate some suggestions

I have a single message that arrives in a ActiveMQ queue. The message
contains an ID that is used by a bean to perform a database lookup. The
lookup can contain anything up to around one million records. Each of these
records need to be added to another queue for further asynchronous
processing. Once all of the records have been added to the processing queue,
I need to create a new message to perform a status update on the original
(single) request. Each of the 'spawned' records is mission critical and
cannot be lost/repeated.

The main problem is figuring out a way to create a million records as part
of a single XA transaction so that any problems can force a rollback. The
messages aren't very big (+-300bytes) each, but a million of them add up :)
Ideally, I would like to find a way to break up the transactions into
batches of 100 messages. Then if I have an outage on message #899000, it
will resume from where it left off once everything is back up & running.

If anyone has some ideas on the best way to implement this sort of
functionality, I will be very grateful.
View this message in context:
Sent from the Camel - Users mailing list archive at

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
postedMay 5, '09 at 6:34p
activeMay 5, '09 at 6:34p

1 user in discussion

Triswork: 1 post



site design / logo © 2022 Grokbase