[ActiveMQ-users] ActiveMQ 5.10 LevelDB Issues

I have a 3 node 5.10 cluster with replicated levelDB as persistent store.


<persistenceAdapter>
          <replicatedLevelDB
           directory="activemq-data"
           replicas="3"
           bind="tcp://0.0.0.0:0"
           zkAddress="queue1:2181,queue2:2181,queue3:2181"
           zkPath="/activemq/leveldb-stores"
           hostname="queue3"
           sync="quorum_disk"
              />
         </persistenceAdapter>


I'm Stresstesting the queue with around 5000 persistent msg/s.
1 Producer
2 Consumers

I get some warning messages on the log:
On Master:
2014-12-16 16:25:28,375 | INFO | Slave has disconnected:
db6c9a23-7025-4384-b02f-dcda763113c3 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-3
2014-12-16 16:25:28,761 | WARN | Unexpected session error:
java.io.FileNotFoundException:
/queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or
directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-1
2014-12-16 16:25:28,761 | INFO | Slave has disconnected:
6a84579c-77fe-41eb-a728-1be472c12894 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-2
2014-12-16 16:25:29,603 | INFO | Slave has connected:
db6c9a23-7025-4384-b02f-dcda763113c3 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-2
2014-12-16 16:25:30,034 | INFO | Slave has connected:
6a84579c-77fe-41eb-a728-1be472c12894 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-1
2014-12-16 16:25:31,360 | INFO | Slave has now caught up:
6a84579c-77fe-41eb-a728-1be472c12894 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-4
2014-12-16 16:25:32,546 | INFO | Slave has now caught up:
db6c9a23-7025-4384-b02f-dcda763113c3 |
org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
hawtdispatch-DEFAULT-2


On the slaves

2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
2014-12-16 16:30:00,287 | WARN | No reader available for position:
27fd52c562, log_infos:
{171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}





It looks to me that from time to time the connection to the master drops.

Should I worry about it ? what No reader available means, is there data
loss risk?

It looks to me that the level db implementation is kind of fragile...

Is somebody using it in production ?

Regards


Christian