Based on our discussions, it seems best to get realtime search going in
small steps. Below are some possible steps to take.

Patch #1: Expose an IndexWriter.getReader method that returns the current
reader and shares the write lock
Patch #2: Implement a realtime ram index class
Patch #3: Implement realtime transactions in IndexWriter or in a subclass of
IndexWriter by implementing a createTransaction method that generates a
realtime Transaction object. When the transaction is flushed, the
transaction index modifications are available via the getReader method of

The remaining question is how to synchronize the flushes to disk with
IndexWriter's other index update locking mechanisms. The flushing could
simply use IW.addIndexes which has in place a locking mechanism. After
flushing to disk, queued deletes would be applied to the newly copied disk
segments. I think this entails opening the newly copied disk segments and
applying deletes that occurred to the corresponding ram segments by cloning
the new disk segments and replacing the deleteddocs bitvector then flushing
the deleteddocs to disk. This system would allow us to avoid using UID in

The API needs to clearly separate realtime transactions vs. the existing
index update method such as addDocument, deleteDocuments, and
updateDocument. I don't think it's possible to transparently implement both
because the underlying implementations behave differently. It is expected
that multiple transaction may be created at once however the
Transaction.flush method would block.

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 22 | next ›
Discussion Overview
groupjava-dev @
postedJan 5, '09 at 7:13p
activeJan 31, '09 at 12:24a



site design / logo © 2021 Grokbase