Is it possible to have two separate process, one performing searches, and
the other performing updates on the same index? I have a system in
production that uses this design and occasionally the search program grinds
to a halt. I first suspected that this was just a load issue, but there
isn't that much load (peak times average 2-3 requests per second, with
occasional bursts of 10-20 requests) and I can't replicate the problem.
The logs show that when the slowdown occurs we are usually answering
requests to search at first, but ongoing searches have stopped finishing
(somewhere inside IndexSearcher.search()). There doesn't seem to be a
single expensive query that might be bringing us to our kness either. So,
I was wondering if maybe it is possible that this is a race condition
caused by our update program, which is a separate program that updates the
index while it is being searched.

Some basic info:

The search program uses a single IndexSearcher to perform all searches.
Results are collected with a HitCollector which uses the same IndexSearcher
to extract each document - there is a requirement that the documents be
returned in a specific order, so we have an external structure to determine
the order, once the ID (not the internal ID) has been extracted.
A separate HitCollector is used for each search.
This IndexSearcher in the search program is swapped for a new one when the
update program has finished an update cycle and notifies the search
The index is about 90k documents, average query returns less than 100 hits.

Thanks for any information, or just for your opinion.


To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
postedMar 23, '04 at 6:40p
activeMar 23, '04 at 6:40p

1 user in discussion

Brad Hendricks: 1 post



site design / logo © 2023 Grokbase