Guys,
I am doing a multi-threaded loading of a lucene index over WCF (so clients call a lucene WCF service to do their adds and queries).. Anways. I have my loader set to a 2 hour timeout, but that still doesn't seem long enough to avoid merges from occasionally killing my connections.. what ways are there to decrease or remove the cost of merging from big loads (in our case about 180 million documents).. We wouldn't mind doing an optimize at the end to keep things rolling..
One of the thoughts I am considering is just setting the max merge file size to about 500 megs, as it seems to time out when merging up to the 1 gig files (probably a cascade merge from small to medium to large that's causing it).
Also on a side note, can you search (read) an index while it is being optimized?
Thanks in advance.
Josh Handel
Senior Lead Consultant
512.328.8181 | Main
512.328.0584 | Fax
512.577-6568 | Cell
www.catapultsystems.com<blocked::blocked::http://www.catapultsystems.com/>
CATAPULT SYSTEMS INC.
ENABLING BUSINESS THROUGH TECHNOLOGY
