We have been using Lucene 3.1 version with Tomcat 4.0 and jdk1.4.
It seems that sometimes we see a "Too many files open" exception which
completely garbles the whole index and whole search functionality
crashes on the web site. It has also been known to crash the complete
JSP container of tomcat.
After looking at the bug list, we found out that it has been reported as
a bug in the Lucene bug list as Bug#29774, #30049, #30452 which claims
to have been resolved with the new version of Lucene.
We have tried everything to reproduce the problem ourselves to figure
out the exact circumstances under which it occurs but with out any luck.
We are absolutely sure that we close all the files, index readers,
writers and searchers.
We would be installing the new version of Lucene but we need to be able
to reproduce the problem consistently to test it.
We would really appreciate it if someone could point us to the root
cause behind this so we can devise a solution around that.
Thanks and regards