FAQ
Hello all,

We have been using Lucene 3.1 version with Tomcat 4.0 and jdk1.4.
It seems that sometimes we see a "Too many files open" exception which
completely garbles the whole index and whole search functionality
crashes on the web site. It has also been known to crash the complete
JSP container of tomcat.

After looking at the bug list, we found out that it has been reported as
a bug in the Lucene bug list as Bug#29774, #30049, #30452 which claims
to have been resolved with the new version of Lucene.

We have tried everything to reproduce the problem ourselves to figure
out the exact circumstances under which it occurs but with out any luck.
We are absolutely sure that we close all the files, index readers,
writers and searchers.

We would be installing the new version of Lucene but we need to be able
to reproduce the problem consistently to test it.

We would really appreciate it if someone could point us to the root
cause behind this so we can devise a solution around that.

Thanks and regards
Neelam Bhatnagar

Technology| Sapient
Presidency Building
Mehrauli-Gurgaon Road
Sector-14, Gurgaon-122001
Haryana, India

Tel: 91.124.2826299
Cell: 91.9899591054
Email: nbhatnagar@sapient.com

Search Discussions

  • Luke Francl at Nov 18, 2004 at 3:48 pm

    On Thu, 2004-11-18 at 07:09, Neelam Bhatnagar wrote:
    Hello all,

    We have been using Lucene 3.1 version with Tomcat 4.0 and jdk1.4.
    It seems that sometimes we see a "Too many files open" exception which
    completely garbles the whole index and whole search functionality
    crashes on the web site. It has also been known to crash the complete
    JSP container of tomcat.
    (I'm assuming you meant Lucene 1.3)

    This exception happens when your process has too many file handles open.
    Values differ by operating system.

    With Lucene, this is caused by having a number of IndexReaders open.
    Each IndexReader will open each file in your index. If you do not close
    your IndexReaders, this exception can happen, especially if you have a
    lot of heap and the IndexReaders are not getting garbage collected.

    My guess is that you are creating a new IndexSearcher for each search
    request and then not closing it after the search is complete.

    Lucene 1.3 added a feature called compound index files that much
    alleviates this problem by greatly reducing the number of files required
    in an index. You can use it by turning on IndexWriter.useCompoundFile(
    true ):

    http://jakarta.apache.org/lucene/docs/api/org/apache/lucene/index/IndexWriter.html#getUseCompoundFile()

    Combined with closing your IndexReaders, this should fix the problem.

    Regards,
    Luke Francl


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedNov 18, '04 at 1:09p
activeNov 18, '04 at 3:48p
posts2
users2
websitelucene.apache.org

2 users in discussion

Neelam Bhatnagar: 1 post Luke Francl: 1 post

People

Translate

site design / logo © 2022 Grokbase