FAQ
To make a search by lucene we use Web application that is working in the
servlet-container tomcat.

Periodically we update the search index by adding and removing some of
the documents. To update the index we make the following actions:

1. Open the index
2. Add/delete documents
3. In case it is needed we optimize the index
4. Close the index

After the updates of search index were done we report the Web
application that there are changes in the index. Web application does
the following:

1. Close old index search object (before closing the application waits
until all steams finish up the work with search index):
searcher.close()

2. Create new index search object - searcher:
File file = new File(pathToIndex);
IndexSearcher searcher = new IndexSearcher(NIOFSDirectory.open(file), true);

After a while, the Web application stops operating because of the error
"Too many open files". If you look at a list of files that were opened
by tomcat process (using the command "ls -la /proc/$PID/fd"), you can
see that there are a lot of deleted ".cfs" files.

There are no errors in the tomcat logs.

Can you help us to figure out why this situation is occurred and how to
solve it up?


Thanks in advance.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Ian Lea at Sep 3, 2010 at 2:23 pm
    http://wiki.apache.org/lucene-java/LuceneFAQ#Why_am_I_getting_an_IOException_that_says_.22Too_many_open_files.22.3F

    You are probably not closing all your searchers properly. What
    version of lucene are you using? I think that something changed in
    recent releases and old searchers that used to get garbage collected
    had their files automatically closed, and that no longer happens.
    Something along those lines anyway. Maybe you need to store a
    reference to the underlying Directory and call close() on that.

    I expect that if you search and dig around in the list archives you'll
    find a better explanation - or wait for someone else to explain it
    properly. But I'm pretty sure that the underlying message is correct
    - make sure you close everything related to your old searchers.

    As an aside, the problem might be less if you skipped the optimize
    step. In general you don't need to call optimize.



    --
    Ian.

    On Fri, Sep 3, 2010 at 2:47 PM, Gennadiy Kozlenko wrote:
    To make a search by lucene we use Web application that is working in the
    servlet-container tomcat.

    Periodically we update the search index by adding and removing some of the
    documents. To update the index we make the following actions:

    1. Open the index
    2. Add/delete documents
    3. In case it is needed we optimize the index
    4. Close the index

    After the updates of search index were done we report the Web application
    that there are changes in the index. Web application does the following:

    1. Close old index search object (before closing the application waits until
    all steams finish up the work with search index):
    searcher.close()

    2. Create new index search object - searcher:
    File file = new File(pathToIndex);
    IndexSearcher searcher = new IndexSearcher(NIOFSDirectory.open(file), true);

    After a while, the Web application stops operating because of the error "Too
    many open files". If you look at a list of files that were opened by tomcat
    process (using the command "ls -la /proc/$PID/fd"),  you can see that there
    are a lot of deleted ".cfs" files.

    There are no errors in the tomcat logs.

    Can you help us to figure out why this situation is occurred and how to
    solve it up?


    Thanks in advance.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Grijesh.singh at Sep 6, 2010 at 1:05 pm
    Ask your sys admin to increase the os max open file limit

    ulimit

    -----
    Grijesh
    --
    View this message in context: http://lucene.472066.n3.nabble.com/Too-many-open-files-tp1412227p1425425.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedSep 3, '10 at 2:04p
activeSep 6, '10 at 1:05p
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase