Grokbase › Groups › Lucene › java-user › April 2003
FAQ
This is an exception that we've got:

03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
/usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
files) java.io.FileNotFoundException:
/usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

Is there a way to predict how many file handles Velocity would require? Is
there a setting I can use to limit this number?

Thank you!


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Search Discussions

  • Andrey Akselrod at Apr 30, 2003 at 3:12 pm ⇧
    Sorry, I meant to ask Is there a way to predict how many file handles LUCENE
    would require?
    :)

    ----- Original Message -----
    From: "Andrey Akselrod" <aakselrod@runtime.com>
    To: "Lucene Users List" <lucene-user@jakarta.apache.org>
    Sent: Wednesday, April 30, 2003 11:08 AM
    Subject: running out of file handles when reindexing

    This is an exception that we've got:

    03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files) java.io.FileNotFoundException:
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

    Is there a way to predict how many file handles Velocity would require? Is
    there a setting I can use to limit this number?

    Thank you!


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Rob Outar at Apr 30, 2003 at 3:21 pm ⇧
    I had a similar problem because I was not explicitly calling close on
    IndexSearcher when I created a new one (if the index had changed) I was
    doing Seacher = new IndexSeacher() and on Linux I was running out of file
    handles I then change it to:

    Searcher.close();
    Seacher = new IndexSeacher()

    and the problem went away.

    Hope that helps.

    Thanks,

    Rob Outar
    OneSAF A&I -- SAIC
    Software\Data Engineer
    321-235-7660
    outarr@saic.com


    -----Original Message-----
    From: Andrey Akselrod
    Sent: Wednesday, April 30, 2003 11:11 AM
    To: Lucene Users List
    Subject: Re: running out of file handles when reindexing


    Sorry, I meant to ask Is there a way to predict how many file handles LUCENE
    would require?
    :)

    ----- Original Message -----
    From: "Andrey Akselrod" <aakselrod@runtime.com>
    To: "Lucene Users List" <lucene-user@jakarta.apache.org>
    Sent: Wednesday, April 30, 2003 11:08 AM
    Subject: running out of file handles when reindexing

    This is an exception that we've got:

    03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files) java.io.FileNotFoundException:
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

    Is there a way to predict how many file handles Velocity would require? Is
    there a setting I can use to limit this number?

    Thank you!


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Rob Outar at Apr 30, 2003 at 3:30 pm ⇧
    The product we are working on allows folks to add files to the index, remove
    files from the index, add/remove fields from/to documents in the index, etc.
    so the index changes quite a bit. I was calling optimize after someone
    added a file or removed a file but performance was poor. After how many
    changes should I call optimize? I have noticed since I stopped calling
    optimize I have TONS of files in my index folder... So I need to perodically
    call optimize to cut down on the number of files... So I am curious what
    other folks are doing. So if anyone has an opinion/solution I would like to
    hear it.

    Thanks,

    Rob


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • David Medinets at Apr 30, 2003 at 3:43 pm ⇧
    Does the application have a period of least activity? Perhaps at 5:30am or
    on a weekend? If so, that might be a good time.

    ----- Original Message -----
    From: "Rob Outar" <routar@ideorlando.org>
    So if anyone has an opinion/solution I would like to
    hear it.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Leo Galambos at Apr 30, 2003 at 4:08 pm ⇧
    It is evergreen of this list, I guess :-).

    Nobody can tell you IMHO. You would execute it, when your app is in
    "idle" state and you have changed the index significantly.

    I've already implemented an algorithm that does not need any optimize(),
    because the index is optimized automatically (moreover it uses less file
    handles - it is important when your app is under DoS time to time).
    Unfortunately, this solution was implemented in another OSS JAVA search
    engine, and it does not run with Lucene architecture AFAIK. I'm sorry.

    -g-

    Rob Outar wrote:
    The product we are working on allows folks to add files to the index, remove
    files from the index, add/remove fields from/to documents in the index, etc.
    so the index changes quite a bit. I was calling optimize after someone
    added a file or removed a file but performance was poor. After how many
    changes should I call optimize? I have noticed since I stopped calling
    optimize I have TONS of files in my index folder... So I need to perodically
    call optimize to cut down on the number of files... So I am curious what
    other folks are doing. So if anyone has an opinion/solution I would like to
    hear it.

    Thanks,

    Rob


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Andrey Akselrod at Apr 30, 2003 at 4:38 pm ⇧
    No, the problem happens during reindexing in writer.addDocument(doc)
    But good catch anyway - I wasn't closing my searches.
    thanks!
    -a

    ----- Original Message -----
    From: "Rob Outar" <routar@ideorlando.org>
    To: "Lucene Users List" <lucene-user@jakarta.apache.org>
    Sent: Wednesday, April 30, 2003 11:20 AM
    Subject: RE: running out of file handles when reindexing

    I had a similar problem because I was not explicitly calling close on
    IndexSearcher when I created a new one (if the index had changed) I was
    doing Seacher = new IndexSeacher() and on Linux I was running out of file
    handles I then change it to:

    Searcher.close();
    Seacher = new IndexSeacher()

    and the problem went away.

    Hope that helps.

    Thanks,

    Rob Outar
    OneSAF A&I -- SAIC
    Software\Data Engineer
    321-235-7660
    outarr@saic.com

    -----Original Message-----
    From: Andrey Akselrod
    Sent: Wednesday, April 30, 2003 11:11 AM
    To: Lucene Users List
    Subject: Re: running out of file handles when reindexing


    Sorry, I meant to ask Is there a way to predict how many file handles LUCENE
    would require?
    :)

    ----- Original Message -----
    From: "Andrey Akselrod" <aakselrod@runtime.com>
    To: "Lucene Users List" <lucene-user@jakarta.apache.org>
    Sent: Wednesday, April 30, 2003 11:08 AM
    Subject: running out of file handles when reindexing

    This is an exception that we've got:

    03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files) java.io.FileNotFoundException:
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many open
    files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

    Is there a way to predict how many file handles Velocity would require?
    Is
    there a setting I can use to limit this number?

    Thank you!


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Otis Gospodnetic at Apr 30, 2003 at 3:14 pm ⇧
    I think that was covered in the Lucene article at http://onjava.com/
    If not, I know it has been covered on this list, so you should be able
    to find it in the archives.
    The number depends, among other things, on the number of fields used,
    whether index is optimized or not, etc.

    Otis

    --- Andrey Akselrod wrote:
    This is an exception that we've got:

    03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many
    open
    files) java.io.FileNotFoundException:
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many
    open
    files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

    Is there a way to predict how many file handles Velocity would
    require? Is
    there a setting I can use to limit this number?

    Thank you!


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    __________________________________
    Do you Yahoo!?
    The New Yahoo! Search - Faster. Easier. Bingo.
    http://search.yahoo.com

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Andrey Akselrod at Apr 30, 2003 at 4:31 pm ⇧
    Sorry, Otis, you are right. It is covered in the list and in the article. I
    searched the list prior to posting, but couldn't find anything. I just
    noticed that I accidentally searched dev list, rather then user list. Dah!
    Thanks for you reply!

    -a

    ----- Original Message -----
    From: "Otis Gospodnetic" <otis_gospodnetic@yahoo.com>
    To: "Lucene Users List" <lucene-user@jakarta.apache.org>
    Sent: Wednesday, April 30, 2003 11:14 AM
    Subject: Re: running out of file handles when reindexing

    I think that was covered in the Lucene article at http://onjava.com/
    If not, I know it has been covered on this list, so you should be able
    to find it in the archives.
    The number depends, among other things, on the number of fields used,
    whether index is optimized or not, etc.

    Otis

    --- Andrey Akselrod wrote:
    This is an exception that we've got:

    03-04-30 03:00:12,806 ERROR [ThreadPoolThread 1]
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many
    open
    files) java.io.FileNotFoundException:
    /usr/local/tomcat_4.1.24/myapps/cl/WEB-INF/search/_1t.f5 (Too many
    open
    files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)

    Is there a way to predict how many file handles Velocity would
    require? Is
    there a setting I can use to limit this number?

    Thank you!


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    __________________________________
    Do you Yahoo!?
    The New Yahoo! Search - Faster. Easier. Bingo.
    http://search.yahoo.com

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedApr 30, '03 at 3:09p
activeApr 30, '03 at 4:38p
posts9
users5
websitelucene.apache.org

5 users in discussion

Andrey Akselrod: 4 posts Rob Outar: 2 posts Leo Galambos: 1 post David Medinets: 1 post Otis Gospodnetic: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase