FAQ
I have an index that's approximately 875MB. I'm using JBoss Application
Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
128MB/512MB. On initial startup, everything works fine. I'm able to
search (although it takes a while doing the first search because it's
loading the index into memory - because I'm sorting on a field)...
everything's fine and dandy. But if I stop the application and
redeploy, I get an OutOfMemoryError when I search (when the index gets
loaded into memory again). I've tried this on both Java 5 and Java 6.



When I stop the application, I made sure (or at least to the best of my
knowledge) that I close all searcher/reader/writer. I'm pretty sure
that there are no locks on the index. I've tried increasing the maximum
JVM size to 1GB. Now I can redeploy the application twice before it
gives me the OutOfMemoryError. It seems like the JVM isn't garbage
collecting the index.



So is this a JVM issue where the index is still in memory and it isn't
being garbage collected? Or am I not closing my searcher/reader/writer
correctly? Has anyone else experienced this?



Thanks,



Van

Search Discussions

  • Otis Gospodnetic at Dec 20, 2006 at 11:45 pm
    Are you using 2.1-dev version of Lucene? Try the latest nightly build, it as a fix for a certain OOM bug (see LUCENE-754).

    Otis

    ----- Original Message ----
    From: Van Nguyen <vnguyen@ur.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, December 20, 2006 6:39:58 PM
    Subject: JAVA JVM Question

    I have an index that's approximately 875MB. I'm using JBoss Application
    Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
    128MB/512MB. On initial startup, everything works fine. I'm able to
    search (although it takes a while doing the first search because it's
    loading the index into memory - because I'm sorting on a field)...
    everything's fine and dandy. But if I stop the application and
    redeploy, I get an OutOfMemoryError when I search (when the index gets
    loaded into memory again). I've tried this on both Java 5 and Java 6.



    When I stop the application, I made sure (or at least to the best of my
    knowledge) that I close all searcher/reader/writer. I'm pretty sure
    that there are no locks on the index. I've tried increasing the maximum
    JVM size to 1GB. Now I can redeploy the application twice before it
    gives me the OutOfMemoryError. It seems like the JVM isn't garbage
    collecting the index.



    So is this a JVM issue where the index is still in memory and it isn't
    being garbage collected? Or am I not closing my searcher/reader/writer
    correctly? Has anyone else experienced this?



    Thanks,



    Van





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Simon Willnauer at Dec 21, 2006 at 3:11 am
    OOM Errors are not uncommon during redeployment on application server
    e.g. servlet container. Redeploy on Tomcat servers very often cause
    OOM due to the perm gen space which get not GCed(that should go away
    with 5.5). The JBoss can usually deal with these issue but just in
    case you could check your PerGenSpace via JMX or some profiler to see
    what happens with your loaded classes. If you redeploy in live /
    production env. make sure you restart your app. server.
    Do you use some common classes which are provided via parent
    classloader of your application? This could also cause such problems.
    If this is a PermGenSpace problem you might fix it by replacing one of
    the components in your software stack. This issues has been widely
    discussed and the finger of blame has been pointed to tomcat, cglib or
    even your JVM impl.

    best regards simon
    On 12/20/06, Otis Gospodnetic wrote:
    Are you using 2.1-dev version of Lucene? Try the latest nightly build, it as a fix for a certain OOM bug (see LUCENE-754).

    Otis

    ----- Original Message ----
    From: Van Nguyen <vnguyen@ur.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, December 20, 2006 6:39:58 PM
    Subject: JAVA JVM Question

    I have an index that's approximately 875MB. I'm using JBoss Application
    Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
    128MB/512MB. On initial startup, everything works fine. I'm able to
    search (although it takes a while doing the first search because it's
    loading the index into memory - because I'm sorting on a field)...
    everything's fine and dandy. But if I stop the application and
    redeploy, I get an OutOfMemoryError when I search (when the index gets
    loaded into memory again). I've tried this on both Java 5 and Java 6.



    When I stop the application, I made sure (or at least to the best of my
    knowledge) that I close all searcher/reader/writer. I'm pretty sure
    that there are no locks on the index. I've tried increasing the maximum
    JVM size to 1GB. Now I can redeploy the application twice before it
    gives me the OutOfMemoryError. It seems like the JVM isn't garbage
    collecting the index.



    So is this a JVM issue where the index is still in memory and it isn't
    being garbage collected? Or am I not closing my searcher/reader/writer
    correctly? Has anyone else experienced this?



    Thanks,



    Van





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Van Nguyen at Dec 21, 2006 at 6:14 pm
    This is a different OOM error. This one is due to Java heap space.
    I've tried using Otis' suggestion and use the latest nightly build (I've
    actually tried using 12/19/2006 and 12/20/2006)... but I am still
    getting this OOS: Java heap space error. I will try to profile this app
    to see if I can get more info.

    I've read about the PermGenSpace errors and I do get that from time to
    time on redeploys and heard that Java 6 was suppose to handle garbage
    collection better. The only common classes are the ones provided by
    JBoss. We have taken to restarting the app server every time we deploy
    an application... but we hope that that's not a permanent solution
    because we do not want to take down all of our applications because we
    need to redeploy one app.


    Van

    -----Original Message-----
    From: Simon Willnauer
    Sent: Wednesday, December 20, 2006 7:11 PM
    To: java-user@lucene.apache.org
    Subject: Re: JAVA JVM Question

    OOM Errors are not uncommon during redeployment on application server
    e.g. servlet container. Redeploy on Tomcat servers very often cause
    OOM due to the perm gen space which get not GCed(that should go away
    with 5.5). The JBoss can usually deal with these issue but just in
    case you could check your PerGenSpace via JMX or some profiler to see
    what happens with your loaded classes. If you redeploy in live /
    production env. make sure you restart your app. server.
    Do you use some common classes which are provided via parent
    classloader of your application? This could also cause such problems.
    If this is a PermGenSpace problem you might fix it by replacing one of
    the components in your software stack. This issues has been widely
    discussed and the finger of blame has been pointed to tomcat, cglib or
    even your JVM impl.

    best regards simon
    On 12/20/06, Otis Gospodnetic wrote:
    Are you using 2.1-dev version of Lucene? Try the latest nightly build,
    it as a fix for a certain OOM bug (see LUCENE-754).
    Otis

    ----- Original Message ----
    From: Van Nguyen <vnguyen@ur.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, December 20, 2006 6:39:58 PM
    Subject: JAVA JVM Question

    I have an index that's approximately 875MB. I'm using JBoss
    Application
    Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
    128MB/512MB. On initial startup, everything works fine. I'm able to
    search (although it takes a while doing the first search because it's
    loading the index into memory - because I'm sorting on a field)...
    everything's fine and dandy. But if I stop the application and
    redeploy, I get an OutOfMemoryError when I search (when the index gets
    loaded into memory again). I've tried this on both Java 5 and Java 6.



    When I stop the application, I made sure (or at least to the best of my
    knowledge) that I close all searcher/reader/writer. I'm pretty sure
    that there are no locks on the index. I've tried increasing the maximum
    JVM size to 1GB. Now I can redeploy the application twice before it
    gives me the OutOfMemoryError. It seems like the JVM isn't garbage
    collecting the index.



    So is this a JVM issue where the index is still in memory and it isn't
    being garbage collected? Or am I not closing my
    searcher/reader/writer
    correctly? Has anyone else experienced this?



    Thanks,



    Van





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Simon Willnauer at Dec 23, 2006 at 5:03 am
    Do you have a stack trace by any chance, well OOM do not create very
    meanful stacktraces but it could help to figure out which error
    occures. PermGenSpace belogs to the java heap btw.
    The JBoss should actually be able to deal with your redeployment but
    in live/production a restart is always requiered.

    best regards simon
    On 12/21/06, Van Nguyen wrote:
    This is a different OOM error. This one is due to Java heap space.
    I've tried using Otis' suggestion and use the latest nightly build (I've
    actually tried using 12/19/2006 and 12/20/2006)... but I am still
    getting this OOS: Java heap space error. I will try to profile this app
    to see if I can get more info.

    I've read about the PermGenSpace errors and I do get that from time to
    time on redeploys and heard that Java 6 was suppose to handle garbage
    collection better. The only common classes are the ones provided by
    JBoss. We have taken to restarting the app server every time we deploy
    an application... but we hope that that's not a permanent solution
    because we do not want to take down all of our applications because we
    need to redeploy one app.


    Van

    -----Original Message-----
    From: Simon Willnauer
    Sent: Wednesday, December 20, 2006 7:11 PM
    To: java-user@lucene.apache.org
    Subject: Re: JAVA JVM Question

    OOM Errors are not uncommon during redeployment on application server
    e.g. servlet container. Redeploy on Tomcat servers very often cause
    OOM due to the perm gen space which get not GCed(that should go away
    with 5.5). The JBoss can usually deal with these issue but just in
    case you could check your PerGenSpace via JMX or some profiler to see
    what happens with your loaded classes. If you redeploy in live /
    production env. make sure you restart your app. server.
    Do you use some common classes which are provided via parent
    classloader of your application? This could also cause such problems.
    If this is a PermGenSpace problem you might fix it by replacing one of
    the components in your software stack. This issues has been widely
    discussed and the finger of blame has been pointed to tomcat, cglib or
    even your JVM impl.

    best regards simon
    On 12/20/06, Otis Gospodnetic wrote:
    Are you using 2.1-dev version of Lucene? Try the latest nightly build,
    it as a fix for a certain OOM bug (see LUCENE-754).
    Otis

    ----- Original Message ----
    From: Van Nguyen <vnguyen@ur.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, December 20, 2006 6:39:58 PM
    Subject: JAVA JVM Question

    I have an index that's approximately 875MB. I'm using JBoss
    Application
    Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
    128MB/512MB. On initial startup, everything works fine. I'm able to
    search (although it takes a while doing the first search because it's
    loading the index into memory - because I'm sorting on a field)...
    everything's fine and dandy. But if I stop the application and
    redeploy, I get an OutOfMemoryError when I search (when the index gets
    loaded into memory again). I've tried this on both Java 5 and Java 6.



    When I stop the application, I made sure (or at least to the best of my
    knowledge) that I close all searcher/reader/writer. I'm pretty sure
    that there are no locks on the index. I've tried increasing the maximum
    JVM size to 1GB. Now I can redeploy the application twice before it
    gives me the OutOfMemoryError. It seems like the JVM isn't garbage
    collecting the index.



    So is this a JVM issue where the index is still in memory and it isn't
    being garbage collected? Or am I not closing my
    searcher/reader/writer
    correctly? Has anyone else experienced this?



    Thanks,



    Van





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Otis Gospodnetic at Dec 23, 2006 at 5:37 am
    The latest Java (6) does produce a stack trace when am OOM error happens. It doesn't necessarily point to the source of the leak, though, just at the unfortunate thing that needed the non-existent memory to do its work.

    Otis

    ----- Original Message ----
    From: Simon Willnauer <simon.willnauer@googlemail.com>
    To: Van Nguyen <vnguyen@ur.com>
    Cc: java-user@lucene.apache.org
    Sent: Saturday, December 23, 2006 12:01:57 AM
    Subject: Re: JAVA JVM Question

    Do you have a stack trace by any chance, well OOM do not create very
    meanful stacktraces but it could help to figure out which error
    occures. PermGenSpace belogs to the java heap btw.
    The JBoss should actually be able to deal with your redeployment but
    in live/production a restart is always requiered.

    best regards simon
    On 12/21/06, Van Nguyen wrote:
    This is a different OOM error. This one is due to Java heap space.
    I've tried using Otis' suggestion and use the latest nightly build (I've
    actually tried using 12/19/2006 and 12/20/2006)... but I am still
    getting this OOS: Java heap space error. I will try to profile this app
    to see if I can get more info.

    I've read about the PermGenSpace errors and I do get that from time to
    time on redeploys and heard that Java 6 was suppose to handle garbage
    collection better. The only common classes are the ones provided by
    JBoss. We have taken to restarting the app server every time we deploy
    an application... but we hope that that's not a permanent solution
    because we do not want to take down all of our applications because we
    need to redeploy one app.


    Van

    -----Original Message-----
    From: Simon Willnauer
    Sent: Wednesday, December 20, 2006 7:11 PM
    To: java-user@lucene.apache.org
    Subject: Re: JAVA JVM Question

    OOM Errors are not uncommon during redeployment on application server
    e.g. servlet container. Redeploy on Tomcat servers very often cause
    OOM due to the perm gen space which get not GCed(that should go away
    with 5.5). The JBoss can usually deal with these issue but just in
    case you could check your PerGenSpace via JMX or some profiler to see
    what happens with your loaded classes. If you redeploy in live /
    production env. make sure you restart your app. server.
    Do you use some common classes which are provided via parent
    classloader of your application? This could also cause such problems.
    If this is a PermGenSpace problem you might fix it by replacing one of
    the components in your software stack. This issues has been widely
    discussed and the finger of blame has been pointed to tomcat, cglib or
    even your JVM impl.

    best regards simon
    On 12/20/06, Otis Gospodnetic wrote:
    Are you using 2.1-dev version of Lucene? Try the latest nightly build,
    it as a fix for a certain OOM bug (see LUCENE-754).
    Otis

    ----- Original Message ----
    From: Van Nguyen <vnguyen@ur.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, December 20, 2006 6:39:58 PM
    Subject: JAVA JVM Question

    I have an index that's approximately 875MB. I'm using JBoss
    Application
    Server 4.04 w/ Apache HTTP Server 2.2. My min/max JVM size is:
    128MB/512MB. On initial startup, everything works fine. I'm able to
    search (although it takes a while doing the first search because it's
    loading the index into memory - because I'm sorting on a field)...
    everything's fine and dandy. But if I stop the application and
    redeploy, I get an OutOfMemoryError when I search (when the index gets
    loaded into memory again). I've tried this on both Java 5 and Java 6.



    When I stop the application, I made sure (or at least to the best of my
    knowledge) that I close all searcher/reader/writer. I'm pretty sure
    that there are no locks on the index. I've tried increasing the maximum
    JVM size to 1GB. Now I can redeploy the application twice before it
    gives me the OutOfMemoryError. It seems like the JVM isn't garbage
    collecting the index.



    So is this a JVM issue where the index is still in memory and it isn't
    being garbage collected? Or am I not closing my
    searcher/reader/writer
    correctly? Has anyone else experienced this?



    Thanks,



    Van





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedDec 20, '06 at 11:40p
activeDec 23, '06 at 5:37a
posts6
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase