FAQ
Hi,

I'm running load tests with Lucene 2.0, SUN's JDK 6 on Windows XP2, dual core
CPU. I have 8 worker threads adding a few hundred K documents, split between
two Lucene indexes, I've started getting

java.io.IOException: The handle is invalid in places like

java.io.RandomAccessFile.writeBytes(Native Method)
java.io.RandomAccessFile.write(RandomAccessFile.java:466)
org.apache.lucene.store.FSIndexOutput.flushBuffer(FSDirectory.java:503)
org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:84)
org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:98)
org.apache.lucene.store.FSIndexOutput.close(FSDirectory.java:506)
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:415)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:90)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)

java.io.RandomAccessFile.close0(Native Method)
java.io.RandomAccessFile.close(RandomAccessFile.java:543)
org.apache.lucene.store.FSIndexInput.close(FSDirectory.java:465)
org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:231)
org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:168)
org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:145)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:735)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)

I saw Mike McCandless JIRA issue

http://issues.apache.org/jira/browse/LUCENE-669

Is the patch referenced there useful for a 2.0 system. I would like to use the
lockless commit stuff, but am waiting until I get the core system working well.

I am also getting IOException in some of my classes, but from the JIRA comments,
it seems that Lucene may be the culprit.

No urgency, I won't get a chance to test it until the New Year, but it's easy to
reproduce, although intermittent.

Antony


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Michael McCandless at Dec 22, 2006 at 5:34 pm

    Antony Bowesman wrote:
    Hi,

    I'm running load tests with Lucene 2.0, SUN's JDK 6 on Windows XP2, dual
    core CPU. I have 8 worker threads adding a few hundred K documents,
    split between two Lucene indexes, I've started getting

    java.io.IOException: The handle is invalid in places like

    java.io.RandomAccessFile.writeBytes(Native Method)
    java.io.RandomAccessFile.write(RandomAccessFile.java:466)
    org.apache.lucene.store.FSIndexOutput.flushBuffer(FSDirectory.java:503)
    org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:84)

    org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:98)

    org.apache.lucene.store.FSIndexOutput.close(FSDirectory.java:506)
    org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:415)
    org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:90)
    org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
    org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
    org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)

    org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
    com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)


    java.io.RandomAccessFile.close0(Native Method)
    java.io.RandomAccessFile.close(RandomAccessFile.java:543)
    org.apache.lucene.store.FSIndexInput.close(FSDirectory.java:465)
    org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:231)

    org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:168)

    org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:145)

    org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:735)
    org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
    org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)

    org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
    com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)


    I saw Mike McCandless JIRA issue

    http://issues.apache.org/jira/browse/LUCENE-669

    Is the patch referenced there useful for a 2.0 system. I would like to
    use the lockless commit stuff, but am waiting until I get the core
    system working well.

    I am also getting IOException in some of my classes, but from the JIRA
    comments, it seems that Lucene may be the culprit.
    This does sound very much like LUCENE-669 (and that bug is indeed
    present in Lucene 2.0). That patch was fairly simple; it may apply
    cleanly or require only small fixes. I would recommend try it &
    seeing if it resolves your IOExceptions?

    If you still have more IOExceptions even after that, maybe test the
    trunk.

    Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Antony Bowesman at Jan 2, 2007 at 9:30 am
    Hi Mike,
    I saw Mike McCandless JIRA issue

    http://issues.apache.org/jira/browse/LUCENE-669

    Is the patch referenced there useful for a 2.0 system. I would like
    to use the lockless commit stuff, but am waiting until I get the core
    system working well.

    I am also getting IOException in some of my classes, but from the JIRA
    comments, it seems that Lucene may be the culprit.
    This does sound very much like LUCENE-669 (and that bug is indeed
    present in Lucene 2.0). That patch was fairly simple; it may apply
    cleanly or require only small fixes. I would recommend try it &
    seeing if it resolves your IOExceptions?
    I made the patch, re-run the tests and the IOExceptions have gone away, so this
    fix looks good, though with Christmas and New Year between the previous test and
    these ones, I'm not 100% sure I had the same setup...

    I'm glad you fixed this one already ;) !

    Thanks
    Antony



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jan 2, 2007 at 5:55 pm

    Antony Bowesman wrote:
    Hi Mike,
    I saw Mike McCandless JIRA issue

    http://issues.apache.org/jira/browse/LUCENE-669

    Is the patch referenced there useful for a 2.0 system. I would like
    to use the lockless commit stuff, but am waiting until I get the core
    system working well.

    I am also getting IOException in some of my classes, but from the
    JIRA comments, it seems that Lucene may be the culprit.
    This does sound very much like LUCENE-669 (and that bug is indeed
    present in Lucene 2.0). That patch was fairly simple; it may apply
    cleanly or require only small fixes. I would recommend try it &
    seeing if it resolves your IOExceptions?
    I made the patch, re-run the tests and the IOExceptions have gone away,
    so this fix looks good, though with Christmas and New Year between the
    previous test and these ones, I'm not 100% sure I had the same setup...

    I'm glad you fixed this one already ;) !
    Awesome, I'm glad to hear that! Let's tentatively hope that indeed
    you were hitting that bug and now it's resolved :)

    Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedDec 22, '06 at 2:58a
activeJan 2, '07 at 5:55p
posts4
users2
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase