FAQ
When the server we're developing comes up, its Lucene indexes are sometimes
locked, especially during development, when it crashes fairly frequently. I
assume that it is possible to corrupt an index by crashing at just the right
time.

I notice that there's a method IndexReader.unlock(). Does this method
ensure that the index has not been corrupted? If not, is there a way of
asking Lucene either to fix up an index or tell you that the index should be
rebuilt?

Thanks for any help.

Michael Weir · Transform Research Inc. · 613.238.1363 x.114


Cognos Forum, June 27-30, 2004: Power Your Performance. Join us in
Orlando for our global customer conference. Engage with your peers. Hear
from top analysts. Learn from the Cognos Business Intelligence and
Enterprise Planning gurus. And turn your potential into higher performance.
Register today at http://www.cognos.com/cognosforum

This message may contain privileged and/or confidential information.
If you have received this e-mail in error or are not the intended recipient,
you may not use, copy, disseminate or distribute it; do not open any
attachments, delete it immediately from your system and notify the sender
promptly by e-mail that you have done so. Thank you.

Search Discussions

  • Doug Cutting at Apr 6, 2004 at 9:10 pm

    Weir, Michael wrote:
    I assume that it is possible to corrupt an index by crashing at just the right
    time.
    It should not be possible to corrupt an index this way.
    I notice that there's a method IndexReader.unlock(). Does this method
    ensure that the index has not been corrupted?
    If you use this method, it is possible to corrupt things. In
    particular, if you unlock an index that another process is modifying,
    then modify it, then these two processes might step on one another. So
    this method should only be called when you are certain that no one else
    is modifying the index.

    There is no index integrity checker. I suppose one could implement one
    by implementing a NullDirectory that ignores all reads and writes, then
    use IndexWriter.addIndexes() to add the index to be checked to a
    NullDirectory. That would parse all of the files in the index.

    Doug

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Kevin A. Burton at Apr 13, 2004 at 12:51 am

    Doug Cutting wrote:
    If you use this method, it is possible to corrupt things. In
    particular, if you unlock an index that another process is modifying,
    then modify it, then these two processes might step on one another.
    So this method should only be called when you are certain that no one
    else is modifying the index.
    We're handling this by using .pid files. We use a standard initializer
    and use your own lock files with process IDs. If you're on UNIX I can
    give you the source to the JNI getpid that I created. I've been meaning
    on Open Sourcing this anyway... putting it into commons probably.

    This way you can prevent multiple initialization if a java process is
    currently running that might be working with your index. Otherwise
    there's no real way to be sure the lock isn't stale (unless time is a
    factor but that slows things down)

    Kevin

    --

    Please reply using PGP.

    http://peerfear.org/pubkey.asc

    NewsMonster - http://www.newsmonster.org/

    Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
    AIM/YIM - sfburtonator, Web - http://peerfear.org/
    GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
    IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org
  • Weir, Michael at Apr 7, 2004 at 2:38 pm
    Thanks for the quick response.
    So if our server is the only process that ever opens the index, I should be
    able to run through the indexes at startup and simply unlock them?

    Michael Weir . Transform Research Inc. . 613.238.1363 x.114


    -----Original Message-----
    From: Doug Cutting
    Sent: Tuesday, April 06, 2004 5:09 PM
    To: Lucene Users List
    Subject: Re: verifying index integrity


    Weir, Michael wrote:
    I assume that it is possible to corrupt an index by crashing at just the right
    time.
    It should not be possible to corrupt an index this way.
    I notice that there's a method IndexReader.unlock(). Does this method
    ensure that the index has not been corrupted?
    If you use this method, it is possible to corrupt things. In
    particular, if you unlock an index that another process is modifying,
    then modify it, then these two processes might step on one another. So
    this method should only be called when you are certain that no one else
    is modifying the index.

    There is no index integrity checker. I suppose one could implement one
    by implementing a NullDirectory that ignores all reads and writes, then
    use IndexWriter.addIndexes() to add the index to be checked to a
    NullDirectory. That would parse all of the files in the index.

    Doug

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org


    Cognos Forum, June 27-30, 2004: Power Your Performance. Join us in
    Orlando for our global customer conference. Engage with your peers. Hear
    from top analysts. Learn from the Cognos Business Intelligence and
    Enterprise Planning gurus. And turn your potential into higher performance.
    Register today at http://www.cognos.com/cognosforum

    This message may contain privileged and/or confidential information.
    If you have received this e-mail in error or are not the intended recipient,
    you may not use, copy, disseminate or distribute it; do not open any
    attachments, delete it immediately from your system and notify the sender
    promptly by e-mail that you have done so. Thank you.
  • Doug Cutting at Apr 8, 2004 at 5:10 pm

    Weir, Michael wrote:
    So if our server is the only process that ever opens the index, I should be
    able to run through the indexes at startup and simply unlock them?
    Yes.

    Doug

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
    For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedApr 6, '04 at 8:43p
activeApr 13, '04 at 12:51a
posts5
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase