FAQ
I am a developer on the JIRA Issue tracker, and we are considering
upgrading our Lucene version from v2.2.0 to v2.3.2.
I have been charged with doing the risk analysis, and project work.

I have read the change lists, and the bugs reported on the Lucene Issue
Tracker (JIRA of course ;), and some of the archived messages on the
Lucene lists.
From this I would conclude that Lucene v2.3.2 has many performance
improvements as well as some other compelling enhancements and bug fixes.
However there also appear to be a number of issues introduced in v2.3.x,
not all of which are fixed in v2.3.2
These include:
* LUCENE-1335 Correctly handle concurrent calls to addIndexes,
optimize, commit
I assume that if the application code doesn't make concurrent calls to
Lucene objects, or synchronises correctly then this will be OK.
* LUCENE-1175 occasional MergeException while indexing
(can occur under load)
This bug seems to be RAMDirectory-specific, and it is not clear whether
it was released or not.
Can anyone confirm this?
* There seem to have been some issues with using autoCommit(false).
Are these all fixed in v2.3.2?
Would it be recommended to use autoCommit(true) to be extra safe?

In addition is the following trap for young players:
* LUCENE-1282 Sun hotspot compiler bug in 1.6.0_04/05 affects Lucene
This is a bug in Sun JDK 6. Fixed in JDK 6 Update 10 (b28).
The application developer needs to take steps to ensure that a broken
Sun JDK 6 is not used.

I would really appreciate any comments on the overall stability of
v2.3.2 compared to v2.2.0, as well as any pointers about any pitfalls to
look out for on upgrade.
For instance, although the message signatures of the API may be
backwardly compatible, it could be that some behaviour has changed in a
significant way.

Thanks for your time,
Mark Lassau.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Tom at Aug 26, 2008 at 7:19 am
    AUTOMATIC REPLY

    Tom Roberts is out of the office till 2nd September 2008.

    LUX reopens on 1st September 2008



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Aug 26, 2008 at 9:13 am

    Mark Lassau wrote:

    I am a developer on the JIRA Issue tracker, and we are considering
    upgrading our Lucene version from v2.2.0 to v2.3.2.
    I have been charged with doing the risk analysis, and project work.

    I have read the change lists, and the bugs reported on the Lucene
    Issue Tracker (JIRA of course ;), and some of the archived messages
    on the Lucene lists.
    We love Jira -- thanks!
    From this I would conclude that Lucene v2.3.2 has many performance
    improvements as well as some other compelling enhancements and bug
    fixes.
    However there also appear to be a number of issues introduced in
    v2.3.x, not all of which are fixed in v2.3.2
    These include:
    * LUCENE-1335 Correctly handle concurrent calls to addIndexes,
    optimize, commit
    I assume that if the application code doesn't make concurrent calls
    to Lucene objects, or synchronises correctly then this will be OK.
    That's the right workaround. Also, commit() isn't available in 2.3,
    but the issue still applies to close(), rollback(), addIndexes() and
    optimize() in 2.3.
    * LUCENE-1175 occasional MergeException while indexing
    (can occur under load)
    This bug seems to be RAMDirectory-specific, and it is not clear
    whether it was released or not.
    Wasn't released, and wasn't RAMDirectory specific.
    * There seem to have been some issues with using autoCommit(false).
    Are these all fixed in v2.3.2?
    To my knowledge, yes.
    Would it be recommended to use autoCommit(true) to be extra safe?
    Well... autoCommit=true is going away in 3.0. You should also see
    better performance with autoCommit=false. And autoCommit=false also
    has nice transactional behavior (you have full control over when the
    changes are committed to the index).
    In addition is the following trap for young players:
    * LUCENE-1282 Sun hotspot compiler bug in 1.6.0_04/05 affects Lucene
    This is a bug in Sun JDK 6. Fixed in JDK 6 Update 10 (b28).
    The application developer needs to take steps to ensure that a
    broken Sun JDK 6 is not used.
    We also committed a workaround to Lucene for this JRE bug, which
    should allow you to use the buggy JRE releases, *but* it's not certain
    the workaround will alway sork. I just experimented empirically until
    I found code that seemed to avoid the error. It's by far safest to
    use a JRE that actually fixes the bug.
    I would really appreciate any comments on the overall stability of
    v2.3.2 compared to v2.2.0, as well as any pointers about any
    pitfalls to look out for on upgrade.
    For instance, although the message signatures of the API may be
    backwardly compatible, it could be that some behaviour has changed
    in a significant way.
    I would think LUCENE-1044 (not corrupting search index if OS or
    machine crashes) is important for Atlassian.

    The CHANGES.txt should be a reliable list of any non-backwards-
    compatible changes and how to revert to the old behavior (and if it
    isn't we'd love to hear about that!). EG merging is now done
    concurrently by default, flushing is by RAM usage not doc count, merge
    selection is by size-in-bytes of each segment (not size-in-doc-count).

    Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Mark Lassau at Aug 27, 2008 at 1:27 am
    Mike,

    Thanks for the prompt response.

    Michael McCandless wrote:
    Mark Lassau wrote:
    I am a developer on the JIRA Issue tracker, and we are considering
    upgrading our Lucene version from v2.2.0 to v2.3.2.
    I have been charged with doing the risk analysis, and project work.

    I have read the change lists, and the bugs reported on the Lucene
    Issue Tracker (JIRA of course ;), and some of the archived messages
    on the Lucene lists.
    We love Jira -- thanks!
    ... and we love Lucene :)

    <snip>
    I would think LUCENE-1044 (not corrupting search index if OS or
    machine crashes) is important for Atlassian.
    LUCENE-1044 is only marked as fixed in v2.4
    I assumed that the possible corruption on hard shutdown existed in
    v2.2.0 (and earlier), so we are no worse off by upgrading to v2.3.2.
    Is that the case?
    The CHANGES.txt should be a reliable list of any
    non-backwards-compatible changes and how to revert to the old behavior
    (and if it isn't we'd love to hear about that!). EG merging is now
    done concurrently by default, flushing is by RAM usage not doc count,
    merge selection is by size-in-bytes of each segment (not
    size-in-doc-count).
    Thanks, I read those. Good job on providing a very useful change summary.
    However, I guess I was also interested in any reports "from the field".
    Sometimes there can be unexpected consequences.
    eg, I read somewhere of one user having multi-threaded issues because of
    a change from Vector to ArrayList in some implementation.

    Cheers,
    Mark.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Aug 27, 2008 at 9:50 am

    Mark Lassau wrote:

    Mike,

    Thanks for the prompt response.

    Michael McCandless wrote:
    Mark Lassau wrote:
    I am a developer on the JIRA Issue tracker, and we are considering
    upgrading our Lucene version from v2.2.0 to v2.3.2.
    I have been charged with doing the risk analysis, and project work.

    I have read the change lists, and the bugs reported on the Lucene
    Issue Tracker (JIRA of course ;), and some of the archived
    messages on the Lucene lists.
    We love Jira -- thanks!
    ... and we love Lucene :)
    Glad to hear it!
    <snip>
    I would think LUCENE-1044 (not corrupting search index if OS or
    machine crashes) is important for Atlassian.
    LUCENE-1044 is only marked as fixed in v2.4
    I assumed that the possible corruption on hard shutdown existed in
    v2.2.0 (and earlier), so we are no worse off by upgrading to v2.3.2.
    Is that the case?
    That's right.
    The CHANGES.txt should be a reliable list of any non-backwards-
    compatible changes and how to revert to the old behavior (and if it
    isn't we'd love to hear about that!). EG merging is now done
    concurrently by default, flushing is by RAM usage not doc count,
    merge selection is by size-in-bytes of each segment (not size-in-
    doc-count).
    Thanks, I read those. Good job on providing a very useful change
    summary.
    However, I guess I was also interested in any reports "from the
    field". Sometimes there can be unexpected consequences.
    eg, I read somewhere of one user having multi-threaded issues
    because of a change from Vector to ArrayList in some implementation.
    OK gotchya.

    I think that particular change was to Document (in how it stores
    fields internally); there was a case where someone was actually adding
    fields to a document with multiple threads and the switch to ArrayList
    broke that code. It's tricky because Document never declared that its
    APIs were thread safe.

    Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedAug 26, '08 at 7:18a
activeAug 27, '08 at 9:50a
posts5
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase