FAQ
Hi,

I try to add and update document in index

At the start its take only 1 to 2 second but after 50 to 60 document add
and update

Its take 40 to 50 second or some time its take more then 1 min



Is there any way to improve performance ?



Please help me



Please check my code



if(documents!=null && documents.size()>0){

IndexWriter writer = null;

boolean createNewIndexDirectory =
!IndexReader.indexExists(FSDirectory.open(indexDir));

int createNewIndexReTries = 0;

Directory directory = FSDirectory.open(indexDir, null);

while(writer==null) {

try{

directory.setLockFactory(new
NativeFSLockFactory(indexDir));



if(analyzer == null){


analyzer = new
StandardAnalyzer(IAsiteIndexConstants.LUCENE_VERSION,AsiteIndexUtil.getS
topWordSet());

}

writer = new IndexWriter(directory, analyzer,
createNewIndexDirectory,IndexWriter.MaxFieldLength.UNLIMITED);

for(Document document : documents){


if(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM_ID)!=null){

formId =
Integer.parseInt(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM
_ID));

}

String strValue =
NumericUtils.longToPrefixCoded(Long.parseLong(document.get(ICustomFormIn
dexConstants.INDEX_FIELD_MSG_ID)));

term = new
Term(ICustomFormIndexConstants.INDEX_FIELD_MSG_ID,strValue);

writer.updateDocument(term, document);

}

}catch (LockObtainFailedException ex){

CheckIndex.Status status = new
CheckIndex(directory).checkIndex();

if(status!=null && status.numBadSegments>0){

log.error("Index found corrupted.
status.numBadSegments ::"+status.numBadSegments);

ex.printStackTrace();

throw ex;

}

}finally{

try{

if(writer!=null){

writer.close();

}



}catch(IOException ex){

}

}



}




Thanks & Regards

Yogesh

Search Discussions

  • Erick Erickson at Jun 13, 2011 at 12:24 pm
    My first question is "what are you trying to do at a higher level"?
    Because asking people to check your code without telling us
    what you're trying to accomplish makes it difficult to know what
    to look at. You might review:

    http://wiki.apache.org/solr/UsingMailingLists

    That said, at a guess, your update statement will first delete
    all the items in the index where the term matches. Is it possible
    that you're deleting/adding the same documents over and
    over and over again?

    Best
    Erick
    On Mon, Jun 13, 2011 at 7:14 AM, Yogesh Dabhi wrote:


    Hi,

    I try to add and update  document in index

    At the start its take only 1 to 2 second but after 50 to 60 document add
    and update

    Its take 40 to 50 second or some time its take more then 1 min



    Is there any  way to improve performance ?



    Please help me



    Please check my code



    if(documents!=null && documents.size()>0){

    IndexWriter writer = null;

    boolean createNewIndexDirectory =
    !IndexReader.indexExists(FSDirectory.open(indexDir));

    int createNewIndexReTries = 0;

    Directory directory = FSDirectory.open(indexDir, null);

    while(writer==null) {

    try{

    directory.setLockFactory(new
    NativeFSLockFactory(indexDir));



    if(analyzer == null){


    analyzer = new
    StandardAnalyzer(IAsiteIndexConstants.LUCENE_VERSION,AsiteIndexUtil.getS
    topWordSet());

    }

    writer = new IndexWriter(directory, analyzer,
    createNewIndexDirectory,IndexWriter.MaxFieldLength.UNLIMITED);

    for(Document document : documents){


    if(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM_ID)!=null){

    formId =
    Integer.parseInt(document.get(ICustomFormIndexConstants.INDEX_FIELD_FORM
    _ID));

    }

    String strValue =
    NumericUtils.longToPrefixCoded(Long.parseLong(document.get(ICustomFormIn
    dexConstants.INDEX_FIELD_MSG_ID)));

    term = new
    Term(ICustomFormIndexConstants.INDEX_FIELD_MSG_ID,strValue);

    writer.updateDocument(term, document);

    }

    }catch (LockObtainFailedException ex){

    CheckIndex.Status status = new
    CheckIndex(directory).checkIndex();

    if(status!=null && status.numBadSegments>0){

    log.error("Index found corrupted.
    status.numBadSegments ::"+status.numBadSegments);

    ex.printStackTrace();

    throw ex;

    }

    }finally{

    try{

    if(writer!=null){

    writer.close();

    }



    }catch(IOException ex){

    }

    }



    }




    Thanks & Regards

    Yogesh
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJun 13, '11 at 11:15a
activeJun 13, '11 at 12:24p
posts2
users2
websitelucene.apache.org

2 users in discussion

Erick Erickson: 1 post Yogesh Dabhi: 1 post

People

Translate

site design / logo © 2022 Grokbase