Hi - I have a query and I wonder if anyone can give me some pointers:

I am creating an index. I am adding information from rows in a database
to create a full text search.

The database will be updated every day, so I want to remove the document
relating to a specific existing row from the database and add a new
document containing the updated row - I believe that this is the way it
needs to be done.

What is the best pattern for this?

For example, can I loop through the reader doing an
IndexReader.Delete(Term) and then an IndexWriter.Add(Document) without
closing the reader and writer each time - i.e. can they both be open at
once? This would just allow me to open everything in the constructor and
close it at the end.

Or, do I need to open / close / open / close each time?

Is my approach correct in the first place?

Any assistance gratefully received!

Murad

Search Discussions

  • René de Vries at Oct 19, 2006 at 9:57 pm
    The approach we take to keep thing in sync, is that we have a trigger on the table that we're indexing. ON INSERT, we write an 'I' and a news_id pointing to the record to be updated in the syntable, so the Lucene Indexing process knows this records needs to be inserted. Same for Delete and Update. Update is implemented as Delete and Insert.

    Here's a piece of example code for the Delete:

    Private Sub deleteDocument(ByVal intNews_Id As Integer, ByVal strIndexOutputDirectory As String)
    ' Search against indexed data
    Dim objIndexSearcher As IndexSearcher = New IndexSearcher(strIndexOutputDirectory)
    Dim objQuery As Query = QueryParser.Parse(intNews_Id, "news_id", New StandardAnalyzer)
    Dim objHits As Hits = objIndexSearcher.Search(objQuery)

    ' Loop thru each retrieved search result to collect
    Dim i0 As Integer = 0
    Dim objDocument As Document = Nothing
    ' First check if the document actually exisits
    If objHits.Length > 0 Then
    objIndexSearcher.reader.Delete(New Term("news_id", intNews_Id))
    End If

    ' Clean up objects
    objIndexSearcher.Close()
    objIndexSearcher = Nothing
    objQuery = Nothing
    objHits = Nothing
    objDocument = Nothing

    End Sub

    Hope it helps,

    René

    -----Original Message-----
    From: Murad James
    Sent: donderdag 19 oktober 2006 23:06
    To: lucene-net-user@incubator.apache.org
    Subject: Can I have an IndexWriter and an IndexReader open on the same index at the same time?

    Hi - I have a query and I wonder if anyone can give me some pointers:

    I am creating an index. I am adding information from rows in a database
    to create a full text search.

    The database will be updated every day, so I want to remove the document
    relating to a specific existing row from the database and add a new
    document containing the updated row - I believe that this is the way it
    needs to be done.

    What is the best pattern for this?

    For example, can I loop through the reader doing an
    IndexReader.Delete(Term) and then an IndexWriter.Add(Document) without
    closing the reader and writer each time - i.e. can they both be open at
    once? This would just allow me to open everything in the constructor and
    close it at the end.

    Or, do I need to open / close / open / close each time?

    Is my approach correct in the first place?

    Any assistance gratefully received!

    Murad
  • Murad James at Oct 19, 2006 at 11:30 pm
    Thanks; I am doing it with a timestamp column to track the updates. I have an existing method in the data layer called GetUpdates(lastUpdated) where lastUpdated is the Sql timestamp of the last update. It gets all the records updated after lastUpdated and returns an SqlDataReader. I am going to write the timestamp as an entry into the index so I can get it back each time I go to check for updates.

    Thanks for the sample - it's similar to the sample code in the IntranetIndexer so I am clear on doing a delete. What I am not clear on is if I can delete then add again without opening and closing an IndexReader then opening and closing an IndexWriter for each update - I assume this is really inefficient although I am not sure.

    -----Original Message-----
    From: René de Vries
    Sent: 19 October 2006 22:57
    To: lucene-net-user@incubator.apache.org
    Subject: RE: Can I have an IndexWriter and an IndexReader open on the same index at the same time?

    The approach we take to keep thing in sync, is that we have a trigger on the table that we're indexing. ON INSERT, we write an 'I' and a news_id pointing to the record to be updated in the syntable, so the Lucene Indexing process knows this records needs to be inserted. Same for Delete and Update. Update is implemented as Delete and Insert.

    Here's a piece of example code for the Delete:

    Private Sub deleteDocument(ByVal intNews_Id As Integer, ByVal strIndexOutputDirectory As String)
    ' Search against indexed data
    Dim objIndexSearcher As IndexSearcher = New IndexSearcher(strIndexOutputDirectory)
    Dim objQuery As Query = QueryParser.Parse(intNews_Id, "news_id", New StandardAnalyzer)
    Dim objHits As Hits = objIndexSearcher.Search(objQuery)

    ' Loop thru each retrieved search result to collect
    Dim i0 As Integer = 0
    Dim objDocument As Document = Nothing
    ' First check if the document actually exisits
    If objHits.Length > 0 Then
    objIndexSearcher.reader.Delete(New Term("news_id", intNews_Id))
    End If

    ' Clean up objects
    objIndexSearcher.Close()
    objIndexSearcher = Nothing
    objQuery = Nothing
    objHits = Nothing
    objDocument = Nothing

    End Sub

    Hope it helps,

    René

    -----Original Message-----
    From: Murad James
    Sent: donderdag 19 oktober 2006 23:06
    To: lucene-net-user@incubator.apache.org
    Subject: Can I have an IndexWriter and an IndexReader open on the same index at the same time?

    Hi - I have a query and I wonder if anyone can give me some pointers:

    I am creating an index. I am adding information from rows in a database
    to create a full text search.

    The database will be updated every day, so I want to remove the document
    relating to a specific existing row from the database and add a new
    document containing the updated row - I believe that this is the way it
    needs to be done.

    What is the best pattern for this?

    For example, can I loop through the reader doing an
    IndexReader.Delete(Term) and then an IndexWriter.Add(Document) without
    closing the reader and writer each time - i.e. can they both be open at
    once? This would just allow me to open everything in the constructor and
    close it at the end.

    Or, do I need to open / close / open / close each time?

    Is my approach correct in the first place?

    Any assistance gratefully received!

    Murad

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouplucene-net-user @
categorieslucene
postedOct 19, '06 at 9:06p
activeOct 19, '06 at 11:30p
posts3
users2
websitelucene.apache.org

2 users in discussion

Murad James: 2 posts René de Vries: 1 post

People

Translate

site design / logo © 2022 Grokbase