FAQ
The setup: Using Lucene.NET in a web environment on Win2k3 servers. One
process runs every 5 minutes, grabbing new rows from the database, and
adding them to a Lucene index. Only additions are made to the index, no
deletions. The mergeFactor is set to 2 to minimize the number of segments.

This index is not searched on, but it is copied to a series of webservers,
where searches are performed. I'm caching the IndexReader used by the
IndexSearcher; this IndexReader is recreated whenever the index files
change.

My questions: What's the proper way to deploy this main index to all the
webservers? Is it safe to just xcopy the index to each individual
webserver? If so, is there a good way to delete the old index files?

Now this is just the setup I've dreamed up; if someone can suggest a better
method, I'm all ears. I'd like to keep the indexing process separated from
the index deployment process. And I don't want to interrupt searches on the
web frontend.

Thanks!



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Otis Gospodnetic at Mar 24, 2005 at 2:48 am
    Hello Monsur,

    Since the index is not being searched while it is being built, I'd use
    a higher mergeFactor.

    As for deployment, I am not sure about the Windows environment, but you
    could try imitating this:

    http://mail-archives.eu.apache.org/mod_mbox/lucene-java-user/200503.mbox/%3C42254963.6000901@apache.org%3E

    Otis


    --- Monsur Hossain wrote:
    The setup: Using Lucene.NET in a web environment on Win2k3 servers.
    One
    process runs every 5 minutes, grabbing new rows from the database,
    and
    adding them to a Lucene index. Only additions are made to the index,
    no
    deletions. The mergeFactor is set to 2 to minimize the number of
    segments.

    This index is not searched on, but it is copied to a series of
    webservers,
    where searches are performed. I'm caching the IndexReader used by
    the
    IndexSearcher; this IndexReader is recreated whenever the index files
    change.

    My questions: What's the proper way to deploy this main index to all
    the
    webservers? Is it safe to just xcopy the index to each individual
    webserver? If so, is there a good way to delete the old index files?

    Now this is just the setup I've dreamed up; if someone can suggest a
    better
    method, I'm all ears. I'd like to keep the indexing process
    separated from
    the index deployment process. And I don't want to interrupt searches
    on the
    web frontend.

    Thanks!



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • George Aroush at Mar 24, 2005 at 2:39 pm
    Hi Monsur,

    On Windows, using Lucene.Net/DotLucene I have done a similar trick to what
    Otis pointed out in his link. However, in my case, I renamed the index
    folder (much faster then copying the index around.) That is, I have two
    folders, lets call one "LiveIndex" the other "LiveSearch". Based on a
    configuration file, when the LiveIndex hits a certain limit, I set a flag (a
    "lock" file if you will) and my search application monitors this flag. When
    it's set, it renames those folders, et. al.

    Regards,

    -- George Aroush

    -----Original Message-----
    From: Otis Gospodnetic
    Sent: Wednesday, March 23, 2005 9:48 PM
    To: java-user@lucene.apache.org
    Subject: Re: Deploying index to multiple webservers

    Hello Monsur,

    Since the index is not being searched while it is being built, I'd use a
    higher mergeFactor.

    As for deployment, I am not sure about the Windows environment, but you
    could try imitating this:

    http://mail-archives.eu.apache.org/mod_mbox/lucene-java-user/200503.mbox/%3C
    42254963.6000901@apache.org%3E

    Otis


    --- Monsur Hossain wrote:
    The setup: Using Lucene.NET in a web environment on Win2k3 servers.
    One
    process runs every 5 minutes, grabbing new rows from the database, and
    adding them to a Lucene index. Only additions are made to the index,
    no deletions. The mergeFactor is set to 2 to minimize the number of
    segments.

    This index is not searched on, but it is copied to a series of
    webservers, where searches are performed. I'm caching the IndexReader
    used by the IndexSearcher; this IndexReader is recreated whenever the
    index files change.

    My questions: What's the proper way to deploy this main index to all
    the webservers? Is it safe to just xcopy the index to each individual
    webserver? If so, is there a good way to delete the old index files?

    Now this is just the setup I've dreamed up; if someone can suggest a
    better method, I'm all ears. I'd like to keep the indexing process
    separated from the index deployment process. And I don't want to
    interrupt searches on the web frontend.

    Thanks!



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedMar 24, '05 at 12:42a
activeMar 24, '05 at 2:39p
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase