FAQ
Hi everyone,

There has been a lot of discussion regarding Lucene+NFS pitfalls. I'm
not sure how to proceed with a more distributed operation.

I'm trying to take the indexing load off of our search server. I can do
this either by building a new server which hosts the Indexer and the
Index, or a server which hosts the Searcher and the Index. The two boxes
will be connected via NFS.

In our system there are no updates or deletes. Just lots of "add" and
"optimize" calls.

What is the best way to handle this sort of situation? My inclination is
build a new Search Server (with fast HDDs and lots of Memory for tomcat)
and leave the indexer on the old server connected via NFS.

Thanks!

-Dave


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Search Discussions

  • Michael McCandless at Apr 27, 2009 at 8:13 pm
    "In theory" either solution will work, from Lucene's standpoint. But
    this is not well-explored territory: you are a pioneer! Please report
    back on your results :)

    Usually (and this may be different in your app), search performance
    trumps indexing performance, so my guess is you'd want the index
    hosted on the same machine doing searching. If you do it this way,
    it's possible you won't need a custom IndexDeletionPolicy, because the
    searchers will hold open files via the local filesystem which should
    properly protect them when an NFS client (the indexer machine) tries
    to delete them.

    Mike
    On Mon, Apr 27, 2009 at 3:42 PM, David Seltzer wrote:
    Hi everyone,

    There has been a lot of discussion regarding Lucene+NFS pitfalls. I'm
    not sure how to proceed with a more distributed operation.

    I'm trying to take the indexing load off of our search server. I can do
    this either by building a new server which hosts the Indexer and the
    Index, or a server which hosts the Searcher and the Index. The two boxes
    will be connected via NFS.

    In our system there are no updates or deletes. Just lots of "add" and
    "optimize" calls.

    What is the best way to handle this sort of situation? My inclination is
    build a new Search Server (with fast HDDs and lots of Memory for tomcat)
    and leave the indexer on the old server connected via NFS.

    Thanks!

    -Dave


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]
  • Sudarsan, Sithu D. at Apr 27, 2009 at 8:19 pm

    What is the best way to handle this sort of situation? My inclination is
    build a new Search Server (with fast HDDs and lots of Memory for tomcat)
    and leave the indexer on the old server connected via NFS.
    - Our current development is on similar lines. Almost no deletes, but
    only lots of ADDDDD...

    - Our approach is similar to yours. Without multi-level searches (which
    we are currently trying to implement), I'm not able to comment on
    performance at this point.
    Sincerely,
    Sithu D Sudarsan
    -----Original Message-----
    From: Michael McCandless
    Sent: Monday, April 27, 2009 4:13 PM
    To: [email protected]
    Subject: Re: Yet another NFS Question...

    "In theory" either solution will work, from Lucene's standpoint. But
    this is not well-explored territory: you are a pioneer! Please report
    back on your results :)

    Usually (and this may be different in your app), search performance
    trumps indexing performance, so my guess is you'd want the index
    hosted on the same machine doing searching. If you do it this way,
    it's possible you won't need a custom IndexDeletionPolicy, because the
    searchers will hold open files via the local filesystem which should
    properly protect them when an NFS client (the indexer machine) tries
    to delete them.

    Mike
    On Mon, Apr 27, 2009 at 3:42 PM, David Seltzer wrote:
    Hi everyone,

    There has been a lot of discussion regarding Lucene+NFS pitfalls. I'm
    not sure how to proceed with a more distributed operation.

    I'm trying to take the indexing load off of our search server. I can do
    this either by building a new server which hosts the Indexer and the
    Index, or a server which hosts the Searcher and the Index. The two boxes
    will be connected via NFS.

    In our system there are no updates or deletes. Just lots of "add" and
    "optimize" calls.

    What is the best way to handle this sort of situation? My inclination is
    build a new Search Server (with fast HDDs and lots of Memory for tomcat)
    and leave the indexer on the old server connected via NFS.

    Thanks!

    -Dave


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedApr 27, '09 at 7:42p
activeApr 27, '09 at 8:19p
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2023 Grokbase