Grokbase › Groups › Lucene › java-user › June 2008
FAQ
I have extended my evaluation (previous evaluation:
http://zzzoot.blogspot.com/2008/06/simultaneous-threaded-query-lucene.html)
to include as well as an increasing # of threads performing concurrent
queries, 1,2,4 and 8 IndexReaders.

The results can be found here:
http://zzzoot.blogspot.com/2008/06/lucene-concurrent-search-performance.html

thanks,

Glen

--

-

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Otis Gospodnetic at Jun 11, 2008 at 7:02 pm ⇧
    Hi Glen,

    Aha, good to see the benefit of multiple IndexReaders/Searchers so clearly. Makes me think we'll want to add a config setting for this in Solr... :)

    As for why 4 is the best choice, I think it's because of those 4 cores that you've got. My guess is that you'll see slightly better performance with 5 threads and then the performance will slowly deteriorate with more readers/searchers.... let's see it!


    Otis
    --
    Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

    ----- Original Message ----
    From: Glen Newton <glen.newton@gmail.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, June 11, 2008 2:07:45 PM
    Subject: Concurrent query benchmarks, with 1,2,4,8 readers

    I have extended my evaluation (previous evaluation:
    http://zzzoot.blogspot.com/2008/06/simultaneous-threaded-query-lucene.html)
    to include as well as an increasing # of threads performing concurrent
    queries, 1,2,4 and 8 IndexReaders.

    The results can be found here:
    http://zzzoot.blogspot.com/2008/06/lucene-concurrent-search-performance.html

    thanks,

    Glen

    --

    -

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Glen Newton at Jun 11, 2008 at 7:12 pm ⇧
    Hi Otis,

    Thanks for the feedback.

    2008/6/11 Otis Gospodnetic <otis_gospodnetic@yahoo.com>:
    Hi Glen,

    Aha, good to see the benefit of multiple IndexReaders/Searchers so clearly. Makes me think we'll want to add a config setting for this in Solr... :)
    Until then, you might want to use: Runtime.availableProcessors()
    as the default #.
    Oh no, that won't work: it gives me 8 (the number of hyperthread
    processors) versus 4 (# of real cores. Hmm, I consider not being able
    to find the number of physical cores as being, well, a bug (I guess
    you could turn-off hyperthreading). Anyone know if there is a JSR
    looking for perhaps: Runtime.availableRealProcessors() ??
    As for why 4 is the best choice, I think it's because of those 4 cores that you've got. My guess is that you'll see slightly better performance with 5 threads and then the performance will slowly deteriorate with more readers/searchers.... let's see it!
    I'm running it & will post when it is done.

    thanks,

    Glen :-)
    Otis
    --
    Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

    ----- Original Message ----
    From: Glen Newton <glen.newton@gmail.com>
    To: java-user@lucene.apache.org
    Sent: Wednesday, June 11, 2008 2:07:45 PM
    Subject: Concurrent query benchmarks, with 1,2,4,8 readers

    I have extended my evaluation (previous evaluation:
    http://zzzoot.blogspot.com/2008/06/simultaneous-threaded-query-lucene.html)
    to include as well as an increasing # of threads performing concurrent
    queries, 1,2,4 and 8 IndexReaders.

    The results can be found here:
    http://zzzoot.blogspot.com/2008/06/lucene-concurrent-search-performance.html

    thanks,

    Glen

    --

    -

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    --

    -

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Lutan at Jun 13, 2008 at 11:04 am ⇧
    TO: Glen Newton Could I get your test code or code architecture for study.
    I have try to using java.util.concurrent package(
    like ArrayBlockingQueue ThreadPoolExecutor;)
    with lucene,but it is no successful.I don't
    know how to design.


    Thanks ! my emial: wsnet@live.cn
    _________________________________________________________________
    多个邮箱同步管理，live mail客户端万人抢用中
    http://get.live.cn/product/mail.html
  • Glen Newton at Jun 13, 2008 at 4:52 pm ⇧
    Lutan,

    Yes, no problem. I am away at a conference next week but plan to
    release the code the following week. Is this OK for you?

    thanks,

    Glen

    2008/6/13 lutan <wsnet@live.cn>:
    TO: Glen Newton Could I get your test code or code architecture for study.
    I have try to using java.util.concurrent package(
    like ArrayBlockingQueue ThreadPoolExecutor;)
    with lucene,but it is no successful.I don't
    know how to design.


    Thanks ! my emial: wsnet@live.cn
    _________________________________________________________________
    多个邮箱同步管理，live mail客户端万人抢用中
    http://get.live.cn/product/mail.html


    --

    -
  • Lutan at Jun 13, 2008 at 5:15 pm ⇧
    TO : Glen Newton
    Thank you very much!
    I am so glad to learn it.
    Date: Fri, 13 Jun 2008 12:51:51 -0400> From: glen.newton@gmail.com> To: java-user@lucene.apache.org> Subject: Re: Concurrent query benchmarks, with 1,2,4,8 readers> > Lutan,> > Yes, no problem. I am away at a conference next week but plan to> release the code the following week. Is this OK for you?> > thanks,> > Glen> > 2008/6/13 lutan <wsnet@live.cn>:> >> > TO: Glen Newton Could I get your test code or code architecture for study.> > I have try to using java.util.concurrent package(> > like ArrayBlockingQueue ThreadPoolExecutor;)> > with lucene,but it is no successful.I don't> > know how to design.> >> >> > Thanks ! my emial: wsnet@live.cn> > _________________________________________________________________> > > > -- > > -
    _________________________________________________________________
    用手机MSN聊天写邮件看空间，无限沟通，分享精彩！
    http://mobile.msn.com.cn/
  • Rakesh Shete at Jun 25, 2008 at 3:10 am ⇧
    Hi Glen,

    Is your source code available? I would like to have a look at it and check if whatever I have tried makes sense.

    --Regards,
    Rakesh S


    Date: Fri, 13 Jun 2008 12:51:51 -0400
    From: glen.newton@gmail.com
    To: java-user@lucene.apache.org
    Subject: Re: Concurrent query benchmarks, with 1,2,4,8 readers

    Lutan,

    Yes, no problem. I am away at a conference next week but plan to
    release the code the following week. Is this OK for you?

    thanks,

    Glen

    2008/6/13 lutan <wsnet@live.cn>:
    TO: Glen Newton Could I get your test code or code architecture for study.
    I have try to using java.util.concurrent package(
    like ArrayBlockingQueue ThreadPoolExecutor;)
    with lucene,but it is no successful.I don't
    know how to design.


    Thanks ! my emial: wsnet@live.cn
    _________________________________________________________________
    多个邮箱同步管理，live mail客户端万人抢用中
    http://get.live.cn/product/mail.html


    --

    -
    _________________________________________________________________
    Timely update on all current affairs, sports, events and all thats in News here on MSN videos.
    http://video.msn.com/?mkt=en-in
  • Grant Ingersoll at Jun 25, 2008 at 10:56 am ⇧
    Note, you can do all kinds of tests like this and others with the
    contrib/benchmark code built right into Lucene.

    -Grant
    On Jun 24, 2008, at 11:09 PM, Rakesh Shete wrote:

    Hi Glen,

    Is your source code available? I would like to have a look at it and
    check if whatever I have tried makes sense.

    --Regards,
    Rakesh S


    Date: Fri, 13 Jun 2008 12:51:51 -0400
    From: glen.newton@gmail.com
    To: java-user@lucene.apache.org
    Subject: Re: Concurrent query benchmarks, with 1,2,4,8 readers

    Lutan,

    Yes, no problem. I am away at a conference next week but plan to
    release the code the following week. Is this OK for you?

    thanks,

    Glen

    2008/6/13 lutan <wsnet@live.cn>:
    TO: Glen Newton Could I get your test code or code architecture
    for study.
    I have try to using java.util.concurrent package(
    like ArrayBlockingQueue ThreadPoolExecutor;)
    with lucene,but it is no successful.I don't
    know how to design.


    Thanks ! my emial: wsnet@live.cn
    _________________________________________________________________
    多个邮箱同步管理，live mail客户端万人抢用中
    http://get.live.cn/product/mail.html


    --

    -
    _________________________________________________________________
    Timely update on all current affairs, sports, events and all thats
    in News here on MSN videos.
    http://video.msn.com/?mkt=en-in
    --------------------------
    Grant Ingersoll
    http://www.lucidimagination.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ








    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Grant Ingersoll at Jun 16, 2008 at 12:13 pm ⇧
    Also note, there is the contrib/benchmark source in Lucene that allows
    one to do many of these tests in a "standard" way, such that the
    results are easily reproducible.

    Do you actually have a situation where you would ever see 8K threads?
    Or even 4k? Or was this just for the purposes of the exercise? I
    would be interested in hearing more about your use case, if you can
    share it.
    On Jun 11, 2008, at 2:07 PM, Glen Newton wrote:

    I have extended my evaluation (previous evaluation:
    http://zzzoot.blogspot.com/2008/06/simultaneous-threaded-query-lucene.html)
    to include as well as an increasing # of threads performing concurrent
    queries, 1,2,4 and 8 IndexReaders.

    The results can be found here:
    http://zzzoot.blogspot.com/2008/06/lucene-concurrent-search-performance.html

    thanks,

    Glen

    --

    -

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    --------------------------
    Grant Ingersoll
    http://www.lucidimagination.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ








    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJun 11, '08 at 6:08p
activeJun 25, '08 at 10:56a
posts9
users5
websitelucene.apache.org

5 users in discussion

Glen Newton: 3 posts Lutan: 2 posts Grant Ingersoll: 2 posts Rakesh Shete: 1 post Otis Gospodnetic: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase