Concurrent query benchmarks

Hi Glen,

Thanks for sharing. Does your benchmarking tool build on top of contrib/benchmark? (not sure if that one lets you specify the number of concurrent threads -- if it does not, perhaps this is an opportunity to add this functionality).

I couldn't find info about the index format (compound or not) you used. It would be good to see the comparison with high number of threads for the 2 index formats. It would also be good to see the numbers when the index has no deletion and when it has some percentage of docs deleted.


Finally, if you end up extending contrib/benchmark, I think just having the ability to pump the results of that into a gnuplot script would be nice to have. I've written a standalone benchmarking tool that did pretty much what yours seems to do, but I wrote it for Technorati, so I can't release it. :(

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

contrib/benchmark's syntax for parallel-execution spawns parallel threads.
I.e. something like
[ Search ] : 20
would run 20 queries in parallel, and this
[ Search ] : 20000 : 5
would run 20,000 queries in parallel, but starting up to 5 new threads per
second.
One more example:
[
{ SearchTrav(20) > : 300
] : 500
Would spawn 500 threads each running 300 queries, where query 20 results are
traversed.
It allows various parallel settings, though not sure it covers exactly what
Glen did.


Yes this would be great!

No, it is a stand-alone program. You give it the index directory, the
default query field, the number of threads, and the filename of a file
that contains one Lucene query per line.hreads. The output is one
line: the # fo threads followed by the #queries handled per second.

I have a shell script which runs the above with increasing #s of threads.
Sorry, I didn't include it. The index in the benchmarks uses the
compound format, with 0% documents deleted.
I would be very willing to contribute what I have, with the gnuplot
scripts that I have. Let me finish off what I am doing for my work and
I will clean things up a bit, write a little documentation.

-Glen

Good work!

I would like to see how it performs with several index reader instances,
which is said to increase concurrency.

Thanks for the positive feedback. :-)

Yes, right now the benchmark only uses one IndexSearcher for all
threads, but I have completed an extension that allows you to either
1) have multiple searchers for the same index; or 2) have multiple
indexes (copies of one another) with a single searcher per copy (to
test when you have your index copies on separate disks, SANS, NAS,
etc).

I will rerun my benchmarks with increasing numbers of readers & post
the results in the next couple of days.

-glen

2008/6/10 Chris Lu <chris.lu@gmail.com>: