FAQ
Hi Otis, Emmanuel


I have written a program to simulate the multi-thread behavior and check the performance. I have attached the program and results.


The JVM parameters when running the program: -Xms128m -Xmx1028m on a P4 2.66 Ghz, 1.99GB RAM machine.

Here are the observations:

1. The response time to fire a query and return back increases as the number of threads increases: from 10-100 (I haven’t used any threadpool. JVM default behavior is used)

2. After optimizing the index, the response time improved significantly. Like in case of 100 threads, the average response time, using optimized index, is between 1-1.3 seconds as compared to 4-5.5 seconds using the non-optimized index

Is there anything else that I should be looking into to further improve the performance?


-- Regards,

Rakesh S

Date: Fri, 23 May 2008 20:24:03 -0700
From: otis_gospodnetic@yahoo.com
Subject: Re: Improving search performance
To: java-user@lucene.apache.org

Hi Emmanuel,

Because there are some synchronized methods, like the one that checks whether a doc is deleted, that get called during search. If you have a pile of threads (op. p. mentioned 100 threads) there could be contention around those methods.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
From: Emmanuel Bernard
To: java-user@lucene.apache.org
Sent: Friday, May 23, 2008 6:41:36 PM
Subject: Re: Improving search performance

Hi
Hibernate Search does not pool the Searcher but pools the underlying
IndexReader(s). From what i've seen, a Searcher is stateless and all
the state is kept in the Readers. so this essentially is equivalent to
reusing the searcher.

Out of curiosity why is a pool of Searcher more efficient?

Emmanuel
On May 22, 2008, at 13:22, Otis Gospodnetic wrote:

Some quick feedback. Those are all very expensive queries
(wildcards and ranges). The first thing I'd do is try without
Hibernate Search (to make sure HS is not the bottleneck). 100
threads is a lot, I'm guessing you are reusing your searcher, which
is good, but you will actually improve performance a bit if you work
with a small pool of searchers instead of a single searcher.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
From: Rakesh Shete
To: java-user-info@lucene.apache.org; java-user@lucene.apache.org
Sent: Thursday, May 22, 2008 1:16:13 PM
Subject: Improving search performance


Hi all,

I have index of size 85MB. My query looks as follows:

+(t:boss* d:boss* dd:boss* tg:boss*) +st:act +ntid:0 +cid:1 +dr:
[20080410 TO
20081010] +rT:[002 TO 005]

All the fields used in the query are stored in the indexes (Indexed
& Stored)

The query response time for me is around 30 seconds when running
mutliple
simultanoeous threads (~100). The no. of matches is ~30k but I
retrieve only the
top 100 results. I am using Hibernate Search which is a wrapper
around Lucene. I
retrieve the "id" filed from the index which is also indexex and
stored.

What is the approach that I should take for improving the
performance?

Will just indexing the values without storing them work (Index &
UnStored)?

My machine configuration is:
P4 2.66GHz 1.99 GB RAM

The code for searching runs in JBoss application server which has a
maximum heap
size of 1024MB. When these 100 threads are running in the
application server the
CPU utilization is 100% and JBoss consumes all of the heap size.

Any pointers on index optimization would be really appreciated.

--Regards,
Rakesh Shete

_________________________________________________________________
No Harvard, No Oxford. We are here. Find out !!
http://ss1.richmedia.in/recurl.asp?pid=500

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
_________________________________________________________________
Catch the latest fashion shows, get beauty tips and learn more on fashion and lifestyle.
http://video.msn.com/?mkt=en-in

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 11 of 16 | next ›
Discussion Overview
groupjava-user @
categorieslucene
postedMay 22, '08 at 5:16p
activeMay 30, '08 at 9:19p
posts16
users7
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase