search performace

I've been effectively off-line for a few days, so I'm not sure if
anyone has replied on this thread yet.

Using boosts will definitely use less resources than sorting. If you
do use sorting for dates, be sure you're doing it numerically rather
than lexicographically.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

I am sorting against an epoch time stored in my index. By using:

contactDocument.add( Field.Keyword( "epoch_time", epoch );

Then I sort by this field. My search time is in the order of 3sec on an
index of about 6G using simple searches against a text field. By using
boosts I was hoping to increase performance. Do you think this will make a
big difference?

Michael

-----Original Message-----
From: Erik Hatcher
Sent: Tuesday, March 15, 2005 8:43 AM
To: java-user@lucene.apache.org
Subject: Re: search performace

I've been effectively off-line for a few days, so I'm not sure if
anyone has replied on this thread yet.

Using boosts will definitely use less resources than sorting. If you
do use sorting for dates, be sure you're doing it numerically rather
than lexicographically.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Is epoch a Date? or a String? If a String, what format is it?

Sorting by a Date keyword field will be sorting as a String value,
which is a fair bit more resource intensive than if it was numeric.

Try using a purely numeric field (though as a String) that can be
represented as an int be sure to specify the sort type as an int and
see if that improves performance. I'm pretty certain you'd still get
better performance by using a boost than a sort though.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Epoch is in seconds... I am also forced to used a date filter on most of
searches... how bad is the performance hit of that.

-----Original Message-----
From: Erik Hatcher
Sent: Thursday, March 17, 2005 9:54 AM
To: java-user@lucene.apache.org
Subject: Re: search performace

Is epoch a Date? or a String? If a String, what format is it?

Sorting by a Date keyword field will be sorting as a String value,
which is a fair bit more resource intensive than if it was numeric.

Try using a purely numeric field (though as a String) that can be
represented as an int be sure to specify the sort type as an int and
see if that improves performance. I'm pretty certain you'd still get
better performance by using a boost than a sort though.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

But you still haven't provided the *type* of epoch. It's a Date? a
String? What do the string values look like?
Only testing will tell. The hit of a filter comes the first time (as
long as you cache and use the same IndexReader), so its not likely to
be a factor over many queries.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

My epoch looks like 1110816121 but is represented by a string.

-----Original Message-----
From: Erik Hatcher
Sent: Thursday, March 17, 2005 11:41 AM
To: java-user@lucene.apache.org
Subject: Re: search performace

But you still haven't provided the *type* of epoch. It's a Date? a
String? What do the string values look like?
Only testing will tell. The hit of a filter comes the first time (as
long as you cache and use the same IndexReader), so its not likely to
be a factor over many queries.

Erik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org