Hey Mark,

yes. I'm running the app on unix. You see the difference between 2.9 and 2.4 here:


2.4 responds much quicker thus increasing throughput severly. I'm having a
single segment only:

-rw-r--r-- 1 asuser asgroup 20 Sep 9 16:40 segments.gen
-rw-r--r-- 1 asuser asgroup 294 Sep 9 16:44 segments_1o
-rw-r--r-- 1 asuser asgroup 2810603184 Sep 9 16:44 _7c.cfs

The FileChannel.read hotspot is indeed strange. Especially if taking into
account that the index is lying on a tmpfs (in memory). And 2.4 should be using
NIOFSDirectory as well?! Will check that.

Thanks a lot for your support!


Mark Miller wrote:
A few quick notes -

Lucene 2.9 old api doesn't appear much worse than Lucene 2.4?

You save a lot with the new Intern impl, because thats not a hotspot
anymore. But then,
RandomAccessFile seeks end up being a lot more of the pie. They look
fairly similar in speed overall?

It looks like the major bottleneck with 2.9 new api is that its using
NIOFSDirectory (your on unix I guess, and it now
defaults to that on non Windows os's), and that appears to be a real
killer for you. Its taking half the time for its
reads. ???

No conclusions yet, but I'm looking it over. Some other guys will come
in with some ideas as well.

Do confirm that those profiling results are on a single segment though.

- Mark

Mark Miller wrote:
Thomas Becker wrote:
Here's the results of profiling 10 different search requests:


But you already gave me a good hint. The index being used is an old one build
with lucene 2.4. I will now try a freshly build 2.9 index and see if performance
improves. Maybe that already solves the issue...stupid me...
That shouldn't be an issue unless there is some odd bug.

We're updating the index every 30 min. at the moment and it gets optimized after
each update.
So this profiling is on an optimized index (eg a single segment) ?
That would be odd indeed, and possibly point to some of the scoring changes.

Mark Miller wrote:

Thomas Becker wrote:

Hey Mark,

thanks for your reply. Will do. Results will follow in a couple of minutes.

Thanks, awesome.

Also, how many segments (approx) are in your index? If there are a lot,
have you/can you try the same tests on an optimized index? Don't want to
get ahead of the profiling results, but just to continue the info loop.

Thomas Becker
Senior JEE Developer

net mobile AG
Zollhof 17
40221 Düsseldorf

Phone: +49 211 97020-195
Fax: +49 211 97020-949
Mobile: +49 173 5146567 (private)
E-Mail: mailto:thomas.becker@net-m.de
Internet: http://www.net-m.de

Registergericht: Amtsgericht Düsseldorf, HRB 48022
Vorstand: Theodor Niehues (Vorsitzender), Frank Hartmann,
Kai Markus Kulas, Dieter Plassmann
Vorsitzender des
Aufsichtsrates: Dr. Michael Briem

To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 57 | next ›
Discussion Overview
groupjava-user @
postedSep 15, '09 at 1:30p
activeSep 16, '09 at 6:09p



site design / logo © 2021 Grokbase