FAQ
On 2.9. NIOFS is only used, if you use FSDirectory.open() instead of
FSDirectory.getDirectory (Deprecated). Can you compare when you use instead
of FSDirectory.open() the direct ctor of SimpleFSDir vs. NIOFSDir vs.
MMapDir and compare?

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
-----Original Message-----
From: Mark Miller
Sent: Tuesday, September 15, 2009 5:30 PM
To: java-user@lucene.apache.org
Subject: Re: lucene 2.9.0RC4 slower than 2.4.1?

Thomas Becker wrote:
Hey Mark,

yes. I'm running the app on unix. You see the difference between 2.9 and 2.4 here:
http://ankeschwarzer.de/tmp/graph.jpg
Right - I know your measurements showed a difference (and will keep that
in mind) - but the profiling results then seem
oddly similar.
2.4 responds much quicker thus increasing throughput severly. I'm having a
single segment only:

-rw-r--r-- 1 asuser asgroup 20 Sep 9 16:40 segments.gen
-rw-r--r-- 1 asuser asgroup 294 Sep 9 16:44 segments_1o
-rw-r--r-- 1 asuser asgroup 2810603184 Sep 9 16:44 _7c.cfs

The FileChannel.read hotspot is indeed strange. Especially if taking into
account that the index is lying on a tmpfs (in memory). And 2.4 should be using
NIOFSDirectory as well?! Will check that.
2.4 did not use NIOFSDirectory by default - you would have had to
manually specified it. Now its used by default if its detects a non
Windows OS.

Any particular reason your profiling output does not show invocations?
Its not essential most likely, but I've found it to be helpful in
comparisons.

We are about to release 2.9, and its been such a long haul, I'd hate to
see a release with an unknown performance trap.

--
- Mark

http://www.lucidimagination.com

Thanks a lot for your support!

Cheers,
Thomas

Mark Miller wrote:
A few quick notes -

Lucene 2.9 old api doesn't appear much worse than Lucene 2.4?

You save a lot with the new Intern impl, because thats not a hotspot
anymore. But then,
RandomAccessFile seeks end up being a lot more of the pie. They look
fairly similar in speed overall?

It looks like the major bottleneck with 2.9 new api is that its using
NIOFSDirectory (your on unix I guess, and it now
defaults to that on non Windows os's), and that appears to be a real
killer for you. Its taking half the time for its
reads. ???

No conclusions yet, but I'm looking it over. Some other guys will come
in with some ideas as well.

Do confirm that those profiling results are on a single segment though.

- Mark


Mark Miller wrote:
Thomas Becker wrote:

Here's the results of profiling 10 different search requests:

http://ankeschwarzer.de/tmp/lucene_24_oldapi.png
http://ankeschwarzer.de/tmp/lucene_29_oldapi.png
http://ankeschwarzer.de/tmp/lucene_29_newapi.png

But you already gave me a good hint. The index being used is an old
one build
with lucene 2.4. I will now try a freshly build 2.9 index and see if
performance
improves. Maybe that already solves the issue...stupid me...

That shouldn't be an issue unless there is some odd bug.


We're updating the index every 30 min. at the moment and it gets
optimized after
each update.

So this profiling is on an optimized index (eg a single segment) ?
That would be odd indeed, and possibly point to some of the scoring
changes.

Mark Miller wrote:


Thomas Becker wrote:


Hey Mark,

thanks for your reply. Will do. Results will follow in a couple of
minutes.



Thanks, awesome.

Also, how many segments (approx) are in your index? If there are a
lot,
have you/can you try the same tests on an optimized index? Don't
want to
get ahead of the profiling results, but just to continue the info
loop.



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 13 of 57 | next ›
Discussion Overview
groupjava-user @
categorieslucene
postedSep 15, '09 at 1:30p
activeSep 16, '09 at 6:09p
posts57
users6
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase