FAQ
I want to cache full text into memory to improve performance.
Full text is only used to highlight in my application(But it's very
time consuming, My avg query time is about 250ms, I guess it will cost
about 50ms if I just get top 10 full text. Things get worse when get
more full text because in disk, it scatters erverywhere for a query.).
My full text per machine is about 200GB. The memory available for
store full text is about 10GB. So I want to compress it in memory.
Suppose compression ratio is 1:5, then I can load 1/4 full text in
memory. I need a Cache component for it. Has anyone faced the problem
before? I need some advice. Is it possbile using external tools such
as MemCached? Thank you.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Findbestopensource at Jul 14, 2010 at 7:03 am
    You have two options
    1. Store the compressed text as part of stored field in Solr.
    2. Using external caching.
    http://www.findbestopensource.com/tagged/distributed-caching
    You could use ehcache / Memcache / Membase.

    The problem with external caching is you need to synchronize the deletions
    and modification. Fetching the stored field from Solr is also faster.

    Regards
    Aditya
    www.findbestopensource.com

    On Wed, Jul 14, 2010 at 12:08 PM, Li Li wrote:

    I want to cache full text into memory to improve performance.
    Full text is only used to highlight in my application(But it's very
    time consuming, My avg query time is about 250ms, I guess it will cost
    about 50ms if I just get top 10 full text. Things get worse when get
    more full text because in disk, it scatters erverywhere for a query.).
    My full text per machine is about 200GB. The memory available for
    store full text is about 10GB. So I want to compress it in memory.
    Suppose compression ratio is 1:5, then I can load 1/4 full text in
    memory. I need a Cache component for it. Has anyone faced the problem
    before? I need some advice. Is it possbile using external tools such
    as MemCached? Thank you.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJul 14, '10 at 6:39a
activeJul 14, '10 at 7:03a
posts2
users2
websitelucene.apache.org

2 users in discussion

Findbestopensource: 1 post Li Li: 1 post

People

Translate

site design / logo © 2022 Grokbase