Grokbase Groups Lucene dev June 2011
FAQ
Ability to mlock certain fields from the terms dict
---------------------------------------------------

Key: LUCENE-3214
URL: https://issues.apache.org/jira/browse/LUCENE-3214
Project: Lucene - Java
Issue Type: Improvement
Reporter: Michael McCandless


This is a hacked up prototype! It works but I'm not sure how to get this to a committable point.

The patch invokes mlock() (tested only on Linux), locking pages from the terms dictionary file that hold terms for a specified field. You can only do this with MMapDirectory.

I used this to lock pages for the "id" field in the NRT stress test; it's an alternative to MemoryCodec. But, it requires you set up the OS to allow the app/user to lock pages in RAM.

It works very well in reducing the NRT reopen latency even when large merges are running...

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Search Discussions

  • Michael McCandless (JIRA) at Jun 18, 2011 at 4:26 pm
    [ https://issues.apache.org/jira/browse/LUCENE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Michael McCandless updated LUCENE-3214:
    ---------------------------------------

    Attachment: LUCENE-3214.patch

    Prototype hacked up but working patch.
    Ability to mlock certain fields from the terms dict
    ---------------------------------------------------

    Key: LUCENE-3214
    URL: https://issues.apache.org/jira/browse/LUCENE-3214
    Project: Lucene - Java
    Issue Type: Improvement
    Reporter: Michael McCandless
    Attachments: LUCENE-3214.patch


    This is a hacked up prototype! It works but I'm not sure how to get this to a committable point.
    The patch invokes mlock() (tested only on Linux), locking pages from the terms dictionary file that hold terms for a specified field. You can only do this with MMapDirectory.
    I used this to lock pages for the "id" field in the NRT stress test; it's an alternative to MemoryCodec. But, it requires you set up the OS to allow the app/user to lock pages in RAM.
    It works very well in reducing the NRT reopen latency even when large merges are running...
    --
    This message is automatically generated by JIRA.
    For more information on JIRA, see: http://www.atlassian.com/software/jira



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieslucene
postedJun 18, '11 at 4:24p
activeJun 18, '11 at 4:26p
posts2
users1
websitelucene.apache.org

1 user in discussion

Michael McCandless (JIRA): 2 posts

People

Translate

site design / logo © 2021 Grokbase