FAQ
[ http://issues.apache.org/jira/browse/LUCENE-305?page=comments#action_12418493 ]

Michael McCandless commented on LUCENE-305:
-------------------------------------------


I'm working towards a patch for this. I changed the name of the issue
to better reflect the goal.

I started with the above approach and made some changes / brought it
up to current Lucene HEAD sources.

I'd like to sanity check the approach to get any feedback:

* Create a new abstract base class called LockFactory (like the
above patch). Purpose of this class is to provide a makeLock()
method that returns a Lock object for given name.
Directory.makeLock just calls this method.

* Create 3 concrete subclasses (for now):

* SimpleFSLockFactory (factored out of FSDirectory)

* SingleProcessLockFactory (factored out of RAMDirectory)

* NoLockFactory

* (then, some time soon: NativeFSLockFactory)

* In Directory, add a setLockFactory() method. This method is
optional. If it is not called, then each Directory has a default
LockFactory that's used instead. FSDirectory defaults to either
SimpleFSLockFactory or NoLockFactory (if setDisabledLocks are
called) and RAMDirectory defaults to SingleProcessLockFactory.

I believe this approach will be fully backwards compatible with
current locking/APIs:

* FSDirectory and RAMDirectory (and anyone who subclasses) use the
same locking they use today. Same lock file names are used and
same defaulting of LOCK_DIR is used.

* If any subclass of Directory has its own makeLock() method, it
will still fully override all of this new logic.

* No existing APIs are changed; only new methods are added and new
classes (*LockFactory).

* Performance should be the same (I'm trying hard not to cause
additional code to be executed in the makeLock() code path). I'll
test this to make sure I'm right :)

This approach means that a user can instantiate a Directory, any
Directory, and then set its locking implementation like this:

Directory.setLockFactory(new SingleProcessLockFactory();

So this should make it easy to use an FSDirectory with a SingleProcess
lock, for example. Users can also subclass their own LockFactory
implementations (eg the MySQL based locking above) and easily use
those for locking.

Please note that the goal here is to make Locking independent of the
Directory, meaning I'm not yet adding any new locking implementations.
The above locking implementations were already pre-existing in Lucene,
just refactored to subclasses of LockFactory. After this (I'll open a
separate issue), I'd like to build a NativeFSLockFactory to address
the original issues/problems we've seen with the
java.io.File.createNewFile() implementation we have now.

This approach differs from the original approach above in that instead
of a global property that sets lock factory class, you explicity call
setLockFactory for a given Directory instance. In addition, this
approach factors out all locking logic (except
getDefaultLockFactory()) inside FSDirectory and RAMDirectory into
LockFactory.

Please let me know if anyone has any concerns / issues with this plan!
I have an initial working version that passes all current unit tests,
but I still need to add some more specific test cases for locking, do
performance testing, etc. Thanks.

[PATCH] Lock Framework - allows custom lock mechanism
-----------------------------------------------------

Key: LUCENE-305
URL: http://issues.apache.org/jira/browse/LUCENE-305
Project: Lucene - Java
Type: Improvement
Components: Store
Versions: unspecified
Environment: Operating System: other
Platform: All
Reporter: Jeff Patterson
Assignee: Lucene Developers
Priority: Minor
Attachments: FSDirectory.java, FSDirectory_patch_file.txt, Lock.java, LockFactory.java, Lock_patch_file.txt, MySQLLocker.java

Proposal: Pluggable Lock Framework for Lucene
Date: Nov 2004
Developer: Jeff Patterson (jeffATwebdoyen.com - http://www.webdoyen.com)
------
Abstract: A framework to allow Lucene users to override the default
FileSystem locking mechanism with a custom lock mechanism.
A Lucene user may develop a new class that extends
org.apache.lucene.store.Lock and implement bodies for the following
methods:
public boolean obtain() - to obtain custom lock
public boolean isLocked() - to detect custom lock
public void release() - to release custom lock
NOTE: When implementing these methods, the developer should make sure to
use the this.getLockName() method on the Lock to identify which lock
is being manipulated (see Modified Files below for more).
After developed, the new class must be added to the classpath (along
with any other supporting classes/libraries needed by the new class),
and the Lucene framework must be alerted of the new class by way of
the "org.apache.lucene.lockClass" -D System property. Example:
java -Dorg.apache.lucene.lockClass=foo.MyCustomLocker LuceneTest
------
Modified Files: The following files were modified to support
this framework (DIFF files at end):
- org.apache.lucene.store.Lock
The member "lockName" and an accompanying protected getter and
setter were added to this class to support naming the lock. This
is transparent to the default lock mechanism and is only useful
when writing a custom lock.
- org.apache.lucene.store.FSDirectory
Instead of instantiating a default Lock, this class now checks
to see if an overridden Lock mechanism is provided, and if so
asks the LockFactory (see below) to provide an overridden Lock
class.
New Files: The following files were added to support this framework:
- org.apache.lucene.store.LockFactory
This class is used to reflect and instantiate by name the custom
Lock implementation. Error handing should be modified in this
class, but that would have required a more extensive code overhaul.
The javadocs for the LockFactory contain a skeleton Java file for
a custom lock implementation.
------
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Search Discussions

  • Chris Hostetter at Jun 29, 2006 at 6:23 pm
    : I'd like to sanity check the approach to get any feedback:

    I don't really know/understand a lot about the current internals of Lucene
    Locking ... but based on the writeup you've added to Jira, your plan of
    attack seems like a sound refactoring approach.

    Your choice to use explicit setters instead of a system properties seems
    to make sense given some of the other semi-recent API changes to eliminate
    the use of system props in lucene (personally i think having both setters
    and system props is cool ... but it doesn't seem like anything about your
    approach would prevent a irectory implimentation from using a sys prop if
    it wanted to)


    -Hoss


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Michael McCandless at Jun 29, 2006 at 9:17 pm

    I don't really know/understand a lot about the current internals of Lucene
    Locking ... but based on the writeup you've added to Jira, your plan of
    attack seems like a sound refactoring approach.

    Your choice to use explicit setters instead of a system properties seems
    to make sense given some of the other semi-recent API changes to eliminate
    the use of system props in lucene (personally i think having both setters
    and system props is cool ... but it doesn't seem like anything about your
    approach would prevent a irectory implimentation from using a sys prop if
    it wanted to)
    Yes, this is very much a refactoring, for starters. After this I'd
    like to create a NativeFSLockFactory that uses native OS locks to
    prevent the "lock timeout" errors when the JVM previously had a hard
    shutdown and to work across remote filesystems (though there's at
    least one case where this may not have worked correctly over NFS; not
    sure about Samba mount).

    If that proves reliable we could eventually make it the default
    locking for FSDirectory.

    I like the idea of both setters & system props. So, if the setter is
    called, we use that LockFactory. Else, if a system prop is set, we
    use that LockFactory. Else we fallback to the original default
    LockFactory implementation (to keep backwards compatibility). I'll
    take this approach.

    Also: apparently Jira does not allow issues to be renamed? I had
    wanted to rename LUCENE-305 this "decouple locking implementation from
    directory implementation".

    Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-dev @
categorieslucene
postedJun 29, '06 at 4:36p
activeJun 29, '06 at 9:17p
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase