FAQ
Here is a potentially disruptive change to Cayenne core:

http://svn.apache.org/viewvc?view=revision&revision=1292437
(also see https://issues.apache.org/jira/browse/CAY-1670 )

This removes the major global lock and has a potential to significantly increase throughput (which is an issue, see CAY-1124 by Tore for instance). The implementation is based on a previously untried by us data structure - ConcurrentLinkedHashMap - http://code.google.com/p/concurrentlinkedhashmap/ . It is used by Apache Cassandra, so presumably it works and can handle the load. I internalized this code, placing it under org.apache.cayenne.util.concurrentlinkedhashmap.

I need help with testing though... Maybe someone has any kind of load tests that can emulate multiple DataContexts attached to a single ServerRuntime and performing various select/modify operations? Or maybe you are daring enough to place this in production? I will do the same on my end.

To give some reassurance, our single-threaded unit tests pass 100% with no changes after I swapped commons-collections LRUMap to ConcurrentLinkedHashMap.

Appreciate feedback on this feature!

Thanks,
Andrus

Search Discussions

  • Tore Halset at Feb 22, 2012 at 7:19 pm
    Wow, thanks a lot! Looking forward to test and use this!

    - Tore.
    On Feb 22, 2012, at 19:54, Andrus Adamchik wrote:

    Here is a potentially disruptive change to Cayenne core:

    http://svn.apache.org/viewvc?view=revision&revision=1292437
    (also see https://issues.apache.org/jira/browse/CAY-1670 )

    This removes the major global lock and has a potential to significantly increase throughput (which is an issue, see CAY-1124 by Tore for instance). The implementation is based on a previously untried by us data structure - ConcurrentLinkedHashMap - http://code.google.com/p/concurrentlinkedhashmap/ . It is used by Apache Cassandra, so presumably it works and can handle the load. I internalized this code, placing it under org.apache.cayenne.util.concurrentlinkedhashmap.

    I need help with testing though... Maybe someone has any kind of load tests that can emulate multiple DataContexts attached to a single ServerRuntime and performing various select/modify operations? Or maybe you are daring enough to place this in production? I will do the same on my end.

    To give some reassurance, our single-threaded unit tests pass 100% with no changes after I swapped commons-collections LRUMap to ConcurrentLinkedHashMap.

    Appreciate feedback on this feature!

    Thanks,
    Andrus
  • Aristedes Maniatis at Feb 23, 2012 at 1:10 am

    On 23/02/12 5:54 AM, Andrus Adamchik wrote:
    Here is a potentially disruptive change to Cayenne core:

    http://svn.apache.org/viewvc?view=revision&revision=1292437
    (also see https://issues.apache.org/jira/browse/CAY-1670 )

    This removes the major global lock and has a potential to significantly increase throughput (which is an issue, see CAY-1124 by Tore for instance). The implementation is based on a previously untried by us data structure - ConcurrentLinkedHashMap - http://code.google.com/p/concurrentlinkedhashmap/ . It is used by Apache Cassandra, so presumably it works and can handle the load. I internalized this code, placing it under org.apache.cayenne.util.concurrentlinkedhashmap.

    I need help with testing though... Maybe someone has any kind of load tests that can emulate multiple DataContexts attached to a single ServerRuntime and performing various select/modify operations? Or maybe you are daring enough to place this in production? I will do the same on my end.

    To give some reassurance, our single-threaded unit tests pass 100% with no changes after I swapped commons-collections LRUMap to ConcurrentLinkedHashMap.

    Appreciate feedback on this feature!

    Some questions rather than feedback at this stage:

    * ConcurrentLinkedHashMap seems to have a roadmap http://code.google.com/p/concurrentlinkedhashmap/wiki/Changelog and ongoing development. Since this is the case, is it wise to fork it into the Cayenne source rather than link to the built artifacts and make it easier to update in the future?

    * How does this new code interact and relate to third party caches such as OScache when used in Cayenne? I was never 100% clear about how the old one interacted, and I guess nothing much had changed in this regard, but it would be good to understand this better.

    * I see that version 2.0 will allow for pluggable eviction policies (They will switch between LRU and LIRS). And they mention the idea of weighting by key/value. Any thoughts about whether that would be useful to Cayenne to be able to implement even more flexible policies, particularly given the demise of OScache?


    Ari

    --
    -------------------------->
    Aristedes Maniatis
    GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
  • Andrus Adamchik at Feb 23, 2012 at 6:10 am

    On Feb 23, 2012, at 4:10 AM, Aristedes Maniatis wrote:

    * ConcurrentLinkedHashMap seems to have a roadmap http://code.google.com/p/concurrentlinkedhashmap/wiki/Changelog and ongoing development. Since this is the case, is it wise to fork it into the Cayenne source rather than link to the built artifacts and make it easier to update in the future?
    There are a few considerations. First, my general aversion to expanding the list of dependencies. Aside from that ConcurrentLinkedHashMap is written for Java 1.6. We are supporting 1.5, so I had to transform the original source to conform. So the plan is to merge changes made on the ConcurrentLinkedHashMap side if we need them.
    * How does this new code interact and relate to third party caches such as OScache when used in Cayenne? I was never 100% clear about how the old one interacted, and I guess nothing much had changed in this regard, but it would be good to understand this better.
    This is snapshot cache. It does not interact with query caches at all at this point. But see below.
    * I see that version 2.0 will allow for pluggable eviction policies (They will switch between LRU and LIRS). And they mention the idea of weighting by key/value. Any thoughts about whether that would be useful to Cayenne to be able to implement even more flexible policies, particularly given the demise of OScache?
    A unified (objects, snapshots, query results) caching solution is a #1 feature for Cayenne 3.2 on my list. Once we start looking at that, it may turn out that ConcurrentLinkedHashMap is only a stopgap measure until we can load snapshots into a unified cache.

    In any event it is useful to start playing with it now, as there's another synchronization aspect of the DataRowStore that is provider-agnostic - per-commit snapshots processing used to be done atomically with DataRowStore locked. Now each snapshot is processed independently, so in a multithread environment the order of snapshot processing can get mixed up. My assertion is that it doesn't matter, but I'd like to prove that somehow.

    Andrus
  • Benjamin Manes at Feb 23, 2012 at 9:05 pm
    Hi,
    I'm the author of CLHM, brought to you by a friendly Google alert. :)

    If Cayenne is currently using Guava, then there won't be a dependency increase. I helped rearchitect MapMaker and design its replacement, CacheBuilder, which are based on CLHM's algorithms. This is a more feature-rich implementation with the support of a team at Google (whereas CLHM is my weekend project). We also fixed my mistake of only weighing by value, since I hadn't realized how popular memory-based weighing would be (I was trying to avoid unbounded collections as values).

    Technically CLHM provides more concurrency than CacheBuilder by not relying on segment locks and later using a lock-free ring buffer. These will be an advantage when it is backed by JDK8's new CHM (see ConcurrentHashMapV8) or Cliff Click's NBHM. A JSR-166 implementation will be a marriage of CHMv8, CLHM, and CacheBuilder.

    CLHM is back-ported to JDK5 on every release, so you'll find the JDK5 source under /tags. I do development on /trunk assuming JDK6.

    Feel free to file issues against CLHM (or Guava) if you'd like enhancements to either. Both are used in large production environments.

    Cheers,
    Ben
  • Andrus Adamchik at Feb 24, 2012 at 6:54 am
    Hi Ben,

    Thanks for dropping a note :) It is very good to know that CLHM is a supported and evolving technology.

    There's no dependency on Guava, so it felt like internalizing ConcurrentLinkedHashMap was the right thing at the moment. This is not a final decision, especially since we'll be working on unifying the 2 caches (object cache and query result cache) at some point. But for now it felt like the easiest way to get things going without exposing the internals to the end users.

    With your message contributing to my confidence in CLHM, the most important remaining question for us right now is whether the removal of synchronizations spanning multiple get/put operations will cause any undesired effects. But this is purely a Cayenne thing, untreated to the underlying cache, so we'll have to deal with that on our own.

    Thanks!
    Andrus

    On Feb 24, 2012, at 12:05 AM, Benjamin Manes wrote:
    Hi,
    I'm the author of CLHM, brought to you by a friendly Google alert. :)

    If Cayenne is currently using Guava, then there won't be a dependency increase. I helped rearchitect MapMaker and design its replacement, CacheBuilder, which are based on CLHM's algorithms. This is a more feature-rich implementation with the support of a team at Google (whereas CLHM is my weekend project). We also fixed my mistake of only weighing by value, since I hadn't realized how popular memory-based weighing would be (I was trying to avoid unbounded collections as values).

    Technically CLHM provides more concurrency than CacheBuilder by not relying on segment locks and later using a lock-free ring buffer. These will be an advantage when it is backed by JDK8's new CHM (see ConcurrentHashMapV8) or Cliff Click's NBHM. A JSR-166 implementation will be a marriage of CHMv8, CLHM, and CacheBuilder.

    CLHM is back-ported to JDK5 on every release, so you'll find the JDK5 source under /tags. I do development on /trunk assuming JDK6.

    Feel free to file issues against CLHM (or Guava) if you'd like enhancements to either. Both are used in large production environments.

    Cheers,
    Ben
  • Benjamin Manes at Feb 24, 2012 at 7:40 am
    Fyi,

    Looking at your backport, I suspect that you compiled with JDK6 in JDK5
    mode. Unfortunately this does not exclude library changes, so I suggest
    using JDK5 instead. The Cayenne backport includes java.util.Deque in my
    LinkedDeque implementation, but this interface was not available in JDK5.
    It also is missing an override to AbstractCollection#toArray() to be safe
    from concurrent modifications causing an ArrayIndexOutOfBoundsException,
    which was fixed in JDK6.

    Feel free to borrow from my JDK5 backport for the v1.2 release if it helps.
    [1]

    - Ben

    [1]
    http://code.google.com/p/concurrentlinkedhashmap/source/browse/#svn%2Ftags%2Fconcurrentlinkedhashmap-lru-1.2_jdk5%2Fsrc%2Fjava%2Fcom%2Fgooglecode%2Fconcurrentlinkedhashmap
    On Thu, Feb 23, 2012 at 10:54 PM, Andrus Adamchik wrote:

    Hi Ben,

    Thanks for dropping a note :) It is very good to know that CLHM is a
    supported and evolving technology.

    There's no dependency on Guava, so it felt like internalizing
    ConcurrentLinkedHashMap was the right thing at the moment. This is not a
    final decision, especially since we'll be working on unifying the 2 caches
    (object cache and query result cache) at some point. But for now it felt
    like the easiest way to get things going without exposing the internals to
    the end users.

    With your message contributing to my confidence in CLHM, the most
    important remaining question for us right now is whether the removal of
    synchronizations spanning multiple get/put operations will cause any
    undesired effects. But this is purely a Cayenne thing, untreated to the
    underlying cache, so we'll have to deal with that on our own.

    Thanks!
    Andrus

    On Feb 24, 2012, at 12:05 AM, Benjamin Manes wrote:
    Hi,
    I'm the author of CLHM, brought to you by a friendly Google alert. :)

    If Cayenne is currently using Guava, then there won't be a dependency
    increase. I helped rearchitect MapMaker and design its replacement,
    CacheBuilder, which are based on CLHM's algorithms. This is a more
    feature-rich implementation with the support of a team at Google (whereas
    CLHM is my weekend project). We also fixed my mistake of only weighing by
    value, since I hadn't realized how popular memory-based weighing would be
    (I was trying to avoid unbounded collections as values).
    Technically CLHM provides more concurrency than CacheBuilder by not
    relying on segment locks and later using a lock-free ring buffer. These
    will be an advantage when it is backed by JDK8's new CHM (see
    ConcurrentHashMapV8) or Cliff Click's NBHM. A JSR-166 implementation will
    be a marriage of CHMv8, CLHM, and CacheBuilder.
    CLHM is back-ported to JDK5 on every release, so you'll find the JDK5
    source under /tags. I do development on /trunk assuming JDK6.
    Feel free to file issues against CLHM (or Guava) if you'd like
    enhancements to either. Both are used in large production environments.
    Cheers,
    Ben
  • Andrus Adamchik at Feb 24, 2012 at 7:43 am

    On Feb 24, 2012, at 10:39 AM, Benjamin Manes wrote:


    Looking at your backport, I suspect that you compiled with JDK6 in JDK5 mode.
    Initially I did (on the Mac there's no 1.5 anymore). This was quickly discovered by the CI build, so I fixed that in a few followup commits. The trunk is full 1.5 compatible.

    Andrus
  • Aristedes Maniatis at Feb 24, 2012 at 7:39 am

    On 24/02/12 8:05 AM, Benjamin Manes wrote:
    Hi,
    I'm the author of CLHM, brought to you by a friendly Google alert. :) Hi Ben
    If Cayenne is currently using Guava, then there won't be a dependency increase. I helped rearchitect MapMaker and design its replacement, CacheBuilder, which are based on CLHM's algorithms. This is a more feature-rich implementation with the support of a team at Google (whereas CLHM is my weekend project). We also fixed my mistake of only weighing by value, since I hadn't realized how popular memory-based weighing would be (I was trying to avoid unbounded collections as values).
    Very interesting. Will you be bringing the LIRS features across to CacheBuilder as well?

    Also, would it be hard to limit the size of the cache in bytes rather than number of entries?
    Technically CLHM provides more concurrency than CacheBuilder by not relying on segment locks and later using a lock-free ring buffer. These will be an advantage when it is backed by JDK8's new CHM (see ConcurrentHashMapV8) or Cliff Click's NBHM. A JSR-166 implementation will be a marriage of CHMv8, CLHM, and CacheBuilder.
    Isn't JSR166 already finished?

    CLHM is back-ported to JDK5 on every release, so you'll find the JDK5 source under /tags. I do development on /trunk assuming JDK6.

    Feel free to file issues against CLHM (or Guava) if you'd like enhancements to either. Both are used in large production environments.
    Andrus, I notice that CacheBuilder can be set to concurrency=1 which means that write operations are sequential even though read operations are still concurrent. Would that help at all with the testing concern you have?


    Ari

    Cheers,
    Ben
    --
    -------------------------->
    Aristedes Maniatis
    GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
  • Andrus Adamchik at Feb 24, 2012 at 7:45 am

    On Feb 24, 2012, at 10:38 AM, Aristedes Maniatis wrote:

    Andrus, I notice that CacheBuilder can be set to concurrency=1 which means that write operations are sequential even though read operations are still concurrent. Would that help at all with the testing concern you have?
    Probably not, as when the cache is accessed from multiple threads, the order of snapshot processing will already get mixed up, even before it hits the map.

    Andrus
  • Benjamin Manes at Feb 24, 2012 at 7:51 am

    Very interesting. Will you be bringing the LIRS features across to
    CacheBuilder as well?

    Eventually, yes. Bringing it to CacheBuilder is actually easier than CLHM,
    since the segments avoid a race condition with weighted entries. There was
    a recent thread on Guava about this [1]. However there isn't a rush since
    we view it as a performance optimization, so users shouldn't need to be
    aware of the change. When implemented, it should offer a 5-10% higher hit
    rate but consume 2 extra references per entry.

    [1]
    http://groups.google.com/group/guava-discuss/browse_thread/thread/f92569495def1194

    Also, would it be hard to limit the size of the cache in bytes rather than
    number of entries?

    Sure, you can use a Weigher to do this. However as mentioned I naively
    thought this wouldn't be commonly used, since no other Java caching library
    offered weights. CLHM will only weigh the value, but CacheBuilder will
    weigh the key and value pair.

    Isn't JSR166 already finished?


    Yes, but no. :-)

    This JSR spans multiple JDK releases, with 166e being the current
    iteration. [2]

    [2] http://g.oswego.edu/dl/concurrency-interest/
    On Thu, Feb 23, 2012 at 11:38 PM, Aristedes Maniatis wrote:
    On 24/02/12 8:05 AM, Benjamin Manes wrote:

    Hi,
    I'm the author of CLHM, brought to you by a friendly Google alert. :)
    Hi Ben


    If Cayenne is currently using Guava, then there won't be a dependency
    increase. I helped rearchitect MapMaker and design its replacement,
    CacheBuilder, which are based on CLHM's algorithms. This is a more
    feature-rich implementation with the support of a team at Google (whereas
    CLHM is my weekend project). We also fixed my mistake of only weighing by
    value, since I hadn't realized how popular memory-based weighing would be
    (I was trying to avoid unbounded collections as values).
    Very interesting. Will you be bringing the LIRS features across to
    CacheBuilder as well?

    Also, would it be hard to limit the size of the cache in bytes rather than
    number of entries?


    Technically CLHM provides more concurrency than CacheBuilder by not
    relying on segment locks and later using a lock-free ring buffer. These
    will be an advantage when it is backed by JDK8's new CHM (see
    ConcurrentHashMapV8) or Cliff Click's NBHM. A JSR-166 implementation will
    be a marriage of CHMv8, CLHM, and CacheBuilder.
    Isn't JSR166 already finished?



    CLHM is back-ported to JDK5 on every release, so you'll find the JDK5
    source under /tags. I do development on /trunk assuming JDK6.

    Feel free to file issues against CLHM (or Guava) if you'd like
    enhancements to either. Both are used in large production environments.
    Andrus, I notice that CacheBuilder can be set to concurrency=1 which means
    that write operations are sequential even though read operations are still
    concurrent. Would that help at all with the testing concern you have?


    Ari


    Cheers,
    Ben
    --
    -------------------------->
    Aristedes Maniatis
    GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
  • Andrus Adamchik at Mar 3, 2012 at 7:07 pm
    So, anyone had a chance to try it?

    My colleagues are working on a test right now. Hopefully will have something next week. Also I did some more staring at the code involved, and I guess all we are doing here is making more obvious the lack of guaranteed order of context refresh events. Since cross-context synchronization was asynchronous already (or as I like to call it - "probabilistically consistent" :)), we are not changing the behavior that much, just changing the probability.

    Still would like to hear the feedback from the field.

    Cheers,
    Andrus

    On Feb 22, 2012, at 1:54 PM, Andrus Adamchik wrote:

    Here is a potentially disruptive change to Cayenne core:

    http://svn.apache.org/viewvc?view=revision&revision=1292437
    (also see https://issues.apache.org/jira/browse/CAY-1670 )

    This removes the major global lock and has a potential to significantly increase throughput (which is an issue, see CAY-1124 by Tore for instance). The implementation is based on a previously untried by us data structure - ConcurrentLinkedHashMap - http://code.google.com/p/concurrentlinkedhashmap/ . It is used by Apache Cassandra, so presumably it works and can handle the load. I internalized this code, placing it under org.apache.cayenne.util.concurrentlinkedhashmap.

    I need help with testing though... Maybe someone has any kind of load tests that can emulate multiple DataContexts attached to a single ServerRuntime and performing various select/modify operations? Or maybe you are daring enough to place this in production? I will do the same on my end.

    To give some reassurance, our single-threaded unit tests pass 100% with no changes after I swapped commons-collections LRUMap to ConcurrentLinkedHashMap.

    Appreciate feedback on this feature!

    Thanks,
    Andrus

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriescayenne
postedFeb 22, '12 at 6:55p
activeMar 3, '12 at 7:07p
posts12
users4
websitecayenne.apache.org

People

Translate

site design / logo © 2022 Grokbase