Grokbase Groups Cayenne dev July 2010
FAQ
Hello 2 everyone!

I was studing Group Cache stuff. And it's pretty cool!
Only thing I am missing is marking some NamedQuery as Group Refresh Query.

It can be useful for this case:

I have 2 selects (NamedQueries) for table Artists, which have SHARED
cache policy:

MyArtistsSelectQuery1 from group ArtistsTableGroup:
select * from Artists where isModern = 0

MyArtistsSelectQuery2 from group ArtistsTableGroup:
select * from Artists where isModern = 1

And one insert:
MyArtistsInsertQuery1
insert into Artists values (....) .....


Select NamedQueries are marked by properties in XML
<property name="cayenne.GenericSelectQuery.cacheStrategy"
value="SHARED_CACHE"/>
<property name="cayenne.GenericSelectQuery.cacheGroups"
value="ArtistsTableGroup"/>
And it's cool!!!

So, I would like to mark insert query as trigger for cache group
removingin SHARED cache, like implicit execution of:
domain.getQueryCache().removeGroup("ArtistsTableGroup");
after successful insert.



I haven't find this feature. Am I missed something or shall we add one
more JIRA issue for 3.1?


Evgeny.

Search Discussions

  • Andrus Adamchik at Jul 20, 2010 at 1:49 pm
    Sorry for not replying to this earlier. Per my comment on CAY-1465, I
    suggest just map entity event listeners to do cache invalidation. I am
    personally doing that everywhere.

    Andrus
    On Jul 13, 2010, at 12:56 PM, Evgeny Ryabitskiy wrote:

    Hello 2 everyone!

    I was studing Group Cache stuff. And it's pretty cool!
    Only thing I am missing is marking some NamedQuery as Group Refresh
    Query.

    It can be useful for this case:

    I have 2 selects (NamedQueries) for table Artists, which have SHARED
    cache policy:

    MyArtistsSelectQuery1 from group ArtistsTableGroup:
    select * from Artists where isModern = 0

    MyArtistsSelectQuery2 from group ArtistsTableGroup:
    select * from Artists where isModern = 1

    And one insert:
    MyArtistsInsertQuery1
    insert into Artists values (....) .....


    Select NamedQueries are marked by properties in XML
    <property name="cayenne.GenericSelectQuery.cacheStrategy"
    value="SHARED_CACHE"/>
    <property name="cayenne.GenericSelectQuery.cacheGroups"
    value="ArtistsTableGroup"/>
    And it's cool!!!

    So, I would like to mark insert query as trigger for cache group
    removingin SHARED cache, like implicit execution of:
    domain.getQueryCache().removeGroup("ArtistsTableGroup");
    after successful insert.



    I haven't find this feature. Am I missed something or shall we add one
    more JIRA issue for 3.1?


    Evgeny.
  • Evgeny Ryabitskiy at Jul 20, 2010 at 2:52 pm
    Thx, for your reply.

    Now, I see only events related with Entity manipulations. Is there a
    way to add listener for NamedQuery execution?

    As I understand Event listeners could be configured in some "static"
    section while Cayenne configuration initialization...
    It's not so bad until you have ~30 cashed tables and almost 100-200
    queries to that tables.
    Problem that it's had to monitor that all Update queries got their
    listener when this information is decomposed between Java Code and XML
    with SQL Templates.

    Idea was to create such architecture where "Persistent" (DBMS related)
    layer is separate from Java logic and caching could be handled by only
    map.xml changes.

    Evgeny.



    2010/7/20 Andrus Adamchik <andrus@objectstyle.org>:
    Sorry for not replying to this earlier. Per my comment on CAY-1465, I
    suggest just map entity event listeners to do cache invalidation. I am
    personally doing that everywhere.

    Andrus
    On Jul 13, 2010, at 12:56 PM, Evgeny Ryabitskiy wrote:

    Hello 2 everyone!

    I was studing Group Cache stuff. And it's pretty cool!
    Only thing I am missing is marking some NamedQuery as Group Refresh Query.

    It can be useful for this case:

    I have 2 selects (NamedQueries) for table Artists, which have SHARED
    cache policy:

    MyArtistsSelectQuery1 from group ArtistsTableGroup:
    select * from Artists where isModern = 0

    MyArtistsSelectQuery2 from group ArtistsTableGroup:
    select * from Artists where isModern = 1

    And one insert:
    MyArtistsInsertQuery1
    insert into Artists values (....) .....


    Select NamedQueries are marked by properties in XML
    <property name="cayenne.GenericSelectQuery.cacheStrategy"
    value="SHARED_CACHE"/>
    <property name="cayenne.GenericSelectQuery.cacheGroups"
    value="ArtistsTableGroup"/>
    And it's cool!!!

    So, I would like to mark insert query as trigger for cache group
    removingin SHARED cache, like implicit execution of:
    domain.getQueryCache().removeGroup("ArtistsTableGroup");
    after successful insert.



    I haven't find this feature. Am I missed something or shall we add one
    more JIRA issue for 3.1?


    Evgeny.
  • Andrus Adamchik at Jul 20, 2010 at 3:17 pm
    Ok, now I finally see what you are saying (sorry for being so
    slow :-)) Yes, a listener will only be notified of entity events and
    running a SQLTemplate won't cause such an event.

    So how do we solve it? I would like to avoid tying a "query execution
    event" directly to cache refresh. Maybe instead we can attach this new
    type of events (at the beginning EJBQL/SQLTemplate events) to the same
    set of listeners as entity events? A listener can be defined to do a
    cache refresh of a certain group, or do something else entirely. Even
    the events can be defined in terms of specific entities. E.g. "query X
    generates 'post-update' event for entity Y" ... or not :-) Something
    to think about...
    Idea was to create such architecture where "Persistent" (DBMS related)
    layer is separate from Java logic and caching could be handled by only
    map.xml changes.

    The note below is a bit of an aside from the main topic. Just a
    general observation.... Interestingly, I have a somewhat opposite
    problem in my apps - I have lots of ad-hoc queries created dynamically
    in the code, and a number of different object modification flows
    working over the same set of entities. So I need different listeners
    for the same entity events across different apps, and as a result I am
    finding that mapping "events" in XML is too inflexible, and I am more
    in favor of in-code configuration of those. I guess at the end we'll
    allow both styles, and allow extra XML files to be merged in the
    shared mapping to address that.

    Andrus

    On Jul 20, 2010, at 5:51 PM, Evgeny Ryabitskiy wrote:
    Thx, for your reply.

    Now, I see only events related with Entity manipulations. Is there a
    way to add listener for NamedQuery execution?

    As I understand Event listeners could be configured in some "static"
    section while Cayenne configuration initialization...
    It's not so bad until you have ~30 cashed tables and almost 100-200
    queries to that tables.
    Problem that it's had to monitor that all Update queries got their
    listener when this information is decomposed between Java Code and XML
    with SQL Templates.

    Idea was to create such architecture where "Persistent" (DBMS related)
    layer is separate from Java logic and caching could be handled by only
    map.xml changes.

    Evgeny.
  • Evgeny Ryabitskiy at Jul 20, 2010 at 7:59 pm

    2010/7/20 Andrus Adamchik <andrus@objectstyle.org>:
    So how do we solve it? I would like to avoid tying a "query execution event"
    directly to cache refresh. Maybe instead we can attach this new type of
    events (at the beginning EJBQL/SQLTemplate events) to the same set of
    listeners as entity events? A listener can be defined to do a cache refresh
    of a certain group, or do something else entirely. Even the events can be
    defined in terms of specific entities. E.g. "query X generates 'post-update'
    event for entity Y" ... or not :-) Something to think about...
    I am glad that it gave us some "food for brain"! :)

    But I should remind that my issue wasn't about adding SQLTemplate events...
    I just show that events can't solve this issue. But even if it could,
    I'm not sure what I would prefer... configuring listeners on
    SQLTemplate events or just put "removeCacheGroup" method invocation
    after each "performNotSelectionQuery".
    Second one seems even easier... and already available in 3.0...

    One more thing... would we pay some CPU cost on SQLTemplate event
    initialization? Like iterating through Collection of Listeners...
    One business operation could generate ~ 8000 SQLTemplate queries. I
    don't wish to lose here any performance!....
    Actually I'm looking for any solution that could speed up them.... but
    it's another topic...

    Evgeny.

    P.S.
    And one more off-top :)
    For me: Most advantage of Cayenne above so popular Hibernate is it
    simplicity. For simple things it does simple actions! And doest it
    fast!
    Maybe this project should stay on this way, and not to follow it's
    neighbor overloading every operation with lot's of features just to be
    super flexible super universal....?
    Please, don't think that I am trying to dictate you where to lead
    Cayenne. Just some opinion :)







    On Jul 20, 2010, at 5:51 PM, Evgeny Ryabitskiy wrote:

    Thx, for your reply.

    Now, I see only events related with Entity manipulations. Is there a
    way to add listener for NamedQuery execution?

    As I understand Event listeners could be configured in some "static"
    section while Cayenne configuration initialization...
    It's not so bad until you have ~30 cashed tables and almost 100-200
    queries to that tables.
    Problem that it's had to monitor that all Update queries got their
    listener when this information is decomposed between Java Code and XML
    with SQL Templates.

    Idea was to create such architecture where "Persistent" (DBMS related)
    layer is separate from Java logic and caching could be handled by only
    map.xml changes.

    Evgeny.
  • Andrus Adamchik at Jul 21, 2010 at 7:38 am
    I totally understand what you are saying about simplicity and
    certainly don't want to sacrifice any of that here. Along these lines
    I am trying to avoid adding special cases of mapping that are not
    generic enough, which actually makes things more complex in the end.
    Both for us, as we end up with too many if/else cross-feature
    interactions throughout the stack, and for our users, as there are too
    many abstractions and no consistent picture of what the framework is
    about. We tried hard to get rid of a few of those in 3.0, such as
    "derived DbEntities" and will keep doing it in 3.1 (such as merging
    EJBQL and SelectQuery). So I am being extra cautious now to avoid the
    next round :-) At the same time this shouldn't stop us from trying
    and experimenting with things of course...

    So back to the issue at hand. Performance... Indeed, there is some
    overhead looking up entity listeners. I think it will be much much
    less with query listeners - a single HashMap lookup vs. recursively
    scanning through the entity class hierarchy (which we should optimize
    too by caching all listeners per class hierarchy). BTW, with listeners
    you may even optimize your 8000 SQLTemplates per transaction scenario,
    as a listener can batch groups to refresh and then refresh them at
    once at the end of the transaction, instead of doing it 8000 times.

    The second aspect is simplicity of mapping. Current entity listeners
    mapping is not entirely intuitive (mainly because of its JPA roots
    with insane hierarchical listener rules), and I'd like to simplify
    that in the future. One angle of attack is to start using annotations
    for callbacks/listeners, so that we don't have to map the method names
    in the Modeler at all. For queries it will be simpler even without
    annotations, as initially we will have a single event type ("post-
    commit"), and you will have to simply specify the listener class (or
    multiple classes) per query and that's it.

    So do you think this is still too complex? (This is an honest
    question... when I looked closer at the problem, I don't feel that
    much opposed to your original solution, just trying to see if we can
    reuse an abstraction that we already have).

    or just put "removeCacheGroup" method invocation
    after each "performNotSelectionQuery".
    Second one seems even easier... and already available in 3.0...
    If all queries invalidate the same cache group, then yes, you should
    totally use it.

    Andrus

    On Jul 20, 2010, at 10:58 PM, Evgeny Ryabitskiy wrote:
    2010/7/20 Andrus Adamchik <andrus@objectstyle.org>:
    So how do we solve it? I would like to avoid tying a "query
    execution event"
    directly to cache refresh. Maybe instead we can attach this new
    type of
    events (at the beginning EJBQL/SQLTemplate events) to the same set of
    listeners as entity events? A listener can be defined to do a cache
    refresh
    of a certain group, or do something else entirely. Even the events
    can be
    defined in terms of specific entities. E.g. "query X generates
    'post-update'
    event for entity Y" ... or not :-) Something to think about...
    I am glad that it gave us some "food for brain"! :)

    But I should remind that my issue wasn't about adding SQLTemplate
    events...
    I just show that events can't solve this issue. But even if it could,
    I'm not sure what I would prefer... configuring listeners on
    SQLTemplate events or just put "removeCacheGroup" method invocation
    after each "performNotSelectionQuery".
    Second one seems even easier... and already available in 3.0...

    One more thing... would we pay some CPU cost on SQLTemplate event
    initialization? Like iterating through Collection of Listeners...
    One business operation could generate ~ 8000 SQLTemplate queries. I
    don't wish to lose here any performance!....
    Actually I'm looking for any solution that could speed up them.... but
    it's another topic...

    Evgeny.

    P.S.
    And one more off-top :)
    For me: Most advantage of Cayenne above so popular Hibernate is it
    simplicity. For simple things it does simple actions! And doest it
    fast!
    Maybe this project should stay on this way, and not to follow it's
    neighbor overloading every operation with lot's of features just to be
    super flexible super universal....?
    Please, don't think that I am trying to dictate you where to lead
    Cayenne. Just some opinion :)







    On Jul 20, 2010, at 5:51 PM, Evgeny Ryabitskiy wrote:

    Thx, for your reply.

    Now, I see only events related with Entity manipulations. Is there a
    way to add listener for NamedQuery execution?

    As I understand Event listeners could be configured in some "static"
    section while Cayenne configuration initialization...
    It's not so bad until you have ~30 cashed tables and almost 100-200
    queries to that tables.
    Problem that it's had to monitor that all Update queries got their
    listener when this information is decomposed between Java Code and
    XML
    with SQL Templates.

    Idea was to create such architecture where "Persistent" (DBMS
    related)
    layer is separate from Java logic and caching could be handled by
    only
    map.xml changes.

    Evgeny.
  • Evgeny Ryabitskiy at Jul 22, 2010 at 9:36 pm

    2010/7/21 Andrus Adamchik <andrus@objectstyle.org>:
    I totally understand what you are saying about simplicity and certainly
    don't want to sacrifice any of that here. Along these lines I am trying to
    avoid adding special cases of mapping that are not generic enough, which
    actually makes things more complex in the end. Both for us, as we end up
    with too many if/else cross-feature interactions throughout the stack, and
    for our users, as there are too many abstractions and no consistent picture
    of what the framework is about. We tried hard to get rid of a few of those
    in 3.0, such as "derived DbEntities" and will keep doing it in 3.1 (such as
    merging EJBQL and SelectQuery). So I am being extra cautious now to avoid
    the next round  :-) At the same time this shouldn't stop us from trying and
    experimenting with things of course...
    It's great to have like-minded person in you :)
    . Performance... Indeed, there is some overhead
    looking up entity listeners. I think it will be much much less with query
    listeners - a single HashMap lookup vs. recursively scanning through the
    entity class hierarchy (which we should optimize too by caching all
    listeners per class hierarchy).
    I'm totally (100%) against losing Performance even on 5% even on 1%!!!
    So I am against adding listeners here....

    BTW, with listeners you may even optimize
    your 8000 SQLTemplates per transaction scenario, as a listener can batch
    groups to refresh and then refresh them at once at the end of the
    transaction, instead of doing it 8000 times.
    Not sure if I understand you... If you are talking about group caching
    per call(transaction) - it's not my case. Queries are mostly different
    and even same queries could/should return different results because
    there could be several inserts/updates between.... Try to catch this
    problem in 32000+ line log :) Really not ready to try it...

    My caching in first way will be used only for dictionary tables in
    SHARED cache and not refreshed too often (not more then once per day).
    In second I may try Context cashing for tables that are not changed
    per operation(transaction). But it's not near feature optimization....

    The second aspect is simplicity of mapping. Current entity listeners mapping
    is not entirely intuitive (mainly because of its JPA roots with insane
    hierarchical listener rules), and I'd like to simplify that in the future.
    One angle of attack is to start using annotations for callbacks/listeners,
    so that we don't have to map the method names in the Modeler at all. For
    queries it will be simpler even without annotations, as initially we will
    have a single event type ("post-commit"), and you will have to simply
    specify the listener class (or multiple classes) per query and that's it.

    So do you think this is still too complex? (This is an honest question...
    when I looked closer at the problem, I don't feel that much opposed to your
    original solution, just trying to see if we can reuse an abstraction that we
    already have).
    Honestly still don't wish to use listeners... in my case it will be
    heap of listeners at one place. Even it will be several heaps (from
    several modules) which should be collected in one.... Sory can't find
    any place for this feature.... :(
    For me it's much better to have such things at local places in code
    (like it's can be done now) or in SQLTemplates (like was in my idea).


    May be I will try explain my point of view.... So you will see it with
    my eyes :)
    Imagine:
    1) You are going to Modeller
    2) Creating simple Row SQL
    3) See options about cache: NO Cache, LOCAL Cache, SHARED Cache

    Cool! But it's actualy only for SELECT queries. So what with
    INSERT/UPDATE/DELETE queries (that could be also in template)?
    For INSERT/UPDATE/DELETE queries such options have no sense.
    But.... They could invalidate cache!!!.. It's actually what they
    usually does... Why not to extend this options for
    INSERT/UPDATE/DELETE in reverse effect...?
    So on SELECT we are caching on INSERT/UPDATE/DELETE we are refreshing
    cache... for me it looks very intuitive and logically......


    If still not looks logically for you, then it was bad idea.... )



    Evgeny.


    On Jul 20, 2010, at 10:58 PM, Evgeny Ryabitskiy wrote:

    2010/7/20 Andrus Adamchik <andrus@objectstyle.org>:
    So how do we solve it? I would like to avoid tying a "query execution
    event"
    directly to cache refresh. Maybe instead we can attach this new type of
    events (at the beginning EJBQL/SQLTemplate events) to the same set of
    listeners as entity events? A listener can be defined to do a cache
    refresh
    of a certain group, or do something else entirely. Even the events can be
    defined in terms of specific entities. E.g. "query X generates
    'post-update'
    event for entity Y" ... or not :-) Something to think about...
    I am glad that it gave us some "food for brain"! :)

    But I should remind that my issue wasn't about adding SQLTemplate
    events...
    I just show that events can't solve this issue. But even if it could,
    I'm not sure what I would prefer... configuring listeners on
    SQLTemplate events or just put "removeCacheGroup" method invocation
    after each "performNotSelectionQuery".
    Second one seems even easier... and already available in 3.0...

    One more thing... would we pay some CPU cost on SQLTemplate event
    initialization? Like iterating through Collection of Listeners...
    One business operation could generate ~ 8000 SQLTemplate queries. I
    don't wish to lose here any performance!....
    Actually I'm looking for any solution that could speed up them.... but
    it's another topic...

    Evgeny.

    P.S.
    And one more off-top :)
    For me: Most advantage of Cayenne above so popular Hibernate is it
    simplicity. For simple things it does simple actions! And doest it
    fast!
    Maybe this project should stay on this way, and not to follow it's
    neighbor overloading every operation with lot's of features just to be
    super flexible super universal....?
    Please, don't think that I am trying to dictate you where to lead
    Cayenne. Just some opinion :)







    On Jul 20, 2010, at 5:51 PM, Evgeny Ryabitskiy wrote:

    Thx, for your reply.

    Now, I see only events related with Entity manipulations. Is there a
    way to add listener for NamedQuery execution?

    As I understand Event listeners could be configured in some "static"
    section while Cayenne configuration initialization...
    It's not so bad until you have ~30 cashed tables and almost 100-200
    queries to that tables.
    Problem that it's had to monitor that all Update queries got their
    listener when this information is decomposed between Java Code and XML
    with SQL Templates.

    Idea was to create such architecture where "Persistent" (DBMS related)
    layer is separate from Java logic and caching could be handled by only
    map.xml changes.

    Evgeny.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriescayenne
postedJul 13, '10 at 9:57a
activeJul 22, '10 at 9:36p
posts7
users2
websitecayenne.apache.org

People

Translate

site design / logo © 2022 Grokbase