FAQ
All of my tests pass now, but I'm still hitting a few issues for 3.1
that the tests didn't reveal.

In previous versions (not sure when it changed), there existed the
ability to intercept prefetch queries using
DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
  Those queries are no longer available -- only the original query with
the prefetchTree goes through those methods.

It's the end of the day here, so I haven't traced through the code yet
to see what's going on, but I still need a way to filter all query
types for specific entities to filter out certain entities (appending
"where INVALIDATED = 'N'"). I've got this working for select
queries, relationship queries, objectIdQueries, but not prefetch
queries.

And I'm still wondering what the difference between
willPerformGenericQuery and willPerformQuery might be. So far, I
just forward willPerformGenericQuery requests through my
willPerformQuery code.

Search Discussions

  • Mike Kienenberger at Sep 24, 2013 at 4:01 pm
    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
        (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
        (working copy)
    @@ -27,6 +27,7 @@

      import org.apache.cayenne.CayenneRuntimeException;
      import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
      import org.apache.cayenne.query.Query;
      import org.apache.cayenne.query.QueryMetadata;
      import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

              return q != null ? q : executedQuery;
          }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
      }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
        (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
        (working copy)
    @@ -772,4 +772,14 @@
                  }
              }
          }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
      }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
        (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
        (working copy)
    @@ -49,4 +49,6 @@
           * @throws NullPointerException if a map parameter is null.
           */
          QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
      }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
        (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
        (working copy)
    @@ -114,9 +114,15 @@

              // pass prefetch subtree to enable joint prefetches...
              prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
              // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
              return true;
          }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
        (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
        (working copy)
    @@ -50,4 +50,8 @@
          public QueryEngine engineForDataMap(DataMap map) {
              return new MockQueryEngine();
          }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
      }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Andrus Adamchik at Sep 24, 2013 at 4:08 pm

    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Mike Kienenberger at Sep 24, 2013 at 4:12 pm
    It's a global rule in most cases. There are a couple of instances
    where I have to be able to turn it off temporarily.

    Let me look at attaching qualifiers to the modeler.

    What do you think of my general idea of exposing PrefetchSelectQueries
    to the DataContextDelegate?

    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Mike Kienenberger at Sep 24, 2013 at 5:04 pm
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.

    And I also have one case where I need to be able to disable the
    qualifier on a specific entity for a specific datacontext.
    I don't think this is possible even if the modeler qualifier was
    working in all other cases. I consider doing something else, like an
    SQL template, but I'd still have entities being fetched in this
    situation where foreign keys from that entity wouldn't find a matching
    foreign entity due to qualifier restriction.



    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Andrus Adamchik at Sep 24, 2013 at 5:49 pm

    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction, and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint" prefetches at least.
    What do you think of my general idea of exposing PrefetchSelectQueries to the DataContextDelegate?
    FWIW, I was hoping DataContextDelegate itself is on the way out.

    So I'd start with entity qualifier, as it is intended exactly for what you are trying to do - filtering selects (presuming we fix the bug above). The "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.

    Andrus

    On Sep 24, 2013, at 8:03 PM, Mike Kienenberger wrote:
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.

    And I also have one case where I need to be able to disable the
    qualifier on a specific entity for a specific datacontext.
    I don't think this is possible even if the modeler qualifier was
    working in all other cases. I consider doing something else, like an
    SQL template, but I'd still have entities being fetched in this
    situation where foreign keys from that entity wouldn't find a matching
    foreign entity due to qualifier restriction.



    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Andrus Adamchik at Sep 24, 2013 at 6:05 pm

    On Sep 24, 2013, at 8:48 PM, Andrus Adamchik wrote:

    he "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.
    Elaborating on this a bit… The old Configuration allowed multiple DataDomains, each of those requiring its own DataMap(s) saved in the project upfront. Good idea, but hard to use in practice.

    ServerRuntime (with single DD each) is more user-friendly. By starting multiple independent runtimes you to easily reuse a single mapping project, tweaking each in-memory copy local to each Runtime (as well as tweaking other parameters like cache). 2 Runtimes can reuse a single DataSource (JNDI, or otherwise), etc.

    Andrus
  • Mike Kienenberger at Sep 24, 2013 at 8:17 pm
    Added as Issue CAY-1875 - PrefetchSelectQuery incorrectly applies
    entity qualifiers
    On Tue, Sep 24, 2013 at 1:48 PM, Andrus Adamchik wrote:

    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction, and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint" prefetches at least.
    What do you think of my general idea of exposing PrefetchSelectQueries to the DataContextDelegate?
    FWIW, I was hoping DataContextDelegate itself is on the way out.

    So I'd start with entity qualifier, as it is intended exactly for what you are trying to do - filtering selects (presuming we fix the bug above). The "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.

    Andrus

    On Sep 24, 2013, at 8:03 PM, Mike Kienenberger wrote:
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.

    And I also have one case where I need to be able to disable the
    qualifier on a specific entity for a specific datacontext.
    I don't think this is possible even if the modeler qualifier was
    working in all other cases. I consider doing something else, like an
    SQL template, but I'd still have entities being fetched in this
    situation where foreign keys from that entity wouldn't find a matching
    foreign entity due to qualifier restriction.



    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Mike Kienenberger at Sep 26, 2013 at 8:09 pm
    So I'm investigating this situation.

             PrefetchSelectQuery prefetchQuery = new PrefetchSelectQuery(
                     prefetchPath,
                     relationship);

    I replaced

             prefetchQuery.setQualifier(classDescriptor.getEntity().translateToRelatedEntity(
                     queryQualifier,
                     prefetchPath));

    with this instead:

             prefetchQuery.setQualifier(relationship.getTargetEntity().translateToRelatedEntity(
                     queryQualifier,
                     prefetchPath));

    which caused a number of cayenne tests to fail during build, but seems
    to be generating correct prefetching query qualifiers.
    On Tue, Sep 24, 2013 at 4:16 PM, Mike Kienenberger wrote:
    Added as Issue CAY-1875 - PrefetchSelectQuery incorrectly applies
    entity qualifiers
    On Tue, Sep 24, 2013 at 1:48 PM, Andrus Adamchik wrote:

    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction, and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint" prefetches at least.
    What do you think of my general idea of exposing PrefetchSelectQueries to the DataContextDelegate?
    FWIW, I was hoping DataContextDelegate itself is on the way out.

    So I'd start with entity qualifier, as it is intended exactly for what you are trying to do - filtering selects (presuming we fix the bug above). The "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.

    Andrus

    On Sep 24, 2013, at 8:03 PM, Mike Kienenberger wrote:
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.

    And I also have one case where I need to be able to disable the
    qualifier on a specific entity for a specific datacontext.
    I don't think this is possible even if the modeler qualifier was
    working in all other cases. I consider doing something else, like an
    SQL template, but I'd still have entities being fetched in this
    situation where foreign keys from that entity wouldn't find a matching
    foreign entity due to qualifier restriction.



    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Mike Kienenberger at Sep 26, 2013 at 8:46 pm
    I think the reason this is working for me is because almost all of my
    qualifiers are "invalidated = 'N'" but this isn't the case in the
    tests, and it's grabbing the qualifier from the wrong entity.
    On Thu, Sep 26, 2013 at 4:08 PM, Mike Kienenberger wrote:
    So I'm investigating this situation.

    PrefetchSelectQuery prefetchQuery = new PrefetchSelectQuery(
    prefetchPath,
    relationship);

    I replaced

    prefetchQuery.setQualifier(classDescriptor.getEntity().translateToRelatedEntity(
    queryQualifier,
    prefetchPath));

    with this instead:

    prefetchQuery.setQualifier(relationship.getTargetEntity().translateToRelatedEntity(
    queryQualifier,
    prefetchPath));

    which caused a number of cayenne tests to fail during build, but seems
    to be generating correct prefetching query qualifiers.
    On Tue, Sep 24, 2013 at 4:16 PM, Mike Kienenberger wrote:
    Added as Issue CAY-1875 - PrefetchSelectQuery incorrectly applies
    entity qualifiers
    On Tue, Sep 24, 2013 at 1:48 PM, Andrus Adamchik wrote:

    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction, and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint" prefetches at least.
    What do you think of my general idea of exposing PrefetchSelectQueries to the DataContextDelegate?
    FWIW, I was hoping DataContextDelegate itself is on the way out.

    So I'd start with entity qualifier, as it is intended exactly for what you are trying to do - filtering selects (presuming we fix the bug above). The "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.

    Andrus

    On Sep 24, 2013, at 8:03 PM, Mike Kienenberger wrote:
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.

    And I also have one case where I need to be able to disable the
    qualifier on a specific entity for a specific datacontext.
    I don't think this is possible even if the modeler qualifier was
    working in all other cases. I consider doing something else, like an
    SQL template, but I'd still have entities being fetched in this
    situation where foreign keys from that entity wouldn't find a matching
    foreign entity due to qualifier restriction.



    On Tue, Sep 24, 2013 at 12:07 PM, Andrus Adamchik
    wrote:
    I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'").
    Is this a global rule, or does it depend on some context (like user role)? If it's the former, you can add a qualifier to affected entities in the Modeler.

    Andrus
    On Sep 24, 2013, at 7:00 PM, Mike Kienenberger wrote:

    Here's one possible way to add support for intercepting prefetch
    queries. I'm not entirely certain it's the best way, but I didn't
    see another obvious point.

    What I did was to call
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) before routing
    the newly-created prefetch select query.

    For DataDomainQueryAction, this will call context.willPerformQuery()
    if there's a non-null context.
    For anything else (DataDomainLegacyQueryAction, MockQueryRouter), it's a noop.

    If the returned query is null, then we skip routing the query and
    return either true or false. I picked true since it might be useful
    to process children of the prefetch even if the prefetch is not
    skipped. My own use case is never going to return null, so I'm fine
    with false.

    There's also no reason why I picked
    QueryRouter.willPerformQuery(PrefetchSelectQuery query) instead of
    QueryRouter.willPerformQuery(Query query) other than it made it more
    obvious that this method was only being used for
    PrefetchSelectQueries. But there may be other kinds of queries which
    should also be going through this method. The more I think about
    this, the more reasonable it seems have it be Query since developers
    might be writing their own Query types, and any Queries being created
    internally should be exposed through
    DataContextDelegate.willPerformQuery, and the QueryRouter is the most
    likely place to be able to forward such new queries.

    This has solved my issues with prefetching under 3.1. I'm still open
    to suggestions for solving my specific problem another way in the
    application code (adding database table views isn't an option), but I
    think exposing prefetch queries (as well as others) is something we
    should be supporting in Cayenne.


    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainLegacyQueryAction.java
    (working copy)
    @@ -27,6 +27,7 @@

    import org.apache.cayenne.CayenneRuntimeException;
    import org.apache.cayenne.map.DataMap;
    +import org.apache.cayenne.query.PrefetchSelectQuery;
    import org.apache.cayenne.query.Query;
    import org.apache.cayenne.query.QueryMetadata;
    import org.apache.cayenne.query.QueryRouter;
    @@ -163,4 +164,8 @@

    return q != null ? q : executedQuery;
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/access/DataDomainQueryAction.java
    (working copy)
    @@ -772,4 +772,14 @@
    }
    }
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + // Notify DataContextDelegate that we have created a new
    PrefetchSelectQuery
    + if (null != context) {
    + Query transformedQuery =
    context.nonNullDelegate().willPerformQuery(context, prefetchQuery);
    + return transformedQuery;
    + } else {
    + return prefetchQuery;
    + }
    + }
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/QueryRouter.java
    (working copy)
    @@ -49,4 +49,6 @@
    * @throws NullPointerException if a map parameter is null.
    */
    QueryEngine engineForDataMap(DataMap map);
    +
    + Query willPerformQuery(PrefetchSelectQuery prefetchQuery);
    }
    Index: framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/main/java/org/apache/cayenne/query/SelectQueryPrefetchRouterAction.java
    (working copy)
    @@ -114,9 +114,15 @@

    // pass prefetch subtree to enable joint prefetches...
    prefetchQuery.setPrefetchTree(node);
    -
    +
    + Query transformedQuery = router.willPerformQuery(prefetchQuery);
    + if (null == transformedQuery) {
    + // Not sure if we want to return false instead.
    Returning true seems safer.
    + return true;
    + }
    +
    // route...
    - prefetchQuery.route(router, resolver, null);
    + transformedQuery.route(router, resolver, null);
    return true;
    }

    Index: framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    ===================================================================
    --- framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (revision 1524993)
    +++ framework/cayenne-jdk1.5-unpublished/src/test/java/org/apache/cayenne/query/MockQueryRouter.java
    (working copy)
    @@ -50,4 +50,8 @@
    public QueryEngine engineForDataMap(DataMap map) {
    return new MockQueryEngine();
    }
    +
    + public Query willPerformQuery(PrefetchSelectQuery prefetchQuery) {
    + return prefetchQuery;
    + }
    }


    On Mon, Sep 23, 2013 at 7:04 PM, Mike Kienenberger wrote:
    All of my tests pass now, but I'm still hitting a few issues for 3.1
    that the tests didn't reveal.

    In previous versions (not sure when it changed), there existed the
    ability to intercept prefetch queries using
    DataContextDelegate.willPerformQuery() or willPerformGenericQuery().
    Those queries are no longer available -- only the original query with
    the prefetchTree goes through those methods.

    It's the end of the day here, so I haven't traced through the code yet
    to see what's going on, but I still need a way to filter all query
    types for specific entities to filter out certain entities (appending
    "where INVALIDATED = 'N'"). I've got this working for select
    queries, relationship queries, objectIdQueries, but not prefetch
    queries.

    And I'm still wondering what the difference between
    willPerformGenericQuery and willPerformQuery might be. So far, I
    just forward willPerformGenericQuery requests through my
    willPerformQuery code.
  • Mike Kienenberger at Sep 27, 2013 at 2:30 pm
    I wrote,
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    On Tue, Sep 24, 2013 at 1:48 PM, Andrus Adamchik wrote:
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction,
    and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint"
    prefetches at least.
    I am closing CAY-1875 as invalid. My original testing of entity
    qualifiers must have been flawed, possibly due to interactions with
    other changes I had already made. Once I removed the
    willPerformQuery code entirely, and disabled my application-level
    qualifiers completely, the prefetching using entity qualifiers worked
    as expect in my application.

    The prefetch entity qualifier is actually added in QualiferTranslator,
    not in SelectQueryPrefetchRouterAction. That root entity qualifier is
    rightly inherited from the original query as we don't want to prefetch
    records for unrelated root entity objects. Sorry for the false alarm.
  • Andrus Adamchik at Sep 27, 2013 at 3:40 pm
    Great. Sorry I didn't have enough time to dig deeper into this, so my advice on this matter wasn't that helpful.

    Andrus
    On Sep 27, 2013, at 5:29 PM, Mike Kienenberger wrote:

    I wrote,
    From what testing I've done so far, the qualifier isn't put on for
    prefetch queries, which leaves me at the same situation as when using
    my datacontext delegate.
    On Tue, Sep 24, 2013 at 1:48 PM, Andrus Adamchik wrote:
    This is bad and is not supposed to happen. Appears to be a bug. I am checking SelectQueryPrefetchRouterAction,
    and it applies *root* entity qualifier to prefetch query instead of prefetched entity. Should be a relatively easy fix for "disjoint"
    prefetches at least.
    I am closing CAY-1875 as invalid. My original testing of entity
    qualifiers must have been flawed, possibly due to interactions with
    other changes I had already made. Once I removed the
    willPerformQuery code entirely, and disabled my application-level
    qualifiers completely, the prefetching using entity qualifiers worked
    as expect in my application.

    The prefetch entity qualifier is actually added in QualiferTranslator,
    not in SelectQueryPrefetchRouterAction. That root entity qualifier is
    rightly inherited from the original query as we don't want to prefetch
    records for unrelated root entity objects. Sorry for the false alarm.
  • Mike Kienenberger at Sep 24, 2013 at 7:44 pm
    Great idea! I had forgotten that I could create a separate
    ServerRuntime now. Yes, that would work fine in my case. The
    overhead isn't an issue in this particular case.

    That is one of three cases where I now use DataContextFilter, and
    definitely the one that I would most like to get rid of as it's the
    most complicated one.


    The second case is where I force a testing failure, but I can use
    DataChannelFilter.onQuery() as a replacement.


    The third case is new, due to a change in behavior between 1.1 and now
    (I think the change happened in 1.2). I could probably use a
    DataChannelFilter here too, or I could also use a custom ServerRuntime
    to fix this one, but I'm hoping that we can treat this change as a bug
    and fix it instead. I'll start a separate thread to discuss this
    one.


    On Tue, Sep 24, 2013 at 2:04 PM, Andrus Adamchik wrote:
    On Sep 24, 2013, at 8:48 PM, Andrus Adamchik wrote:

    he "special DataContext" case where the qualifier should be ignored can probably be handled by starting a separate ServerRuntime, where you can strip off the qualifiers. For whatever overhead it creates (ideally not much), this has an advantage of cleanly separating "spaces" with different ORM rules.
    Elaborating on this a bit… The old Configuration allowed multiple DataDomains, each of those requiring its own DataMap(s) saved in the project upfront. Good idea, but hard to use in practice.

    ServerRuntime (with single DD each) is more user-friendly. By starting multiple independent runtimes you to easily reuse a single mapping project, tweaking each in-memory copy local to each Runtime (as well as tweaking other parameters like cache). 2 Runtimes can reuse a single DataSource (JNDI, or otherwise), etc.

    Andrus

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriescayenne
postedSep 23, '13 at 11:05p
activeSep 27, '13 at 3:40p
posts13
users2
websitecayenne.apache.org

People

Translate

site design / logo © 2021 Grokbase