FAQ
Hello;

I have a little question on the ObjectContext in 3.0.2.

I'm keeping an ObjectContext in an HttpSession instance's attributes for
re-use between request-response cycles. When I serialize this
ObjectContext instance, it (+ other objects) stores out to < 8k in my
test case, but then deserializing the ObjectContext causes an
OutOfMemory -- even with substantial quantities of memory available.

Has anybody else seen this or is there anything to be weary of before I
look into it further?

Regards

--
Andrew Lindesay

Search Discussions

  • Andrus Adamchik at Feb 16, 2012 at 6:44 am
    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to < 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay
  • Andrus Adamchik at Feb 16, 2012 at 6:50 am
    an OutOfMemory -- even with substantial quantities of memory available.
    This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to < 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay
  • Andrew Lindesay at Feb 16, 2012 at 7:04 pm
    Hi Andrus;

    Thanks for getting back.

    I tried this, but it did not change anything. It is most unusual. When
    I get some more time, I will see if I am able to attach jProfiler and
    see what is actually going on!

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory available.
    This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to< 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrew Lindesay at Feb 17, 2012 at 10:19 am
    Hi;

    It looks like it is holding onto lots (hundreds of megs) of;

    org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
    org.apache.commons.collecitons.map.LRUMap
    org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
    ...velocity
    Not quite understanding yet... as it seems like this wouldn't be
    something stored on an object context.

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory available.
    This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to< 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrus Adamchik at Feb 17, 2012 at 10:29 am
    Interesting... I didn't realize we have a static Velocity runtime in the app. This must to be changed to instance variable with correct scope, (and if needed shut down properly when ServerRuntime is shutdown). Could you please log a Jira - we need to investigate.

    BTW, are you doing hot redeploy of your app? This in combination with static Velocity runtime might produce the observed behavior.

    Andrus

    On Feb 17, 2012, at 1:19 PM, Andrew Lindesay wrote:

    Hi;

    It looks like it is holding onto lots (hundreds of megs) of;

    org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
    org.apache.commons.collecitons.map.LRUMap
    org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
    ...velocity
    Not quite understanding yet... as it seems like this wouldn't be something stored on an object context.

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory available.
    This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to< 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrew Lindesay at Feb 17, 2012 at 10:51 pm
    Hi Andrus;

    It does seem rather strange.

    I'm starting the application with maven (from Eclipse) and
    jetty-maven-plugin. No hot redeploy. No actual container per se. The
    class-loader should be the same so shouldn't be re-creating "isolated
    singletons or statics". I'm using a Filter to serialize-->deserialize
    the attributes on the HttpSession in order to test how some
    infrastructure handles serialization-->deserialization of the session
    attributes.

    If I explicitly don't serialize-->deserialize the ObjectContext then
    there is no problem; so serializing ObjectContext as the root object
    does seem to cause the problem.

    The incremental increase in the memory consumption seems to be roughly
    exponential in relation to the number of deserializations of the
    ObjectContext. That may be a clue; I may download Apache commons and
    see if I can see anything obvious there in the LRU map. Like you say
    though; it probably shouldn't be serializing this stuff anyway though.

    It looks like "SQLTemplateResourceManager" is created via reflection so
    it may be hard to profile for memory usage.

    I'll log a Jira when I get a moment.

    Regards;
    On 17/02/12 11:29 PM, Andrus Adamchik wrote:
    Interesting... I didn't realize we have a static Velocity runtime in the app. This must to be changed to instance variable with correct scope, (and if needed shut down properly when ServerRuntime is shutdown). Could you please log a Jira - we need to investigate.

    BTW, are you doing hot redeploy of your app? This in combination with static Velocity runtime might produce the observed behavior.

    Andrus

    On Feb 17, 2012, at 1:19 PM, Andrew Lindesay wrote:

    Hi;

    It looks like it is holding onto lots (hundreds of megs) of;

    org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
    org.apache.commons.collecitons.map.LRUMap
    org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
    ...velocity
    Not quite understanding yet... as it seems like this wouldn't be something stored on an object context.

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory available.
    This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's attributes for re-use between request-response cycles. When I serialize this ObjectContext instance, it (+ other objects) stores out to< 8k in my test case, but then deserializing the ObjectContext causes an OutOfMemory -- even with substantial quantities of memory available.

    Has anybody else seen this or is there anything to be weary of before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrew Lindesay at Feb 22, 2012 at 10:35 am
    Hi there;

    It does appear that the problem is in an Apache Commons abstract
    superclass. What is happening is that whilst running;

    AbstractReferenceMap.doReadObject(..)

    ...the method...

    AbstractReferenceMap.checkCapacity(..)

    ...is invoked and it seems to be doubling the size of the Map's entry
    array on de-serialization. Through sequential serialization +
    de-serialization this yields the exponential growth in the memory
    consumption.

    It seems that the problem is caused by the 'threshold' value being
    calculated after to populating the Map during de-serialization.

    I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to
    exhibit the same problem.

    The release cycles for commons seems to be quite long these days. Hmmm.....

    Regards;
    On 18/02/12 11:50 AM, Andrew Lindesay wrote:
    Hi Andrus;

    It does seem rather strange.

    I'm starting the application with maven (from Eclipse) and
    jetty-maven-plugin. No hot redeploy. No actual container per se. The
    class-loader should be the same so shouldn't be re-creating "isolated
    singletons or statics". I'm using a Filter to serialize-->deserialize
    the attributes on the HttpSession in order to test how some
    infrastructure handles serialization-->deserialization of the session
    attributes.

    If I explicitly don't serialize-->deserialize the ObjectContext then
    there is no problem; so serializing ObjectContext as the root object
    does seem to cause the problem.

    The incremental increase in the memory consumption seems to be roughly
    exponential in relation to the number of deserializations of the
    ObjectContext. That may be a clue; I may download Apache commons and see
    if I can see anything obvious there in the LRU map. Like you say though;
    it probably shouldn't be serializing this stuff anyway though.

    It looks like "SQLTemplateResourceManager" is created via reflection so
    it may be hard to profile for memory usage.

    I'll log a Jira when I get a moment.

    Regards;
    On 17/02/12 11:29 PM, Andrus Adamchik wrote:
    Interesting... I didn't realize we have a static Velocity runtime in
    the app. This must to be changed to instance variable with correct
    scope, (and if needed shut down properly when ServerRuntime is
    shutdown). Could you please log a Jira - we need to investigate.

    BTW, are you doing hot redeploy of your app? This in combination with
    static Velocity runtime might produce the observed behavior.

    Andrus

    On Feb 17, 2012, at 1:19 PM, Andrew Lindesay wrote:

    Hi;

    It looks like it is holding onto lots (hundreds of megs) of;

    org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
    org.apache.commons.collecitons.map.LRUMap
    org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
    ...velocity
    Not quite understanding yet... as it seems like this wouldn't be
    something stored on an object context.

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory
    available.
    This is a hint actually.. Maybe the problem is not heap size, but
    perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some
    profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's
    attributes for re-use between request-response cycles. When I
    serialize this ObjectContext instance, it (+ other objects) stores
    out to< 8k in my test case, but then deserializing the
    ObjectContext causes an OutOfMemory -- even with substantial
    quantities of memory available.

    Has anybody else seen this or is there anything to be weary of
    before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrus Adamchik at Feb 22, 2012 at 11:22 am
    I think we can improve deserialization performance in general and solve this specific issue if we make ObjectStore.objectMap (this is the AbstractReferenceMap causing trouble) transient. There's no value is saving and restoring this map. There's another map called "changes", which is a regular HashMap, and that needs to be preserved.

    Now the implementation can be a little tricky, as "objectMap" is provided by the DI factory and we need to create an empty map of the same type on deserialization. This may require refactoring of ObjectStore creation flow. Should be doable though...

    Andrus

    On Feb 22, 2012, at 1:34 PM, Andrew Lindesay wrote:

    Hi there;

    It does appear that the problem is in an Apache Commons abstract superclass. What is happening is that whilst running;

    AbstractReferenceMap.doReadObject(..)

    ...the method...

    AbstractReferenceMap.checkCapacity(..)

    ...is invoked and it seems to be doubling the size of the Map's entry array on de-serialization. Through sequential serialization + de-serialization this yields the exponential growth in the memory consumption.

    It seems that the problem is caused by the 'threshold' value being calculated after to populating the Map during de-serialization.

    I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to exhibit the same problem.

    The release cycles for commons seems to be quite long these days. Hmmm.....

    Regards;
    On 18/02/12 11:50 AM, Andrew Lindesay wrote:
    Hi Andrus;

    It does seem rather strange.

    I'm starting the application with maven (from Eclipse) and
    jetty-maven-plugin. No hot redeploy. No actual container per se. The
    class-loader should be the same so shouldn't be re-creating "isolated
    singletons or statics". I'm using a Filter to serialize-->deserialize
    the attributes on the HttpSession in order to test how some
    infrastructure handles serialization-->deserialization of the session
    attributes.

    If I explicitly don't serialize-->deserialize the ObjectContext then
    there is no problem; so serializing ObjectContext as the root object
    does seem to cause the problem.

    The incremental increase in the memory consumption seems to be roughly
    exponential in relation to the number of deserializations of the
    ObjectContext. That may be a clue; I may download Apache commons and see
    if I can see anything obvious there in the LRU map. Like you say though;
    it probably shouldn't be serializing this stuff anyway though.

    It looks like "SQLTemplateResourceManager" is created via reflection so
    it may be hard to profile for memory usage.

    I'll log a Jira when I get a moment.

    Regards;
    On 17/02/12 11:29 PM, Andrus Adamchik wrote:
    Interesting... I didn't realize we have a static Velocity runtime in
    the app. This must to be changed to instance variable with correct
    scope, (and if needed shut down properly when ServerRuntime is
    shutdown). Could you please log a Jira - we need to investigate.

    BTW, are you doing hot redeploy of your app? This in combination with
    static Velocity runtime might produce the observed behavior.

    Andrus

    On Feb 17, 2012, at 1:19 PM, Andrew Lindesay wrote:

    Hi;

    It looks like it is holding onto lots (hundreds of megs) of;

    org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
    org.apache.commons.collecitons.map.LRUMap
    org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
    ...velocity
    Not quite understanding yet... as it seems like this wouldn't be
    something stored on an object context.

    Regards;
    On 16/02/12 7:50 PM, Andrus Adamchik wrote:
    an OutOfMemory -- even with substantial quantities of memory
    available.
    This is a hint actually.. Maybe the problem is not heap size, but
    perm gen size instead? Try increasing that:

    -XX:MaxPermSize=256m
    On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:

    This is strange indeed. Haven't seen such behavior. May need some
    profiling on your end I guess.

    Andrus

    On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
    Hello;

    I have a little question on the ObjectContext in 3.0.2.

    I'm keeping an ObjectContext in an HttpSession instance's
    attributes for re-use between request-response cycles. When I
    serialize this ObjectContext instance, it (+ other objects) stores
    out to< 8k in my test case, but then deserializing the
    ObjectContext causes an OutOfMemory -- even with substantial
    quantities of memory available.

    Has anybody else seen this or is there anything to be weary of
    before I look into it further?

    Regards

    --
    Andrew Lindesay

    --
    Andrew Lindesay
    www.silvereye.co.nz

    --
    Andrew Lindesay
    www.silvereye.co.nz
  • Andrew Lindesay at Feb 23, 2012 at 8:11 am
    Hi Andrus;

    For what it is worth; I have tested commons 3.1 and 3.2 with a small
    test rig and the problem appears to be gone in 3.2 -- although not
    needing to serialize at all would be ideal if you can see a way forward
    that way!

    Regards;
    On 23/02/12 12:22 AM, Andrus Adamchik wrote:
    I think we can improve deserialization performance in general and solve this specific issue if we make ObjectStore.objectMap (this is the AbstractReferenceMap causing trouble) transient. There's no value is saving and restoring this map. There's another map called "changes", which is a regular HashMap, and that needs to be preserved.
    Now the implementation can be a little tricky, as "objectMap" is provided by the DI factory and we need to create an empty map of the same type on deserialization. This may require refactoring of ObjectStore creation flow. Should be doable though...

    --
    Andrew Lindesay
    www.silvereye.co.nz

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriescayenne
postedFeb 14, '12 at 11:56a
activeFeb 23, '12 at 8:11a
posts10
users2
websitecayenne.apache.org

People

Translate

site design / logo © 2022 Grokbase