[Cayenne-user] Question on ObjectContext and Deserialization

This is strange indeed. Haven't seen such behavior. May need some profiling on your end I guess.

Andrus

This is a hint actually.. Maybe the problem is not heap size, but perm gen size instead? Try increasing that:

-XX:MaxPermSize=256m

Hi Andrus;

Thanks for getting back.

I tried this, but it did not change anything. It is most unusual. When
I get some more time, I will see if I am able to attach jProfiler and
see what is actually going on!

Regards;

Hi;

It looks like it is holding onto lots (hundreds of megs) of;

org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
Not quite understanding yet... as it seems like this wouldn't be
something stored on an object context.

Regards;

Interesting... I didn't realize we have a static Velocity runtime in the app. This must to be changed to instance variable with correct scope, (and if needed shut down properly when ServerRuntime is shutdown). Could you please log a Jira - we need to investigate.

BTW, are you doing hot redeploy of your app? This in combination with static Velocity runtime might produce the observed behavior.

Andrus

Hi Andrus;

It does seem rather strange.

I'm starting the application with maven (from Eclipse) and
jetty-maven-plugin. No hot redeploy. No actual container per se. The
class-loader should be the same so shouldn't be re-creating "isolated
singletons or statics". I'm using a Filter to serialize-->deserialize
the attributes on the HttpSession in order to test how some
infrastructure handles serialization-->deserialization of the session
attributes.

If I explicitly don't serialize-->deserialize the ObjectContext then
there is no problem; so serializing ObjectContext as the root object
does seem to cause the problem.

The incremental increase in the memory consumption seems to be roughly
exponential in relation to the number of deserializations of the
ObjectContext. That may be a clue; I may download Apache commons and
see if I can see anything obvious there in the LRU map. Like you say
though; it probably shouldn't be serializing this stuff anyway though.

It looks like "SQLTemplateResourceManager" is created via reflection so
it may be hard to profile for memory usage.

I'll log a Jira when I get a moment.

Regards;

Hi there;

It does appear that the problem is in an Apache Commons abstract
superclass. What is happening is that whilst running;

AbstractReferenceMap.doReadObject(..)

...the method...

AbstractReferenceMap.checkCapacity(..)

...is invoked and it seems to be doubling the size of the Map's entry
array on de-serialization. Through sequential serialization +
de-serialization this yields the exponential growth in the memory
consumption.

It seems that the problem is caused by the 'threshold' value being
calculated after to populating the Map during de-serialization.

I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to
exhibit the same problem.

The release cycles for commons seems to be quite long these days. Hmmm.....

Regards;

I think we can improve deserialization performance in general and solve this specific issue if we make ObjectStore.objectMap (this is the AbstractReferenceMap causing trouble) transient. There's no value is saving and restoring this map. There's another map called "changes", which is a regular HashMap, and that needs to be preserved.

Now the implementation can be a little tricky, as "objectMap" is provided by the DI factory and we need to create an empty map of the same type on deserialization. This may require refactoring of ObjectStore creation flow. Should be doable though...

Andrus

Hi Andrus;

For what it is worth; I have tested commons 3.1 and 3.2 with a small
test rig and the problem appears to be gone in 3.2 -- although not
needing to serialize at all would be ideal if you can see a way forward
that way!

Regards;