At 9:10 PM +0200 5/14/04, Richard Jelinek wrote:
Is there any reason why store grabs that much memory? Looks like if a
clone of the data structure would take place in memory.
Is there any reason why store grabs that much memory? Looks like if a
clone of the data structure would take place in memory.
seems to point to every piece of data you are storing. I don't know
if it's for referencing counting, storing data offsets or both,
didn't go that far. We were storing a hash with 70,000 items, each
pointing at hash of probably 7 items or so. (Not *my* idea of an
optimal data structure.) So roughly half a million items. At free
time Storable was freeing 1.5 million hash entries.
I was recently in the internals because we were getting memory faults
on store in 5.8.0. The cleanup routine was looping through a large
hash, setting the items to undefined prior to freeing the hash, and
the standard perl iterator routine was missing 2 or 3 items that the
undef routine was finding--which resulted in a segfault. Only seemed
to happen on Solaris (8 and 9), not on Linux or MacOSX, and from
looking at the code I suspect it's fixed in 5.8.1 and later--all of
that code has changed a bit to better support Storable. In the
meantime I just patched our copy of Storable to access the underlying
hash structures directly.
P.S. There's something odd about someone with a domain name of
"petamem" complaining performance on low memory machines :-).
--
Kee Hinckley
http://www.messagefire.com/ Next Generation Spam Defense
http://commons.somewhere.com/buzz/ Writings on Technology and Society
I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
