Grokbase
x

c...@gmail.com (c...@gmail.com)

Profile | Posts (2)

User Information

Display Name:c...@gmail.com
Partial Email Address:c...@gmail.com
Posts:
2 total
2 in PAR

5 Most Recent

1) c...@gmail.com Re: [perl #50352] Perl 5.10 Storable extremely slow for large trees of data
| +1 vote
------=_Part_722_26894628.1201703689390 Content-Type: text/plain; charset=ISO-8859-1...
Perl 5 Porters
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
------=_Part_722_26894628.1201703689390
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Using the supplied data file:

C:\temp>\perl\bin\perl.exe xmledit.pl
Retrieved.
Benchmark: timing 20 iterations of Storable...
  Storable:  3 wallclock secs ( 1.45 usr +  1.78 sys =  3.23 CPU) @  6.20/s
(n=20)

C:\temp>\perl_560\bin\perl.exe xmledit.pl
Retrieved.
Benchmark: timing 20 iterations of Storable...
  Storable:  1 wallclock secs ( 0.87 usr +  0.00 sys =  0.87 CPU) @ 22.88/s
(n=20)

Both Perls were from ActiveState.

5.6 Build 623's Storable had to have come from PPM -- we wouldn't have
hand-built a production version of Storable or Perl.

------=_Part_722_26894628.1201703689390--
2) c...@gmail.com Re: [perl #50352] Perl 5.10 Storable extremely slow for large trees of data
| +1 vote
------=_Part_12483_10064958.1201631776584 Content-Type: text/plain; charset=ISO-8859-1...
Perl 5 Porters
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
------=_Part_12483_10064958.1201631776584
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Jan 29, 2008 1:07 PM, Steve Hay <SteveHay@planit.com> wrote:

> Yes, the usemymalloc=n means your 5.6 was using the system malloc anyway,
> so that doesn't explain your problem (although the perl malloc would still
> solve it if you were able to use it).
>
> I noticed that you were using 5.6.0 rather than the 5.6.2 that I tried, so
> I thought I'd give 5.6.0 a try too. I found that Storable 2.18 doesn't
> build with it, and I had to go back to 2.12 to get one that works.
> 5.6.0+2.12 (with the system malloc) takes about 0.7 secs here, which is a
> shade faster than 5.6.2+2.18, but not by much.
>
> The part of 2.13 onwards that doesn't build with 5.6.0 is UTF8 stuff
> (bytes_from_utf8 caused a linker error), and I see 2.13's ChangeLog
> mentions handling utf8 data. Perhaps this is the difference? (i.e. Your
> 5.6.0 used an old Storable that doesn't do UTF8 properly, and 5.10.0 is
> slower because it uses the current Storable which does handle it?)
>

That is a possibility.  We're not using any UTF8 stuff here at all -- it's
all 8-bit ASCII -- so I wouldn't have noticed a difference at all over the
years.

I checked the Storable in the 5.6 distribution we're using here -- Version
1.010.  Maybe I can get that version to compile against Perl 5.10?  I
honestly don't know.

------=_Part_12483_10064958.1201631776584--
3) c...@gmail.com Re: [perl #50352] Perl 5.10 Storable extremely slow for large trees of data
| +1 vote
------=_Part_11610_25389409.1201623887859 Content-Type: text/plain; charset=ISO-8859-1...
Perl 5 Porters
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
------=_Part_11610_25389409.1201623887859
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

As a followup to myself, I ran perl -V on the 5.6 build that's nice and
fast.  The usemymalloc=n would seem to indicate that it's already using the
system-supplied malloc.

Summary of my perl5 (revision 5 version 6 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=undef d_sfio=undef uselargefiles=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
  Compiler:
    cc='cl', optimize='-O1 -MD -DNDEBUG', gccversion=
    cppflags='-DWIN32'
    ccflags ='-O1 -MD -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT
-DHAVE_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_
MSVCRT_READFIX'
    stdchar='char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=4
    alignbytes=8, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release
-libpath:"D:\Perl\lib\CORE"  -machine:x86'
    libpth="C:\Program Files\Mts\Lib" "D:\Perl\lib\CORE"
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib
odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl56.lib
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release
-libpath:"D:\Perl\lib\CORE"  -machine:x86'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS PERL_IMPLICIT_CONTEXT
PERL_IMPLICIT_SYS
  Locally applied patches:
        ActivePerl Build 623
  Built under MSWin32
  Compiled at Dec 15 2000 16:27:07
  @INC:
    T:/Perl/lib
    T:/Perl/site/lib
    .






On Jan 29, 2008 11:19 AM, Clinton Pierce <clintp@gmail.com> wrote:

> More detail:
> >
> > There is definitely a difference between 5.6.2 and 5.10.0, but a far
> > more significant difference is brought about by using different mallocs:
> >
> > With 5.10.0 the freeze takes about 2.7 secs with the system malloc and
> > 0.03 secs with perl's malloc.
> > With 5.6.2 it takes about 0.9 secs with the system malloc and 0.03 secs
> > with perl's malloc.
> >
> > Not sure why the system malloc figure is slower with 5.10.0, but if the
> > 0.03 secs is more like the time that you were seeing previously, could
> > it be that your 5.6.x build was using perl's malloc and now your 5.10.0
> > build is using the system malloc? (ActivePerl builds use the system
> > malloc because perl's malloc currently doesn't work with "-D
> > PERL_IMPLICIT_SYS", which is required for the fork() emulation.)
> >
>
> Good find.
>
> And that pretty much leaves me up a creek. I can't really use a
> self-built Perl for Production in my situation, and the supplied one is far,
> far too slow. Maybe I'll have to find another distribution source or stay
> on 5.6.
>
>
>
>

------=_Part_11610_25389409.1201623887859--
4) c...@gmail.com Re: [perl #50352] Perl 5.10 Storable extremely slow for large trees of data
| +1 vote
------=_Part_11588_28197430.1201623548159 Content-Type: text/plain; charset=ISO-8859-1...
Perl 5 Porters
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
------=_Part_11588_28197430.1201623548159
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

>
> More detail:
>
> There is definitely a difference between 5.6.2 and 5.10.0, but a far
> more significant difference is brought about by using different mallocs:
>
> With 5.10.0 the freeze takes about 2.7 secs with the system malloc and
> 0.03 secs with perl's malloc.
> With 5.6.2 it takes about 0.9 secs with the system malloc and 0.03 secs
> with perl's malloc.
>
> Not sure why the system malloc figure is slower with 5.10.0, but if the
> 0.03 secs is more like the time that you were seeing previously, could
> it be that your 5.6.x build was using perl's malloc and now your 5.10.0
> build is using the system malloc? (ActivePerl builds use the system
> malloc because perl's malloc currently doesn't work with "-D
> PERL_IMPLICIT_SYS", which is required for the fork() emulation.)
>

Good find.

And that pretty much leaves me up a creek.  I can't really use a self-built
Perl for Production in my situation, and the supplied one is far, far too
slow.  Maybe I'll have to find another distribution source or stay on 5.6.

------=_Part_11588_28197430.1201623548159--
5) c...@gmail.com Re: [perl #50352] Perl 5.10 Storable extremely slow for large trees of data
| +1 vote
------=_Part_10462_21822573.1201605249893 Content-Type: text/plain; charset=ISO-8859-1...
Perl 5 Porters
[ Profile | Reply to group ] [ Flat  Thread  Threaded ]
------=_Part_10462_21822573.1201605249893
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

I commented to my own bug report, and a sample structure is attached.

Building Storable 2.18 for 5.6 is going to be really, really tough these
days on my Windows machine which is where I found the problem. If I had a
time machine...

On Jan 29, 2008 4:53 AM, Nicholas Clark via RT <perlbug-followup@perl.org>
wrote:

> On Mon, Jan 28, 2008 at 11:26:34AM -0800, Clinton A. Pierce wrote:
>
> > To duplicate a large expat-parsed XML structure (with refs of refs of
> refs),
> >
> > I'm doing the following:
> >
> >
> > my $temp = Storable::freeze $originalXml;
> > my $copyXml = Storable::thaw($temp);
> >
> >
> > And by "large" I mean about 8MB when dumped with Data::Dumper. Under
> Perl
> > 5.6
> > the "freeze" takes fractions of a second. Under Perl 5.10 it takes many
> > seconds
> > (between 5-8 seconds).
>
> Do you have a sample data structure you can attach?
> Do you have results for perl 5.8.8?
>
> 5.10 comes with Storable 2.18. If you download this from CPAN and build it
> for 5.6, is it slow?
>
> Nicholas Clark
>
>
>

------=_Part_10462_21822573.1201605249893--

spacer
Profile | Posts (2)
Home > People > c...@gmail.com