FAQ
DateTimers,

While profiling Sqitch, I found that the biggest suck on its time (now that I have removed Moose and Mouse in favor of Moo) is DateTime. The reason? It loads all of the locales in its BEGIN block. Run this to see for yourself:

     perl -d:NYTProf -e 'use DateTime;' && nytprofhtml --open

Here's the topline:
Top 15 Subroutines
Calls P F Exclusive
Time Inclusive
Time Subroutine
466 1 1 15.6ms 24.4ms DateTime::Locale::_register
1 1 1 11.2ms 119ms main::BEGIN@1
468 2 2 8.65ms 8.65ms Params::Validate::XS::validate (xsub)
1 1 1 8.29ms 8.50ms DateTime::Locale::BEGIN@11
9 9 6 5.23ms 6.44ms base::import (recurses: max depth 2, inclusive time 1.79ms)
1 1 1 3.65ms 7.98ms DateTime::BEGIN@16
1 1 1 3.53ms 4.54ms DateTime::TimeZone::Local::BEGIN@8
1 1 1 3.22ms 21.6ms DateTime::BEGIN@13
1 1 1 2.99ms 3.08ms DateTime::TimeZone::BEGIN@8
1 1 1 2.97ms 3.19ms DateTime::BEGIN@9
1 1 1 2.91ms 4.26ms DateTime::Locale::BEGIN@10
1 1 1 2.74ms 6.58ms DateTime::Locale::add_aliases
24 24 18 2.73ms 2.97ms Exporter::import
1 1 1 2.53ms 2.87ms Try::Tiny::ScopeGuard::BEGIN@144
1 1 1 2.38ms 3.00ms Data::OptList::BEGIN@10

Does DateTime *really* need to always load all 466 locales on startup? Couldn't it load them on demand?

Thanks,

David

Search Discussions

  • Darren Duncan at Aug 5, 2014 at 10:10 pm
    Coincidentally, see this blog post I saw yesterday:

    http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html

    This has a lot to say about performance issues.

    -- Darren Duncan
  • David E. Wheeler at Aug 5, 2014 at 10:34 pm

    On Aug 5, 2014, at 3:09 PM, Darren Duncan wrote:

    Coincidentally, see this blog post I saw yesterday:

    http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html
    Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support.

    Best,

    David
  • Kaare Rasmussen at Aug 6, 2014 at 9:21 am

    On 08/06/2014 12:34 AM, David E. Wheeler wrote:
    Yeah, I saw that, too, but Sqitch requires localization, which
    Time::Moment does not support. Best, David
    I guess it's not enough for you, but T:M's strftime does support
    locales, it seems.
  • David E. Wheeler at Aug 6, 2014 at 4:12 pm

    On Aug 6, 2014, at 2:21 AM, Kaare Rasmussen wrote:

    Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David
    I guess it's not enough for you, but T:M's strftime does support locales, it seems.
    It does? The docs all say that it uses the C locale. Which is a good default, but unlocalizable, unless I’m missing something.

    Best,

    David
  • Kaare Rasmussen at Aug 6, 2014 at 4:37 pm

    Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support. Best, David
    I guess it's not enough for you, but T:M's strftime does support locales, it seems.
    It does? The docs all say that it uses the C locale. Which is a good default, but unlocalizable, unless I’m missing something.
    Yeah, guess I was sleepy when reading the docs.

    I also guess it's a future development point, but it won't help you now.
  • Darren Duncan at Aug 6, 2014 at 9:02 pm

    On 2014-08-05, 3:34 PM, David E. Wheeler wrote:
    On Aug 5, 2014, at 3:09 PM, Darren Duncan wrote:

    Coincidentally, see this blog post I saw yesterday:

    http://blogs.perl.org/users/chansen/2014/08/timemoment-vs-datetime.html
    Yeah, I saw that, too, but Sqitch requires localization, which Time::Moment does not support.
    By the way, my pointing to the blog post wasn't so much to specifically
    recommend Time::Moment as a solution for you, as that it had information
    comparing resource usage of a number of temporal data solutions, and in
    particular reinforced your comment of DateTime using a lot of resources,
    relative to most others. -- Darren Duncan
  • Olivier Mengué at Aug 5, 2014 at 11:18 pm
    Here is another way to profile DateTime for the modules it loads at compile
    time:

         perl -d:TraceUse -MDateTime -c -e0

    Here is below the output (latest releases of DateTime, DateTime::Locale,
    DateTime::TimeZone on perl 5.20.0).
    DateTime::Locale::Catalog is the module that contains all the locales names.
    David, is it that module that you worry about or have you found something
    worse?

    Modules used from -e:
        1. DateTime 1.10, -e line 0 [main]
        2. strict 1.08, DateTime.pm line 5
        3. warnings 1.23, DateTime.pm line 6
        4. warnings::register 1.03, DateTime.pm line 7
        5. Carp 1.3301, DateTime.pm line 9
        6. Exporter 5.70, Carp.pm line 99
        7. DateTime::Duration 1.10, DateTime.pm line 10
        8. DateTime::Helpers 1.10, DateTime/Duration.pm line 8
        9. Scalar::Util 1.38, DateTime/Helpers.pm line 6
       10. List::Util 1.38, Scalar/Util.pm line 11
       11. XSLoader 0.17, List/Util.pm line 21
       12. Params::Validate 1.10, DateTime/Duration.pm line 9
       13. Module::Implementation 0.07, Params/Validate.pm line 9
       14. Module::Runtime 0.014, Module/Implementation.pm line 12
       23. Params::Validate::XS 1.10, Module/Runtime.pm line 317
       43. Class::Load::XS 0.08, Module/Runtime.pm line 317
       15. Try::Tiny 0.22, Module/Implementation.pm line 13
       16. Sub::Name 0.05, Try/Tiny.pm line 18 (eval 4)
       17. base 2.22, Sub/Name.pm line 49
       18. vars 1.03, base.pm line 4
       19. DynaLoader 1.25, base.pm line 99
       20. Config 5.020000, DynaLoader.pm line 22
       52. DateTime::Locale::en, base.pm line 99
       53. DateTime::Locale::root, base.pm line 99
       21. constant 1.31, Try/Tiny.pm line 144 [Try::Tiny::ScopeGuard]
       22. Params::Validate::Constants 1.10, Params/Validate.pm line 10
       24. overload 1.22, DateTime/Duration.pm line 18
       25. overloading 0.02, overload.pm line 83
       26. DateTime::Locale 0.45, DateTime.pm line 12
       27. DateTime::Locale::Base, DateTime/Locale.pm line 10
       28. List::MoreUtils 0.33, DateTime/Locale/Base.pm line 8
       29. DateTime::Locale::Catalog, DateTime/Locale.pm line 11
       30. utf8 1.13, DateTime/Locale/Catalog.pm line 19
       51. DateTime::Locale::en_US, DateTime/Locale.pm line 280 (eval 21)
       31. DateTime::TimeZone 1.72, DateTime.pm line 13
       32. DateTime::TimeZone::Catalog 1.72, DateTime/TimeZone.pm line 10
       33. DateTime::TimeZone::Floating 1.72, DateTime/TimeZone.pm line 11
       34. parent 0.228, DateTime/TimeZone/Floating.pm line 6
       35. Class::Singleton 1.4, parent.pm line 20
       36. DateTime::TimeZone::OffsetOnly 1.72, parent.pm line 20
       37. DateTime::TimeZone::UTC 1.72,
    DateTime/TimeZone/OffsetOnly.pm line 8
       38. DateTime::TimeZone::Local 1.72, DateTime/TimeZone.pm line 12
       39. Class::Load 0.21, DateTime/TimeZone/Local.pm line 6
       40. Data::OptList 0.109, Class/Load.pm line 10
       41. Params::Util 1.07, Data/OptList.pm line 10
       42. Sub::Install 0.927, Data/OptList.pm line 11
       44. File::Spec 3.47, DateTime/TimeZone/Local.pm line 8
       45. File::Spec::Unix 3.47, File/Spec.pm line 22
       46. POSIX 1.38_03, DateTime.pm line 16
       47. Fcntl 1.11, POSIX.pm line 17
       48. Tie::Hash 1.05, POSIX.pm line 422 [POSIX::SigRt]
       49. integer 1.01, DateTime.pm line 756
       50. DateTime::Infinite 1.10, DateTime.pm line 68
    Possible proxies:
        3 base.pm line 99, sub base::import
        2 Module/Runtime.pm line 317, sub Module::Runtime::require_module
        2 parent.pm line 20, sub parent::import





    2014-08-05 23:06 GMT+02:00 David E. Wheeler <david@justatheory.com>:
    DateTimers,

    While profiling Sqitch, I found that the biggest suck on its time (now
    that I have removed Moose and Mouse in favor of Moo) is DateTime. The
    reason? It loads all of the locales in its BEGIN block. Run this to see for
    yourself:

    perl -d:NYTProf -e 'use DateTime;' && nytprofhtml --open

    Here's the topline:
    Top 15 Subroutines CallsP FExclusive
    Time Inclusive
    TimeSubroutine 4661 115.6ms 24.4ms DateTime::Locale::_register 1 11 11.2ms
    119ms main::BEGIN@1 4682 28.65ms 8.65ms Params::Validate::XS::validate
    (xsub) 1 11 8.29ms8.50ms DateTime::Locale::BEGIN@11 99 65.23ms 6.44ms
    base::import (recurses: max depth 2, inclusive time 1.79ms) 1 11 3.65ms
    7.98ms DateTime::BEGIN@16 11 13.53ms 4.54msDateTime::TimeZone::Local::
    BEGIN@8 1 11 3.22ms21.6ms DateTime::BEGIN@13 11 12.99ms 3.08ms
    DateTime::TimeZone::BEGIN@8 1 11 2.97ms3.19ms DateTime::BEGIN@9 11 12.91ms
    4.26ms DateTime::Locale::BEGIN@10 1 11 2.74ms6.58ms DateTime::Locale::
    add_aliases 2424 182.73ms 2.97ms Exporter::import 1 11 2.53ms2.87ms
    Try::Tiny::ScopeGuard::BEGIN@144 11 12.38ms 3.00ms Data::OptList::BEGIN@10

    Does DateTime *really* need to always load all 466 locales on startup?
    Couldn't it load them on demand?

    Thanks,

    David
  • David E. Wheeler at Aug 5, 2014 at 11:34 pm

    On Aug 5, 2014, at 4:18 PM, Olivier Mengué wrote:

    Here is below the output (latest releases of DateTime, DateTime::Locale, DateTime::TimeZone on perl 5.20.0).
    DateTime::Locale::Catalog is the module that contains all the locales names.
    David, is it that module that you worry about or have you found something worse?
    I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed.

    Best,

    David
  • Dave Rolsky at Aug 7, 2014 at 6:23 pm

    On Tue, 5 Aug 2014, David E. Wheeler wrote:
    On Aug 5, 2014, at 4:18 PM, Olivier Mengué wrote:

    Here is below the output (latest releases of DateTime, DateTime::Locale, DateTime::TimeZone on perl 5.20.0).
    DateTime::Locale::Catalog is the module that contains all the locales names.
    David, is it that module that you worry about or have you found something worse?
    I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed.
    And you would be correct. The individual locale classes are not loaded
    until you call DateTime::Locale->load

    What could speed things up would be to pre-generate the data structures
    that are built by calling ->register on everything in the Catalog module.

    That all said, DT::Locale is woefully crufty and out of date. I haven't
    kept up to date with the CLDR data for years because the approach that
    DT::Locale uses to generate individual locales makes it incredibly hard to
    deal with data format changes in CLDR.

    Nick Patch had mentioned that he was interested in working on proper CLDR
    support for Perl (which is more than just DateTime info), and I was hoping
    he'd take all this over ;)


    -dave

    /*============================================================
    http://VegGuide.org http://blog.urth.org
    Your guide to all that's veg House Absolute(ly Pointless)
    ============================================================*/
  • David E. Wheeler at Aug 9, 2014 at 10:54 pm

    On Aug 7, 2014, at 11:23 AM, Dave Rolsky wrote:

    I think the overhead comes in the DateTime::Locale::_register() function, which is called once for each locale returned by DateTime::Locale::Catalog. I would not think any locales would be loaded until they were needed.
    And you would be correct. The individual locale classes are not loaded until you call DateTime::Locale->load
    I see. So _register does not load them, but is still the startup bottleneck. What on earth is it doing?
    What could speed things up would be to pre-generate the data structures that are built by calling ->register on everything in the Catalog module.
    It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no?
    That all said, DT::Locale is woefully crufty and out of date. I haven't kept up to date with the CLDR data for years because the approach that DT::Locale uses to generate individual locales makes it incredibly hard to deal with data format changes in CLDR.
    Ah. So some locale results will probably be wrong now, eh?
    Nick Patch had mentioned that he was interested in working on proper CLDR support for Perl (which is more than just DateTime info), and I was hoping he'd take all this over ;)
    Is CLDR::Number part of that, perhaps?

       http://search.cpan.org/dist/CLDR-Number/

    I have updated Sqitch to defer loading DateTime at all until it is needed. Alas, it is often needed. Sure would be nice to cut down on this startup time in general.

    Best,

    David
  • Dave Rolsky at Aug 11, 2014 at 5:01 pm

    On Sat, 9 Aug 2014, David E. Wheeler wrote:

    What could speed things up would be to pre-generate the data structures that are built by calling ->register on everything in the Catalog module.
    It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no?
    Or rather than building this all up via repeated calls to _register, the
    Catalog code could just contain the fully realized structure.
    That all said, DT::Locale is woefully crufty and out of date. I haven't kept up to date with the CLDR data for years because the approach that DT::Locale uses to generate individual locales makes it incredibly hard to deal with data format changes in CLDR.
    Ah. So some locale results will probably be wrong now, eh?
    Oh yes.
    Nick Patch had mentioned that he was interested in working on proper CLDR support for Perl (which is more than just DateTime info), and I was hoping he'd take all this over ;)
    Is CLDR::Number part of that, perhaps?

    http://search.cpan.org/dist/CLDR-Number/
    Probably. But CLDR is a huge beast. I really think we need a Perl
    interface to libicu, although there'd be a serious mismatch between how it
    handles dates (POSIX style with no leap seconds) and DateTime. So just
    having the interface would just be a first step.


    -dave

    /*============================================================
    http://VegGuide.org http://blog.urth.org
    Your guide to all that's veg House Absolute(ly Pointless)
    ============================================================*/
  • David E. Wheeler at Aug 11, 2014 at 6:02 pm

    On Aug 11, 2014, at 10:01 AM, Dave Rolsky wrote:

    It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no?
    Or rather than building this all up via repeated calls to _register, the Catalog code could just contain the fully realized structure.
    How hard would that be to do?

    D
  • Dave Rolsky at Aug 11, 2014 at 6:51 pm

    On Mon, 11 Aug 2014, David E. Wheeler wrote:
    On Aug 11, 2014, at 10:01 AM, Dave Rolsky wrote:

    It builds up in-memory data structures for all of the locales? Wow. Yeah, seems like those could be encapsulated in the locales themselves or something, no?
    Or rather than building this all up via repeated calls to _register, the Catalog code could just contain the fully realized structure.
    How hard would that be to do?
    Hard and not hard ;)

    The Catalog is currently generated as part of the code that generates each
    individual locale. Changing that part of the code generator isn't too
    hard. The trick is you have to use the exact version of the CLDR data that
    this generator can parse, which is CLDR 1.7.1. You can probably get it
    from their historical archives or repo.


    -ave

    /*============================================================
    http://VegGuide.org http://blog.urth.org
    Your guide to all that's veg House Absolute(ly Pointless)
    ============================================================*/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdatetime @
categoriesperl
postedAug 5, '14 at 9:07p
activeAug 11, '14 at 6:51p
posts14
users5
websitemetacpan.org...

People

Translate

site design / logo © 2019 Grokbase