FAQ
Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR
for types (collected during running Python tests on 32-bit Linux).


type count % acc.%


builtin_function_or_method 116012007 36.29% 36.29%
method 52465386 16.41% 52.70%
int 42828741 13.40% 66.09%
str 37017098 11.58% 77.67%
generator 14026583 4.39% 82.06%
list_iterator 8731329 2.73% 84.79%
bytes 7217934 2.26% 87.04%
tuple_iterator 5042563 1.58% 88.62%
float 4672980 1.46% 90.08%
set 3319699 1.04% 91.12%
_io.StringIO 3000369 0.94% 92.06%
str_iterator 2126838 0.67% 92.73%
list 2031059 0.64% 93.36%
dict 1691993 0.53% 93.89%
method-wrapper 1573139 0.49% 94.38%
function 1472062 0.46% 94.84%
traceback 1388278 0.43% 95.28%
tuple 1132071 0.35% 95.63%
memoryview 1092173 0.34% 95.97%
cell 1049496 0.33% 96.30%
managedbuffer 1036889 0.32% 96.63%
bytearray 711969 0.22% 96.85%
range_iterator 496924 0.16% 97.00%
range 483971 0.15% 97.15%
super 472447 0.15% 97.30%
map 449567 0.14% 97.44%
frame 427320 0.13% 97.58%
set_iterator 423392 0.13% 97.71%
Leaf 398705 0.12% 97.83%
symtable 374412 0.12% 97.95%


Types for which free lists already are used: builtin_function_or_method,
method, float, tuple, list, dict, frame. Some free list implementations
(e.g. for tuple) don't call PyObject_INIT/PyObject_INIT_VAR. That is why
numbers are such low for tuples.


Perhaps it is worth to add free lists for other types: int, str, bytes,
generator, list and tuple iterators?


Shortened tables for variable-sized objects (that calls PyObject_INIT_VAR):


int 42828741 13.40%
                               0 425353 0.99% 0.99%
                               1 21399290 49.96% 50.96%
                               2 10496856 24.51% 75.47%
                               3 4873346 11.38% 86.85%
                               4 1021563 2.39% 89.23%
                               5 1246444 2.91% 92.14%
                               6 733676 1.71% 93.85%
                               7 123074 0.29% 94.14%
                               8 139203 0.33% 94.47%
...


bytes 7217934 2.26%
                               0 842 0.01% 0.01%
                               1 179469 2.49% 2.50%
                               2 473306 6.56% 9.06%
                               3 254968 3.53% 12.59%
                               4 1169164 16.20% 28.79%
                               5 72806 1.01% 29.79%
                               6 128668 1.78% 31.58%
                               7 169694 2.35% 33.93%
                               8 155154 2.15% 36.08%
                               9 67320 0.93% 37.01%
                              10 51703 0.72% 37.73%
                              11 42574 0.59% 38.32%
                              12 108947 1.51% 39.83%
                              13 40812 0.57% 40.39%
                              14 126783 1.76% 42.15%
                              15 37873 0.52% 42.67%
                              16 447482 6.20% 48.87%
                              17 194320 2.69% 51.56%
                              18 251685 3.49% 55.05%
                              19 159435 2.21% 57.26%
                              20 212521 2.94% 60.20%
...
                              31 18751 0.26% 67.32%
                              32 159781 2.21% 69.54%
                              33 8332 0.12% 69.65%
...
                              63 19841 0.27% 79.21%
                              64 144982 2.01% 81.22%
                              65 5216 0.07% 81.29%
...
                             127 1354 0.02% 85.44%
                             128 376539 5.22% 90.66%
                             129 17468 0.24% 90.90%
...
                             255 178 0.00% 92.39%
                             256 11993 0.17% 92.55%
                             257 124 0.00% 92.56%
...

Search Discussions

  • Larry Hastings at May 9, 2015 at 7:51 pm

    On 05/09/2015 12:01 PM, Serhiy Storchaka wrote:
    Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR
    for types (collected during running Python tests on 32-bit Linux).

    Can you produce these statistics for a 64-bit build?




    //arry/
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150509/8f0265b4/attachment.html>
  • Serhiy Storchaka at May 9, 2015 at 10:06 pm

    On 09.05.15 22:51, Larry Hastings wrote:
    On 05/09/2015 12:01 PM, Serhiy Storchaka wrote:
    Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR
    for types (collected during running Python tests on 32-bit Linux).
    Can you produce these statistics for a 64-bit build?

    Sorry, no. All my computers are ran under 32-bit Linux.
  • Ian Cordasco at May 9, 2015 at 11:25 pm

    On May 9, 2015 5:07 PM, "Serhiy Storchaka" wrote:
    On 09.05.15 22:51, Larry Hastings wrote:
    On 05/09/2015 12:01 PM, Serhiy Storchaka wrote:

    Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR
    for types (collected during running Python tests on 32-bit Linux).

    Can you produce these statistics for a 64-bit build?

    Sorry, no. All my computers are ran under 32-bit Linux.


    _______________________________________________
    Python-Dev mailing list
    Python-Dev at python.org
    https://mail.python.org/mailman/listinfo/python-dev
    Unsubscribe:
    https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com


    Can you share how you gathered them so someone could run them on a 64-bit
    build?
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150509/4f40418d/attachment.html>
  • Serhiy Storchaka at May 10, 2015 at 6:22 am

    On 10.05.15 02:25, Ian Cordasco wrote:
    Can you share how you gathered them so someone could run them on a
    64-bit build?

    This is quick and dirty patch. It generates 8 GB log file!


    patch --merge -p1 <PyObject_INIT_stat.diff
    make -s -j2
    ./python -Wd -m test.regrtest -w -uall 2>PyObject_INIT.log
    python3 PyObject_INIT_stat.py <PyObject_INIT.log >PyObject_INIT.stat


    Perhaps compiling with COUNT_ALLOCS will produce similar statistic for
    types (but without statistics for sizes) and should be much faster.
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: PyObject_INIT_stat.diff
    Type: text/x-patch
    Size: 952 bytes
    Desc: not available
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150510/e0d035ad/attachment.bin>
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: PyObject_INIT_stat.py
    Type: text/x-python
    Size: 1090 bytes
    Desc: not available
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150510/e0d035ad/attachment.py>
  • Larry Hastings at May 10, 2015 at 3:32 pm

    On 05/09/2015 11:22 PM, Serhiy Storchaka wrote:
    On 10.05.15 02:25, Ian Cordasco wrote:
    Can you share how you gathered them so someone could run them on a
    64-bit build?
    This is quick and dirty patch. It generates 8 GB log file!

    I ran it under 64-bit Linux. Actually it generated a 10GB log file. It
    was stalled at test_multiprocessing_fork for five hours so I killed it.


    What follows are the (apparently) partial results. I think it makes a
    good case for a one-element freelist for 64-bit builds.




    //arry/


    --


    type count % acc.%


    builtin_function_or_method 131028598 37.25% 37.25%
    method 52062496 14.80% 52.05%
    int 47600237 13.53% 65.59%
    str 43841584 12.46% 78.05%
    generator 14038624 3.99% 82.04%
    float 8617481 2.45% 84.49%
    list_iterator 8214121 2.34% 86.83%
    bytes 7884898 2.24% 89.07%
    tuple_iterator 5172174 1.47% 90.54%
    _io.StringIO 3482733 0.99% 91.53%
    set 3335168 0.95% 92.48%
    str_iterator 2856373 0.81% 93.29%
    list 2245981 0.64% 93.93%
    dict 1682253 0.48% 94.41%
    method-wrapper 1574412 0.45% 94.86%
    function 1475393 0.42% 95.28%
    traceback 1417094 0.40% 95.68%
    tuple 1181899 0.34% 96.01%
    memoryview 1103226 0.31% 96.33%
    cell 1047245 0.30% 96.63%
    managedbuffer 1044764 0.30% 96.92%
    bytearray 714337 0.20% 97.13%
    range_iterator 498240 0.14% 97.27%
    range 485325 0.14% 97.41%
    super 473542 0.13% 97.54%
    map 446608 0.13% 97.67%
    frame 426570 0.12% 97.79%
    set_iterator 424526 0.12% 97.91%
    Leaf 391824 0.11% 98.02%
    symtable 376815 0.11% 98.13%


    int 47600237 13.53%
                                   0 294964 0.62% 0.62%
                                   1 36135772 75.92% 76.53%
                                   2 4504046 9.46% 86.00%
                                   3 2109837 4.43% 90.43%
                                   4 1277995 2.68% 93.11%
                                   5 542775 1.14% 94.25%
                                   6 485451 1.02% 95.27%
    ...


    bytes 7884898 2.24%
                                   0 849 0.01% 0.01%
                                   1 250357 3.18% 3.19%
                                   2 450310 5.71% 8.90%
                                   3 259659 3.29% 12.19%
                                   4 1157554 14.68% 26.87%
                                   5 77493 0.98% 27.85%
                                   6 139816 1.77% 29.63%
                                   7 165399 2.10% 31.72%
                                   8 191821 2.43% 34.16%
                                   9 63009 0.80% 34.96%
                                  10 48751 0.62% 35.57%
                                  11 50505 0.64% 36.22%
                                  12 94186 1.19% 37.41%
                                  13 33927 0.43% 37.84%
                                  14 123546 1.57% 39.41%
                                  15 36565 0.46% 39.87%
                                  16 447183 5.67% 45.54%
                                  17 186609 2.37% 47.91%
                                  18 1301737 16.51% 64.42%
    ...


    tuple 1181899 0.34%
                                   0 47 0.00% 0.00%
                                   1 120156 10.17% 10.17%
                                   2 340983 28.85% 39.02%
                                   3 80924 6.85% 45.87%
                                   4 78908 6.68% 52.54%
                                   5 35502 3.00% 55.55%
                                   6 171292 14.49% 70.04%
                                   7 136474 11.55% 81.59%
                                   8 48435 4.10% 85.69%
    ...






    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-dev/attachments/20150510/a1793b1e/attachment-0001.html>
  • Serhiy Storchaka at May 10, 2015 at 8:23 am
    Here is comparable statistic collected from tests ran with an executable
    buil with COUNT_ALLOCS.


    type count % acc.%


    tuple 448855278 29.50% 29.50%
    frame 203515969 13.38% 42.88%
    str 182658237 12.01% 54.89%
    builtin_function_or_method 156724634 10.30% 65.19%
    int 106561963 7.00% 72.19%
    method 88269762 5.80% 78.00%
    list 50340630 3.31% 81.31%
    slice 36650028 2.41% 83.71%
    dict 34429310 2.26% 85.98%
    generator 33035375 2.17% 88.15%
    bytes 29230573 1.92% 90.07%
    function 24953392 1.64% 91.71%
    list_iterator 21236155 1.40% 93.11%
    tuple_iterator 16800947 1.10% 94.21%
    cell 16369317 1.08% 95.29%
    float 7079162 0.47% 95.75%
    _sre.SRE_Match 6342612 0.42% 96.17%
    set 5322829 0.35% 96.52%
    TokenInfo 5077251 0.33% 96.85%
    code 3643664 0.24% 97.09%
    traceback 3510709 0.23% 97.32%
    memoryview 2860799 0.19% 97.51%
    managedbuffer 2762975 0.18% 97.69%
    method-wrapper 2590642 0.17% 97.86%
    Name 1681233 0.11% 97.97%
    bytearray 1598429 0.11% 98.08%
    _io.StringIO 1439456 0.09% 98.17%
    weakref 1341485 0.09% 98.26%
    super 911811 0.06% 98.32%
    range 798254 0.05% 98.37%
  • Stefan Behnel at May 11, 2015 at 7:53 pm

    Serhiy Storchaka schrieb am 09.05.2015 um 21:01:
    Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR for
    types (collected during running Python tests on 32-bit Linux).

    I'm aware that this includes lots of tests for the Python code in the
    stdlib, so these numbers are most likely not too far from what real-world
    code would give, but wouldn't it be even better to collect statistics from
    a (quick) benchmark suite run? Test suites tend to be very regular, flat
    and broad, unlike most production code.


    Stefan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-dev @
categoriespython
postedMay 9, '15 at 7:01p
activeMay 11, '15 at 7:53p
posts8
users4
websitepython.org

People

Translate

site design / logo © 2017 Grokbase