I'm optimizing the tree for performance now. The problem is that markup
creation is very slow.



Everything is overridden for speed, but I still measure 0.7 sec for 100
nodes. That's far too slow.



Widget creation is most time-consuming part, but parsing is also an
expensive and memory-hungry operation.

I think we need a way to stop-resume parsing of DOM (xml/Parse.js).



E.g for the tree we specify

<dojo:Tree lazyParse="true">

<dojo:TreeNode> node level 1

<dojo:TreeNode> node level 2

<dojo:TreeNode> node level 3

</>



When a person clicks expand node1 then widgets runs parsing of one more
level and new widgets are created.



- only one more level to process, not all

- widget should know (or be able to know) if it has subwidgets (to
show +- expand labels).

We can do that in markup with explicit isFolder=true attribute, is there
another way ?





Any widget that puts lots of information in markup and allows lazy parsing
will benefit from this feature.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://dojotoolkit.org/pipermail/dojo-contributors/attachments/20060626/d98eb439/attachment.html

Search Discussions

  • Jesse Kuhnert at Jun 26, 2006 at 7:08 pm
    Forgive me for interjecting as I've only casually been following what you
    have been doing as anything involving performance always excites me .. ;)
    This also isn't meant to detract from what you are doing, just very high
    level observations of the process in general.

    -) The first and easiest thing to look at is why are so many widgets being
    created ? In traditional gui development it is common to have only a small
    set of objects responsible for rendering when dealing with large data
    sets..This allows them to be used as a sort of "stamp" without literally
    creating a 1-to-1 map between data and UI object. I think any toolkit would
    start to choke after too much of this regardless of whether it's an
    interpreted language or closer to the core. Maybe another approach would be
    to start thinking about working on some sort of data manager sort of utility
    to allow seperation of UI and data?

    -) From the very very brief reading I've done in the mozilla codebase at
    least (please keep your BS radar settings on high as a lot of this is either
    inferred or only dimly remembered from method comments / similar patterns of
    logic) while it may be called the "DOM" and have all of the programmatic
    typical dom methods it is nothing like the non-gui DOM versions people use
    to parse xml under normal conditions..(In fact I think the typical case
    still applies as they are using native parsers anyways, but the resultant
    structure isn't a cost-free thing to muck around with ).

    You could try segmenting off your own section of code and try a different
    approach to widget parsing (going even farther into territory where I don't
    know what I'm talking about, at least with browser stuff) of content in
    general. I don't remember what the current parser does, but if you did a
    sort of two phase parse of these widgets it may help as well. The first
    phase for doing all of the creation/manipulation of basic dom structure, and
    the second for doing anything involving style/looking up other
    widgets/etc...The overall goal would be to try and save up all of the
    expensive operations that require lots of cross checks and general browser
    quietness for last when everything is already hopefully fairly quiet. If
    things go well and you are able to definitively determine what is/isn't
    expensive breaking them up like this may yield great results.

    The dom.style references I was making in a previous email were from my
    understanding of how the values are read. (which may be wrong) Something
    along the lines of style size/etc not being calculated until someone asks
    for it specifically was what I remembered. This should then be cached for
    future reads, unless someone does something to change the state of the dom
    in some way. (this state changing logic is probably more advanced than
    forcing re-calculations on ~any~ change, but the basic idea is probably
    there?...) This makes sense, as even underneath the covers the normal gui
    toolkits that we all know and love (ok at least maybe gtk / qt / windows for
    me, though I can't say I'm a very big fan of windows or gtk...) would still
    not make calculating the most efficient thing to do. It's still better than
    doing it during rendering as that would add all kinds of overhead that may
    not even be needed in a lot of instances.

    -) Try the widget parse / xml idea in combination with the data manager
    using JSON structures instead? If it even makes a difference, you could try
    creating JSON data structures that have function hooks embedded within them
    to automatically add/do whatever needs to be done with the data while it's
    being eval'ed instead of having it happen again on a second iteration of the
    data after it's been parsed? I don't know if they xsl implementations
    available via javascript offer any sort of callback hooks but if they did
    that would be awesome.

    Those are my completely made up ideas. Can't say I've tried any of them in
    javascript code at all, but definitely have in a few languages(esp the one
    mozilla is implemented in)- all with more or less the same performant
    results. It's a crap shoot but thought I'd share my disjointed (probably
    unhelpful) thoughts anyways..

    On 6/26/06, Ilia Kantor wrote:

    I'm optimizing the tree for performance now. The problem is that markup
    creation is very slow.



    Everything is overridden for speed, but I still measure 0.7 sec for 100
    nodes. That's far too slow.



    Widget creation is most time-consuming part, but parsing is also an
    expensive and memory-hungry operation.

    I think we need a way to stop-resume parsing of DOM (xml/Parse.js).



    E.g for the tree we specify

    <dojo:Tree lazyParse="true">

    <dojo:TreeNode> node level 1

    <dojo:TreeNode> node level 2

    <dojo:TreeNode> node level 3

    </>



    When a person clicks expand node1 then widgets runs parsing of one more
    level and new widgets are created.



    - only one more level to process, not all

    - widget should know (or be able to know) if it has subwidgets
    (to show +- expand labels).

    We can do that in markup with explicit isFolder=true attribute, is there
    another way ?





    Any widget that puts lots of information in markup and allows lazy parsing
    will benefit from this feature.



    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors


    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://dojotoolkit.org/pipermail/dojo-contributors/attachments/20060626/0b140d46/attachment.html
  • Ilia Kantor at Jun 27, 2006 at 12:51 am
    -) I briefly examined the idea and couldn't find where lies a real win. It
    introduces additional complexity, but how to get performance out of it?
    Amount of rendering is same for both cases. We have to store data somewhere
    anyway, what's the performance diff if it is a widget itself or another
    place ?

    Maybe you could suggest the code for TreeV3 that implements such approach
    and show the performance boost ?



    _____

    From: dojo-contributors-bounces@dojotoolkit.org
    On Behalf Of Jesse
    Kuhnert
    Sent: Tuesday, June 27, 2006 5:09 AM
    To: dojo dev.
    Subject: Re: [dojo-contributors] Lazy Parsing and fast widget creation



    Forgive me for interjecting as I've only casually been following what you
    have been doing as anything involving performance always excites me .. ;)
    This also isn't meant to detract from what you are doing, just very high
    level observations of the process in general.

    -) The first and easiest thing to look at is why are so many widgets being
    created ? In traditional gui development it is common to have only a small
    set of objects responsible for rendering when dealing with large data
    sets..This allows them to be used as a sort of "stamp" without literally
    creating a 1-to-1 map between data and UI object. I think any toolkit would
    start to choke after too much of this regardless of whether it's an
    interpreted language or closer to the core. Maybe another approach would be
    to start thinking about working on some sort of data manager sort of utility
    to allow seperation of UI and data?

    -) From the very very brief reading I've done in the mozilla codebase at
    least (please keep your BS radar settings on high as a lot of this is either
    inferred or only dimly remembered from method comments / similar patterns of
    logic) while it may be called the "DOM" and have all of the programmatic
    typical dom methods it is nothing like the non-gui DOM versions people use
    to parse xml under normal conditions..(In fact I think the typical case
    still applies as they are using native parsers anyways, but the resultant
    structure isn't a cost-free thing to muck around with ).

    You could try segmenting off your own section of code and try a different
    approach to widget parsing (going even farther into territory where I don't
    know what I'm talking about, at least with browser stuff) of content in
    general. I don't remember what the current parser does, but if you did a
    sort of two phase parse of these widgets it may help as well. The first
    phase for doing all of the creation/manipulation of basic dom structure, and
    the second for doing anything involving style/looking up other
    widgets/etc...The overall goal would be to try and save up all of the
    expensive operations that require lots of cross checks and general browser
    quietness for last when everything is already hopefully fairly quiet. If
    things go well and you are able to definitively determine what is/isn't
    expensive breaking them up like this may yield great results.

    The dom.style references I was making in a previous email were from my
    understanding of how the values are read. (which may be wrong) Something
    along the lines of style size/etc not being calculated until someone asks
    for it specifically was what I remembered. This should then be cached for
    future reads, unless someone does something to change the state of the dom
    in some way. (this state changing logic is probably more advanced than
    forcing re-calculations on ~any~ change, but the basic idea is probably
    there?...) This makes sense, as even underneath the covers the normal gui
    toolkits that we all know and love (ok at least maybe gtk / qt / windows for
    me, though I can't say I'm a very big fan of windows or gtk...) would still
    not make calculating the most efficient thing to do. It's still better than
    doing it during rendering as that would add all kinds of overhead that may
    not even be needed in a lot of instances.

    -) Try the widget parse / xml idea in combination with the data manager
    using JSON structures instead? If it even makes a difference, you could try
    creating JSON data structures that have function hooks embedded within them
    to automatically add/do whatever needs to be done with the data while it's
    being eval'ed instead of having it happen again on a second iteration of the
    data after it's been parsed? I don't know if they xsl implementations
    available via javascript offer any sort of callback hooks but if they did
    that would be awesome.

    Those are my completely made up ideas. Can't say I've tried any of them in
    javascript code at all, but definitely have in a few languages(esp the one
    mozilla is implemented in)- all with more or less the same performant
    results. It's a crap shoot but thought I'd share my disjointed (probably
    unhelpful) thoughts anyways..



    On 6/26/06, Ilia Kantor wrote:

    I'm optimizing the tree for performance now. The problem is that markup
    creation is very slow.



    Everything is overridden for speed, but I still measure 0.7 sec for 100
    nodes. That's far too slow.



    Widget creation is most time-consuming part, but parsing is also an
    expensive and memory-hungry operation.

    I think we need a way to stop-resume parsing of DOM (xml/Parse.js).



    E.g for the tree we specify

    <dojo:Tree lazyParse="true">

    <dojo:TreeNode> node level 1

    <dojo:TreeNode> node level 2

    <dojo:TreeNode> node level 3

    </>



    When a person clicks expand node1 then widgets runs parsing of one more
    level and new widgets are created.



    - only one more level to process, not all

    - widget should know (or be able to know) if it has subwidgets (to
    show +- expand labels).

    We can do that in markup with explicit isFolder=true attribute, is there
    another way ?





    Any widget that puts lots of information in markup and allows lazy parsing
    will benefit from this feature.




    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors






    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://dojotoolkit.org/pipermail/dojo-contributors/attachments/20060627/90a60d6b/attachment-0001.html
  • Jesse Kuhnert at Jun 27, 2006 at 6:08 am
    Not nearly as functional client-side wise, but my own tree implementation
    has been used to display roughly 100k records or so...Paging implementations
    were used for most of the data but you get my point..

    http://opencomponentry.com:8080/tacos/app?page=ajax%2FTreeExample&service=page
    On 6/27/06, Ilia Kantor wrote:

    -) I briefly examined the idea and couldn't find where lies a real win.
    It introduces additional complexity, but how to get performance out of it?
    Amount of rendering is same for both cases. We have to store data somewhere
    anyway, what's the performance diff if it is a widget itself or another
    place ?

    Maybe you could suggest the code for TreeV3 that implements such approach
    and show the performance boost ?


    ------------------------------

    *From:* dojo-contributors-bounces@dojotoolkit.org [mailto:
    dojo-contributors-bounces@dojotoolkit.org] *On Behalf Of *Jesse Kuhnert
    *Sent:* Tuesday, June 27, 2006 5:09 AM
    *To:* dojo dev.
    *Subject:* Re: [dojo-contributors] Lazy Parsing and fast widget creation



    Forgive me for interjecting as I've only casually been following what you
    have been doing as anything involving performance always excites me .. ;)
    This also isn't meant to detract from what you are doing, just very high
    level observations of the process in general.

    -) The first and easiest thing to look at is why are so many widgets being
    created ? In traditional gui development it is common to have only a small
    set of objects responsible for rendering when dealing with large data
    sets..This allows them to be used as a sort of "stamp" without literally
    creating a 1-to-1 map between data and UI object. I think any toolkit would
    start to choke after too much of this regardless of whether it's an
    interpreted language or closer to the core. Maybe another approach would be
    to start thinking about working on some sort of data manager sort of utility
    to allow seperation of UI and data?

    -) From the very very brief reading I've done in the mozilla codebase at
    least (please keep your BS radar settings on high as a lot of this is either
    inferred or only dimly remembered from method comments / similar patterns of
    logic) while it may be called the "DOM" and have all of the programmatic
    typical dom methods it is nothing like the non-gui DOM versions people use
    to parse xml under normal conditions..(In fact I think the typical case
    still applies as they are using native parsers anyways, but the resultant
    structure isn't a cost-free thing to muck around with ).

    You could try segmenting off your own section of code and try a different
    approach to widget parsing (going even farther into territory where I don't
    know what I'm talking about, at least with browser stuff) of content in
    general. I don't remember what the current parser does, but if you did a
    sort of two phase parse of these widgets it may help as well. The first
    phase for doing all of the creation/manipulation of basic dom structure, and
    the second for doing anything involving style/looking up other
    widgets/etc...The overall goal would be to try and save up all of the
    expensive operations that require lots of cross checks and general browser
    quietness for last when everything is already hopefully fairly quiet. If
    things go well and you are able to definitively determine what is/isn't
    expensive breaking them up like this may yield great results.

    The dom.style references I was making in a previous email were from my
    understanding of how the values are read. (which may be wrong) Something
    along the lines of style size/etc not being calculated until someone asks
    for it specifically was what I remembered. This should then be cached for
    future reads, unless someone does something to change the state of the dom
    in some way. (this state changing logic is probably more advanced than
    forcing re-calculations on ~any~ change, but the basic idea is probably
    there?...) This makes sense, as even underneath the covers the normal gui
    toolkits that we all know and love (ok at least maybe gtk / qt / windows for
    me, though I can't say I'm a very big fan of windows or gtk...) would still
    not make calculating the most efficient thing to do. It's still better than
    doing it during rendering as that would add all kinds of overhead that may
    not even be needed in a lot of instances.

    -) Try the widget parse / xml idea in combination with the data manager
    using JSON structures instead? If it even makes a difference, you could try
    creating JSON data structures that have function hooks embedded within them
    to automatically add/do whatever needs to be done with the data while it's
    being eval'ed instead of having it happen again on a second iteration of the
    data after it's been parsed? I don't know if they xsl implementations
    available via javascript offer any sort of callback hooks but if they did
    that would be awesome.

    Those are my completely made up ideas. Can't say I've tried any of them in
    javascript code at all, but definitely have in a few languages(esp the one
    mozilla is implemented in)- all with more or less the same performant
    results. It's a crap shoot but thought I'd share my disjointed (probably
    unhelpful) thoughts anyways..

    On 6/26/06, *Ilia Kantor* wrote:

    I'm optimizing the tree for performance now. The problem is that markup
    creation is very slow.



    Everything is overridden for speed, but I still measure 0.7 sec for 100
    nodes. That's far too slow.



    Widget creation is most time-consuming part, but parsing is also an
    expensive and memory-hungry operation.

    I think we need a way to stop-resume parsing of DOM (xml/Parse.js).



    E.g for the tree we specify

    <dojo:Tree lazyParse="true">

    <dojo:TreeNode> node level 1

    <dojo:TreeNode> node level 2

    <dojo:TreeNode> node level 3

    </>



    When a person clicks expand node1 then widgets runs parsing of one more
    level and new widgets are created.



    - only one more level to process, not all

    - widget should know (or be able to know) if it has subwidgets
    (to show +- expand labels).

    We can do that in markup with explicit isFolder=true attribute, is there
    another way ?





    Any widget that puts lots of information in markup and allows lazy parsing
    will benefit from this feature.




    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors




    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.

    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors


    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://dojotoolkit.org/pipermail/dojo-contributors/attachments/20060627/2c6a481e/attachment.html
  • Jesse Kuhnert at Jun 27, 2006 at 7:13 am
    You might even want to look at SortableTable, there may be a rough spot here
    or there in the impl that could be improved but the overall pattern used
    within it looks like something that would handle hundreds of data items a
    little more cleanly.

    Sometimes pausing and trying a different approach with experimentation will
    shed new light on problems..
    On 6/27/06, Jesse Kuhnert wrote:

    Not nearly as functional client-side wise, but my own tree implementation
    has been used to display roughly 100k records or so...Paging implementations
    were used for most of the data but you get my point..


    http://opencomponentry.com:8080/tacos/app?page=ajax%2FTreeExample&service=page

    On 6/27/06, Ilia Kantor wrote:

    -) I briefly examined the idea and couldn't find where lies a real win.
    It introduces additional complexity, but how to get performance out of it?
    Amount of rendering is same for both cases. We have to store data somewhere
    anyway, what's the performance diff if it is a widget itself or another
    place ?

    Maybe you could suggest the code for TreeV3 that implements such
    approach and show the performance boost ?


    ------------------------------

    *From:* dojo-contributors-bounces@dojotoolkit.org [mailto:
    dojo-contributors-bounces@dojotoolkit.org] * On Behalf Of *Jesse Kuhnert
    *Sent:* Tuesday, June 27, 2006 5:09 AM
    *To:* dojo dev.
    *Subject:* Re: [dojo-contributors] Lazy Parsing and fast widget creation



    Forgive me for interjecting as I've only casually been following what
    you have been doing as anything involving performance always excites me ..
    ;) This also isn't meant to detract from what you are doing, just very high
    level observations of the process in general.

    -) The first and easiest thing to look at is why are so many widgets
    being created ? In traditional gui development it is common to have only a
    small set of objects responsible for rendering when dealing with large data
    sets..This allows them to be used as a sort of "stamp" without literally
    creating a 1-to-1 map between data and UI object. I think any toolkit would
    start to choke after too much of this regardless of whether it's an
    interpreted language or closer to the core. Maybe another approach would be
    to start thinking about working on some sort of data manager sort of utility
    to allow seperation of UI and data?

    -) From the very very brief reading I've done in the mozilla codebase at
    least (please keep your BS radar settings on high as a lot of this is either
    inferred or only dimly remembered from method comments / similar patterns of
    logic) while it may be called the "DOM" and have all of the programmatic
    typical dom methods it is nothing like the non-gui DOM versions people use
    to parse xml under normal conditions..(In fact I think the typical case
    still applies as they are using native parsers anyways, but the resultant
    structure isn't a cost-free thing to muck around with ).

    You could try segmenting off your own section of code and try a
    different approach to widget parsing (going even farther into territory
    where I don't know what I'm talking about, at least with browser stuff) of
    content in general. I don't remember what the current parser does, but if
    you did a sort of two phase parse of these widgets it may help as well. The
    first phase for doing all of the creation/manipulation of basic dom
    structure, and the second for doing anything involving style/looking up
    other widgets/etc...The overall goal would be to try and save up all of the
    expensive operations that require lots of cross checks and general browser
    quietness for last when everything is already hopefully fairly quiet. If
    things go well and you are able to definitively determine what is/isn't
    expensive breaking them up like this may yield great results.

    The dom.style references I was making in a previous email were from my
    understanding of how the values are read. (which may be wrong) Something
    along the lines of style size/etc not being calculated until someone asks
    for it specifically was what I remembered. This should then be cached for
    future reads, unless someone does something to change the state of the dom
    in some way. (this state changing logic is probably more advanced than
    forcing re-calculations on ~any~ change, but the basic idea is probably
    there?...) This makes sense, as even underneath the covers the normal gui
    toolkits that we all know and love (ok at least maybe gtk / qt / windows for
    me, though I can't say I'm a very big fan of windows or gtk...) would still
    not make calculating the most efficient thing to do. It's still better than
    doing it during rendering as that would add all kinds of overhead that may
    not even be needed in a lot of instances.

    -) Try the widget parse / xml idea in combination with the data manager
    using JSON structures instead? If it even makes a difference, you could try
    creating JSON data structures that have function hooks embedded within them
    to automatically add/do whatever needs to be done with the data while it's
    being eval'ed instead of having it happen again on a second iteration of the
    data after it's been parsed? I don't know if they xsl implementations
    available via javascript offer any sort of callback hooks but if they did
    that would be awesome.

    Those are my completely made up ideas. Can't say I've tried any of them
    in javascript code at all, but definitely have in a few languages(esp the
    one mozilla is implemented in)- all with more or less the same performant
    results. It's a crap shoot but thought I'd share my disjointed (probably
    unhelpful) thoughts anyways..

    On 6/26/06, *Ilia Kantor* wrote:

    I'm optimizing the tree for performance now. The problem is that markup
    creation is very slow.



    Everything is overridden for speed, but I still measure 0.7 sec for 100
    nodes. That's far too slow.



    Widget creation is most time-consuming part, but parsing is also an
    expensive and memory-hungry operation.

    I think we need a way to stop-resume parsing of DOM (xml/Parse.js).



    E.g for the tree we specify

    <dojo:Tree lazyParse="true">

    <dojo:TreeNode> node level 1

    <dojo:TreeNode> node level 2

    <dojo:TreeNode> node level 3

    </>



    When a person clicks expand node1 then widgets runs parsing of one more
    level and new widgets are created.



    - only one more level to process, not all

    - widget should know (or be able to know) if it has subwidgets
    (to show +- expand labels).

    We can do that in markup with explicit isFolder=true attribute, is there
    another way ?





    Any widget that puts lots of information in markup and allows lazy
    parsing will benefit from this feature.




    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors




    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.

    _______________________________________________
    dojo-contributors mailing list
    dojo-contributors@dojotoolkit.org
    http://dojotoolkit.org/mailman/listinfo/dojo-contributors


    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.


    --
    Jesse Kuhnert
    Tacos/Tapestry, team member/developer

    Open source based consulting work centered around
    dojo/tapestry/tacos/hivemind.
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://dojotoolkit.org/pipermail/dojo-contributors/attachments/20060627/37cff316/attachment-0001.html
  • Bill Keese at Jun 27, 2006 at 6:23 pm

    Jesse Kuhnert wrote:
    Not nearly as functional client-side wise, but my own tree
    implementation has been used to display roughly 100k records or
    so...Paging implementations were used for most of the data but you get
    my point..
    Display 100K records, or just connect to a dataset with 100K records?

    Ilia's Tree can handle lazy-loading too so it should be able to handle
    any sized data set. The recent discussion centered around writing a lot
    of tree data into the markup (ie, *not* using lazy-loading), which
    doesn't seem so important to me. I'm sure it's important to someone,
    but probably not to most of our users.

    --

    Bill

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdojo-contributors @
categoriesdojo
postedJun 26, '06 at 5:17p
activeJun 27, '06 at 6:23p
posts6
users3
websitedojotoolkit.org

People

Translate

site design / logo © 2022 Grokbase