Grokbase Groups Perl qa March 2009
FAQ
I'm mostly sending this email because I had an idea, and it's late, and I don't
want to forget.

Today it occured to me that many Test:: extensions are more about better
diagnostic output than better test comparison.

This stinks:

want: foo bar
have: foo bar

So you write some plugin that does the same old comparison but shows:

want: foo\N{SPACE}bar
have: foo\N{KLINGON MOMENT OF SILENCE}bar

What if display types could be provided informationally in the TAP stream. The
stream could include:

want: foo bar
have: foo bar
presentation: Some::Plugin

With the plugin installed, the presentation layer could fix the output to be
useful. Without it, you still have the data displayed and could presumably
re-display the TAP stream later with the presenter plugin in place.

I am not sure whether this is a good idea, or already well-established as a way
to deal with this annoyance. Good night.

--
rjbs

Search Discussions

  • Michael G Schwern at Mar 31, 2009 at 10:23 pm

    Ricardo SIGNES wrote:
    I'm mostly sending this email because I had an idea, and it's late, and I don't
    want to forget.

    Today it occured to me that many Test:: extensions are more about better
    diagnostic output than better test comparison.

    This stinks:

    want: foo bar
    have: foo bar

    So you write some plugin that does the same old comparison but shows:

    want: foo\N{SPACE}bar
    have: foo\N{KLINGON MOMENT OF SILENCE}bar

    What if display types could be provided informationally in the TAP stream. The
    stream could include:

    want: foo bar
    have: foo bar
    presentation: Some::Plugin

    With the plugin installed, the presentation layer could fix the output to be
    useful. Without it, you still have the data displayed and could presumably
    re-display the TAP stream later with the presenter plugin in place.

    I am not sure whether this is a good idea, or already well-established as a way
    to deal with this annoyance. Good night.
    Your annoyance is valid, but you've reversed the presentation responsibility.

    The test should not be dictating presentation. If it has such clear ideas
    about how the data is to be presented it might as well just format it itself.
    And that's what we want to avoid, because the test author is a poor judge
    of how you want the data presented. It also means the test and the
    presentation have to coordinate what Some::Plugin means.

    The whole point of the structured diagnostics is to allow the user of the test
    to decide how it's to be displayed. You should simply tell your TAP reader
    how you want diagnostics formatted. For a textual reader you have to do this
    before the test is run, but imagine a GUI reader where you can use a
    contextual menu to change the display of a single assertion after the fact.

    That said, there is a suggested key to give the test control. "display"
    allows the test to suggest "please display the test diagnostics like this",
    but it's just some text. So...

    want: foo bar
    have: foo bar
    display: |
    want: foo\N{SPACE}bar
    have: foo\N{KLINGON MOMENT OF SILENCE}bar

    But it's really more useful for more complicated formatting. Something simple
    like making subtle whitespace differences visible should be a feature of the
    TAP consumer.


    --
    But there's no sense crying over every mistake.
    You just keep on trying till you run out of cake.
    -- Jonathan Coulton, "Still Alive"
  • David Golden at Apr 1, 2009 at 1:48 am

    On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern wrote:
    But it's really more useful for more complicated formatting.  Something simple
    like making subtle whitespace differences visible should be a feature of the
    TAP consumer.
    Which is why it will be nice when the "consumer" isn't just getting
    text output on a filehandle.

    /me wants structured TAP capture and easy Harness plugins to change
    how things are presented.

    I hope I can finally get CPAN Testers 2.0 far along this year so that
    next year at a QA hackathon, I can spend more time with the
    TAP/Harness team.

    -- David
  • Michael G Schwern at Apr 1, 2009 at 10:31 am

    David Golden wrote:
    On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern wrote:
    But it's really more useful for more complicated formatting. Something simple
    like making subtle whitespace differences visible should be a feature of the
    TAP consumer.
    Which is why it will be nice when the "consumer" isn't just getting
    text output on a filehandle.

    /me wants structured TAP capture and easy Harness plugins to change
    how things are presented.

    I hope I can finally get CPAN Testers 2.0 far along this year so that
    next year at a QA hackathon, I can spend more time with the
    TAP/Harness team.
    TAP::Harness already reads structured diagnostics, if it sees a "TAP version
    13" header. Doesn't really do anything with it, but the data is available.
    It's Test::Builder that's dragging its feet.

    AndyA hacked in some basic diagnostic handling to Test::More with
    Test::More::Diagnostic if you want to play.
    http://search.cpan.org/dist/Test-More-Diagnostic


    --
    You know what the chain of command is? It's the chain I go get and beat you
    with 'til you understand who's in ruttin' command here.
    -- Jayne Cobb, "Firefly"
  • Eric Wilhelm at Apr 1, 2009 at 1:57 am
    # from Michael G Schwern
    # on Tuesday 31 March 2009 15:22:
    want:  foo bar
    have:  foo bar
    display: |
    want: foo\N{SPACE}bar
    have: foo\N{KLINGON MOMENT OF SILENCE}bar

    But it's really more useful for more complicated formatting.
    Something simple like making subtle whitespace differences visible
    should be a feature of the TAP consumer.
    My thoughts exactly. If the Test::* modules causing this itch are truly
    only doing formatting, structured diagnostics should be plenty of input
    for them.

    And this 'display:' key is actually a bit unsettling because now you've
    got preformatted (encoded) formatting coming from the test. But, I can
    see a case where not all is() are created equal -- so perhaps the
    producer would simply tag the result, allowing that to be dispatched to
    a formatter:

    want: foo bar
    have: foo*bar
    tag: utf-klingon

    Then you have some dispatcher on the harness (presumably a builtin or
    one plugin which itself has plugins) to read the 'tag:' and decide
    which code does the formatting -- TAP::Formatter::Dispatch or so.

    Aside: I'm assuming that the 'have: foo bar' in Schwern's example
    includes a unicode nbsp or something.

    --Eric
    --
    The opinions expressed in this e-mail were randomly generated by
    the computer and do not necessarily reflect the views of its owner.
    --Management
    ---------------------------------------------------
    http://scratchcomputing.com
    ---------------------------------------------------
  • Ricardo SIGNES at Apr 5, 2009 at 4:44 pm
    * Michael G Schwern [2009-03-31T18:22:50]
    What if display types could be provided informationally in the TAP stream.
    The stream could include:

    want: foo bar
    have: foo bar
    presentation: Some::Plugin

    With the plugin installed, the presentation layer could fix the output to be
    useful. Without it, you still have the data displayed and could presumably
    re-display the TAP stream later with the presenter plugin in place.
    Your annoyance is valid, but you've reversed the presentation responsibility.

    The test should not be dictating presentation.
    It isn't dictating. It's suggesting a useful default.

    Until cpan testers are submitting TAP harness, which would allow the author to
    redisplay the TAP, this would lower the prereqs for installing a distribution
    (you don't need a presentation-like test plugin to test the module).
    The whole point of the structured diagnostics is to allow the user of the test
    to decide how it's to be displayed. You should simply tell your TAP reader
    how you want diagnostics formatted. For a textual reader you have to do this
    before the test is run, but imagine a GUI reader where you can use a
    contextual menu to change the display of a single assertion after the fact.
    I understand that -- but we don't have those yet, and even if we did, we're not
    getting the real TAP back, yet, from smokers.

    --
    rjbs
  • Michael G Schwern at Apr 7, 2009 at 3:42 pm

    Ricardo SIGNES wrote:
    * Michael G Schwern [2009-03-31T18:22:50]
    What if display types could be provided informationally in the TAP stream.
    The stream could include:

    want: foo bar
    have: foo bar
    presentation: Some::Plugin

    With the plugin installed, the presentation layer could fix the output to be
    useful. Without it, you still have the data displayed and could presumably
    re-display the TAP stream later with the presenter plugin in place.
    Your annoyance is valid, but you've reversed the presentation responsibility.

    The test should not be dictating presentation.
    It isn't dictating. It's suggesting a useful default.

    Until cpan testers are submitting TAP harness, which would allow the author to
    redisplay the TAP, this would lower the prereqs for installing a distribution
    (you don't need a presentation-like test plugin to test the module).
    The presence of a presentation keyword doesn't lower the prereqs over not
    having it. At best it keeps it flat. By making it a suggestion (which is
    fine) it adds a "recommended" dependency on the plugin module.

    Even if it is "a useful default" what does "Some::Plugin" mean and to whom?
    TAP::Harness? What about some other TAP consumer? The alternative is to come
    up with a generic set of display modes understood by TAP parsers which we're
    no where near ready for.

    The simple way out, which the OP probably didn't realize, is to simply say
    "Presentation" which makes it unofficial [1] and then you can do whatever you
    want without worrying about the far reaching consequences.

    I will confess, what I'm feeling here is that this is going in the direction
    of xUnit where the display modes are locked into the testing system the author
    has chosen.

    The whole point of the structured diagnostics is to allow the user of the test
    to decide how it's to be displayed. You should simply tell your TAP reader
    how you want diagnostics formatted. For a textual reader you have to do this
    before the test is run, but imagine a GUI reader where you can use a
    contextual menu to change the display of a single assertion after the fact.
    I understand that -- but we don't have those yet, and even if we did, we're not
    getting the real TAP back, yet, from smokers.
    Which would be easier: writing a presentation plugin system, writing plugins,
    waiting for Test::Builder to support structured diagnostics, adding a
    presentation keyword to all applicable tests and getting smokers to install
    all the plugins; or changing the smoking modules to send back the TAP using
    the existing TAP::Harness::Archive and waiting for the smokers to upgrade
    normally?


    [1] I know, we broke down over how do designate an official vs unofficial key
    in Oslo. Ovid and I talked about it briefly in Birmingham and /^[a-z]/ ==
    "official", everything else unofficial is on again at least for draft purposes.


    --
    7. Not allowed to add "In accordance with the prophesy" to the end of
    answers I give to a question an officer asks me.
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
    http://skippyslist.com/list/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupqa @
categoriesperl
postedMar 31, '09 at 12:00a
activeApr 7, '09 at 3:42p
posts7
users4
websiteqa.perl.org

People

Translate

site design / logo © 2021 Grokbase