On Mon, Dec 21, 2015 at 5:10 PM, Karen Etheridge wrote:

The problem is that parent_todo is not considered when setting the OK's
in the details in subtests. That means if a subtest ok fails, and the
parent is in todo, the 'ok' and 'actual_ok' are both set to false.

Why is this a bug? After all, it's only the top level of TAP that ever
matters when it comes to the overall test pass/failure, and a subtest's
pass or fail status will be adjusted accordingly for the parent TODO at the
*parent's* level. TODOs don't need to propagate downward to all contained
This bug has nothing to do with TAP. '->details' is used to get a record of
all tests that have passed through the Test::Builder instance. These
details should accurately reflect if the test was a pass or fail, they do
not. By not using parent_TODO here Test::Builder is at the very least being
inconsistent with itself. When a failure occurs in a subtest where the
parent is todo Test::Builder will send the diagnostics and errors to STDOUT
instead of STDERR like it does with any TODO failure. This is the __ONLY__
place where Test::Builder does not account for the parent_TODO on a
subtest, and in that way it is being inconsistent if not buggy.

Further, if we required that TODO state must propagate down to all
contained subtests, that means that subtests must always be aware of that
upper state -- which rules out any use of separate processes to to create
the subtest results, which is conceptually possible (something else,
thinking that it is a top-level test, can provide us the TAP, and then we
nest it as a subtest into another test -- and this should be possible both
with TAP strings and with an event streamer).
I can't control TAP from other sources, but as I said, this bug has nothing
to do with TAP. This has to do with Test::Builders internal understanding
of tests __it__ produced. Outside TAP never has, nor will it ever effect
this, they do not appear in the details structure.


Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 4 | next ›
Discussion Overview
groupcpan-workers @
postedDec 22, '15 at 12:04a
activeDec 22, '15 at 5:07a

2 users in discussion

Chad Granum: 3 posts Karen Etheridge: 1 post



site design / logo © 2021 Grokbase