FAQ
The problem is that parent_todo is not considered when setting the OK's
in the details in subtests. That means if a subtest ok fails, and the
parent is in todo, the 'ok' and 'actual_ok' are both set to false.

Why is this a bug? After all, it's only the top level of TAP that ever
matters when it comes to the overall test pass/failure, and a subtest's
pass or fail status will be adjusted accordingly for the parent TODO at the
*parent's* level. TODOs don't need to propagate downward to all contained
subtests.

Further, if we required that TODO state must propagate down to all
contained subtests, that means that subtests must always be aware of that
upper state -- which rules out any use of separate processes to to create
the subtest results, which is conceptually possible (something else,
thinking that it is a top-level test, can provide us the TAP, and then we
nest it as a subtest into another test -- and this should be possible both
with TAP strings and with an event streamer).

On Mon, Dec 21, 2015 at 4:04 PM, Chad Granum wrote:

Test::Builder appends all Ok's to a structure it can return via
$tb->details. These Ok's have the fields 'ok' and 'actual_ok'. 'actual_ok'
is the true/false value passed to $tb->ok, 'ok' is adjusted to be true if
the test was run under TODO.


https://metacpan.org/source/EXODIST/Test-Simple-1.001014/lib/Test/Builder.pm#L845

The bug is in subtest handling. When you start a subtest you essentially
create a new Test::Builder singleton, and the TODO state is clear int he
new one, but it has a field 'parent_TODO' that gets set. This is used to
ensure that failures in a subtest when the parent has TODO set do not count
as actual failures.

The problem is that parent_todo is not considered when setting the OK's in
the details in subtests. That means if a subtest ok fails, and the parent
is in todo, the 'ok' and 'actual_ok' are both set to false.

The fix is easy, just change line 845 to look at parent_TODO. However this
breaks Test::ClassMoose because it uses details to verify its results.
https://metacpan.org/source/DROLSKY/Test-Class-Moose-0.67/t/basic.t#L55

I spoke with autarch about this test since he is the current maintainer.
He agrees with me that 'ok' should be true in that test, and that this is
probably a bug.

I have not yet figured out what else this may break on cpan, I will try to
do that tomorrow.

*What I am looking for from cpan-workers:*

1) Do you think this is actually a bug, or does it work as you think it
should?
2) Even if we all agree it is a bug, can we fix it, or do we need to
preserve it to avoid breaking things?

-Chad



Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 4 | next ›
Discussion Overview
groupcpan-workers @
categoriesperl
postedDec 22, '15 at 12:04a
activeDec 22, '15 at 5:07a
posts4
users2
websitecpan.org

2 users in discussion

Chad Granum: 3 posts Karen Etheridge: 1 post

People

Translate

site design / logo © 2021 Grokbase