Grokbase Groups Perl qa November 2006
FAQ
I just did a scan of minicpan to see how often test.pl is used and how often both test.pl and t/*.t are used. Results attached. Here's the summary.

12156 distributions scanned (sometimes multiple versions of a single dist)
2206 with only test.pl
130 with both

I've poked around in some of them that distribute both and have seen a few patterns...

1) Using it as a human-parsable benchmarking/utility script (DBI, Class::Tree)
2) Using it to run the t/*.t tests against the installed version (CGI::AppBuilder)
3) Leftover test stub from h2xs generation (Crypt::Passwd, Data::Match, WWW::Form).
4) A few tests are in test.pl and most are in t/*.t (Inline::Ruby)

#1 is the only legit use and its pretty rare.
#2 means the tests are normally run twice. Its also redundant now that we have prove.
#3 and #4 are likely simply oversights on the author's part.

So making "no_test_pl" a kwalitee test is a good idea.

Search Discussions

  • Steffen Mueller at Nov 1, 2006 at 4:59 pm
    Hi,

    Michael G Schwern schrieb:
    So making "no_test_pl" a kwalitee test is a good idea.
    So adding this to Module::CPANTS::Kwalitee::Files in the obvious place
    should do exactly that. Perhaps we should call it "old_testing_scheme"
    or so instead, but I'll go with your suggestion:

    {
    name=>'no_test_pl',
    error=>q{This distribution contains a file called 'test.pl' in
    its main distribution directory. This indicates that it uses the old
    style of creating test suites. Nowadays, tests belong into one or more
    *.t files in the t/ subdirectory.},
    remedy=>q{Refactor the monolithic test.pl script into one or
    more t/*.t test scripts.},
    code=>sub {
    my $d=shift;
    return 0 if $d->{file_test_pl};
    return 1;
    },
    },


    HTH,
    Steffen
  • Michael G Schwern at Nov 1, 2006 at 7:51 pm

    Steffen Mueller wrote:
    {
    name=>'no_test_pl',
    error=>q{This distribution contains a file called 'test.pl' in
    its main distribution directory. This indicates that it uses the old
    style of creating test suites. Nowadays, tests belong into one or more
    *.t files in the t/ subdirectory.},
    This doesn't really explain why test.pl is bad. It says "its old" but there's nothing wrong with old, if it works. The real problem is that test.pl is unparsed by "make test" and thus cannot be checked by automated testing. Try something like...

    This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not parsed. When "make test" is run by automated installers (for example, the CPAN shell) your tests will always appear to have passed no matter what the output of 'test.pl'. Please rename 'test.pl' to a '*.t' file in the t/ directory (for example, t/basic.t). All of t/*.t will be run, and parsed, by "make test".
  • Steffen Mueller at Nov 1, 2006 at 8:29 pm

    Michael G Schwern schrieb:
    This doesn't really explain why test.pl is bad.
    [...]

    Agreed!
    This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not parsed. When "make test" is run by automated installers (for example, the CPAN shell) your tests will always appear to have passed no matter what the output of 'test.pl'. Please rename 'test.pl' to a '*.t' file in the t/ directory (for example, t/basic.t). All of t/*.t will be run, and parsed, by "make test".
    Much better!

    Steffen
  • Demerphq at Nov 1, 2006 at 9:54 pm

    On 11/1/06, Michael G Schwern wrote:
    This distribution uses 'test.pl' for its tests. The output of 'test.pl' is not parsed. When
    "make test" is run by automated installers (for example, the CPAN
    shell) your tests will
    always appear to have passed no matter what the output of 'test.pl'.
    Is that true when test.pl dies? Maybe it should say that if you must
    use test.pl it should die on test failure.

    Yves


    --
    perl -Mre=debug -e "/just|another|perl|hacker/"
  • Michael G Schwern at Nov 1, 2006 at 11:51 pm

    demerphq wrote:
    On 11/1/06, Michael G Schwern wrote:
    This distribution uses 'test.pl' for its tests. The output of
    'test.pl' is not parsed. When
    "make test" is run by automated installers (for example, the CPAN
    shell) your tests will
    always appear to have passed no matter what the output of 'test.pl'.
    Is that true when test.pl dies? Maybe it should say that if you must
    use test.pl it should die on test failure.
    Technically that is correct. However, 99% of the time folks are using test.pl because that's what h2xs gave them and not because they prefer it over t/*.t. Its not worth mentioning workarounds in a succinct explanation, there's little reason reason to put tests in test.pl.
  • Steffen Mueller at Nov 1, 2006 at 5:04 pm
    Hi again,

    Michael G Schwern schrieb:
    So making "no_test_pl" a kwalitee test is a good idea.
    of course, adding a mention in the docs helps, too. So I should have
    made it a patch. Sorry. I didn't have the svn sources.

    Hence:

    =item * has_test_pl

    And if people decide that this should be optional, add an "is_extra =>
    1," into the hash ref of the last mail.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupqa @
categoriesperl
postedNov 1, '06 at 3:10p
activeNov 1, '06 at 11:51p
posts7
users3
websiteqa.perl.org

People

Translate

site design / logo © 2021 Grokbase