FAQ
25. Controlling test suite runs

Proposal:

The META spec should add flags to indicate that distribution's
tests may be shuffled or run in parallel. (SlavenRezic)

Comments:

* The prove utility provides the interesting options --shuffle and --jobs.
With --shuffle the test suite is not run in alphabetic but random order.
With --jobs multiple test scripts run in parallel. This could speed up
testing, especially on multi-processor machines.

* Unfortunately authors may write their test suites that neither one or the
other option may be used: authors may have some "initialization" and
"cleanup" script in the test suite (so shuffle cannot be used), or may
use a file or other resource in many test scripts (so parallel runs
cannot be used). So we cannot assume that --shuffle and --jobs may be
used by default.

* A sample definition could look like this (please think of better key
names):

test_run:
may_be_shuffled: yes
may_run_in_parallel: yes

Maybe even a control file to manage chunks/blocks that the module authors
already set up (Tux).

test_conf_file: t/t_conf

* I quite like this idea in principle, mostly because I've never seen a
better suggestion for how the Makefile.PL or Build.PL could make parallel
testing work. That said, it seems like it SHOULD be possible to make this
happen without having to go via metadata (which is for the configuration
to communicate to the CPAN client). Could the CPAN client really do
anything about this? --Adam K

Search Discussions

  • Ricardo Signes at Oct 9, 2009 at 1:18 pm
    * David Golden [2009-10-09T07:53:28]
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    I like the idea in general. I'd just set aside a name for the entry and ask
    the harness guys for a spec.

    --
    rjbs
  • Steffen Mueller at Oct 9, 2009 at 2:43 pm

    Ricardo Signes wrote:
    * David Golden [2009-10-09T07:53:28]
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    I like the idea in general. I'd just set aside a name for the entry and ask
    the harness guys for a spec.
    +1.

    Smart guy, that rjbs. Reading my thoughts and writing replies for me.

    Steffen
  • Graham Barr at Oct 9, 2009 at 2:24 pm

    On Oct 9, 2009, at 6:53 AM, David Golden wrote:
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    I do not think the META is the right place for this.

    It seems something that could be put in Build.PL or Makefile.PL as I do
    not see its use beyond after one of those have been run.

    -1

    Graham.
  • Barbie at Oct 31, 2009 at 6:30 pm

    On Fri, Oct 09, 2009 at 09:23:44AM -0500, Graham Barr wrote:
    On Oct 9, 2009, at 6:53 AM, David Golden wrote:
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    I do not think the META is the right place for this.
    Agreed.

    Its target seems to be aimed specifically at the Test Harness and thus
    has limited if any use elsewhere. Perhaps the TAP guys could suggest a
    Test-Spec :)

    Cheers,
    Barbie.
    --
    Birmingham Perl Mongers <http://birmingham.pm.org>
    Memoirs Of A Roadie <http://barbie.missbarbell.co.uk>
    CPAN Testers Blog <http://blog.cpantesters.org>
    YAPC Conference Surveys <http://yapc-surveys.org>
  • Tim Bunce at Oct 9, 2009 at 3:43 pm

    On Fri, Oct 09, 2009 at 07:53:28AM -0400, David Golden wrote:
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    Doesn't seem to belong in META to me (echoing Adam K's point).

    Extending prove etc to recognise a config file in the test directory
    would be a better direction to go in.

    Tim.
  • David Golden at Oct 9, 2009 at 8:49 pm

    On Fri, Oct 9, 2009 at 7:53 AM, David Golden wrote:
    25. Controlling test suite runs

    Proposal:

    The META spec should add flags to indicate that distribution's
    tests may be shuffled or run in parallel. (SlavenRezic)
    -1

    I don't think this belongs in META.

    Module::Build already has "use_tap_harness" and "tap_harness_args" so
    Build.PL can set this behavior.

    -- David
  • David E. Wheeler at Oct 9, 2009 at 10:13 pm

    On Oct 9, 2009, at 1:49 PM, David Golden wrote:

    -1

    I don't think this belongs in META.
    -1 Agreed.

    David

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcpan-workers @
categoriesperl
postedOct 9, '09 at 11:53a
activeOct 31, '09 at 6:30p
posts8
users7
websitecpan.org

People

Translate

site design / logo © 2018 Grokbase