FAQ
While trying to add a test for a regexp that causes a segfault,
I added a line to t/op/re_tests, with a 'B' in the third column,
expecting the test to be skipped.

Running ./perl -MTestInit t/op/regexp.t ended in a segfault.

Looking at the code in regexp.t, it seems that regardless of the
third column, every line with a pattern and a subject will cause
the pattern to be matched against the subject. "Skipping" just means
"report ok, regardless of the actual result".

Is this intentional? If not, I'll fix it to really skip matches.
If yes, what's the appropriate way to write a TODO test for a
pattern that causes a segfault?



Abigail

Search Discussions

  • Moritz Lenz at Oct 9, 2008 at 6:24 am

    Abigail wrote:
    While trying to add a test for a regexp that causes a segfault,
    I added a line to t/op/re_tests, with a 'B' in the third column,
    expecting the test to be skipped.

    Running ./perl -MTestInit t/op/regexp.t ended in a segfault.

    Looking at the code in regexp.t, it seems that regardless of the
    third column, every line with a pattern and a subject will cause
    the pattern to be matched against the subject. "Skipping" just means
    "report ok, regardless of the actual result".

    Is this intentional? If not, I'll fix it to really skip matches.
    If yes, what's the appropriate way to write a TODO test for a
    pattern that causes a segfault?
    IMHO it would be helpful to have both skip and TODO available. TODO
    tests have the huge advantage that they you notice immediately when they
    start passing, so I would only mark those tests as skip that really
    cause harm when run.

    Moritz
  • Michael G Schwern at Oct 9, 2008 at 3:48 pm

    Moritz Lenz wrote:
    Abigail wrote:
    While trying to add a test for a regexp that causes a segfault,
    I added a line to t/op/re_tests, with a 'B' in the third column,
    expecting the test to be skipped.

    Running ./perl -MTestInit t/op/regexp.t ended in a segfault.

    Looking at the code in regexp.t, it seems that regardless of the
    third column, every line with a pattern and a subject will cause
    the pattern to be matched against the subject. "Skipping" just means
    "report ok, regardless of the actual result".

    Is this intentional? If not, I'll fix it to really skip matches.
    If yes, what's the appropriate way to write a TODO test for a
    pattern that causes a segfault?
    IMHO it would be helpful to have both skip and TODO available. TODO
    tests have the huge advantage that they you notice immediately when they
    start passing, so I would only mark those tests as skip that really
    cause harm when run.
    test.pl has "todo_skip" which combines the "don't run the code" nature of a
    SKIP with the "its a bug" nature of TODO.

    But for segfaulting tests there is fresh_perl_is() and fresh_perl_like() which
    executes the code as a one-liner and checks the output. For example, in
    t/op/universal.t

    # This segfaulted in a blead.
    fresh_perl_is('package Foo; Foo->VERSION; print "ok"', 'ok');


    --
    39. Not allowed to ask for the day off due to religious purposes, on the
    basis that the world is going to end, more than once.
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
    http://skippyslist.com/list/
  • Abigail at Oct 11, 2008 at 6:18 pm

    On Thu, Oct 09, 2008 at 11:47:06AM -0400, Michael G Schwern wrote:
    Moritz Lenz wrote:
    Abigail wrote:
    While trying to add a test for a regexp that causes a segfault,
    I added a line to t/op/re_tests, with a 'B' in the third column,
    expecting the test to be skipped.

    Running ./perl -MTestInit t/op/regexp.t ended in a segfault.

    Looking at the code in regexp.t, it seems that regardless of the
    third column, every line with a pattern and a subject will cause
    the pattern to be matched against the subject. "Skipping" just means
    "report ok, regardless of the actual result".

    Is this intentional? If not, I'll fix it to really skip matches.
    If yes, what's the appropriate way to write a TODO test for a
    pattern that causes a segfault?
    IMHO it would be helpful to have both skip and TODO available. TODO
    tests have the huge advantage that they you notice immediately when they
    start passing, so I would only mark those tests as skip that really
    cause harm when run.
    test.pl has "todo_skip" which combines the "don't run the code" nature of a
    SKIP with the "its a bug" nature of TODO.

    But for segfaulting tests there is fresh_perl_is() and fresh_perl_like() which
    executes the code as a one-liner and checks the output. For example, in
    t/op/universal.t

    # This segfaulted in a blead.
    fresh_perl_is('package Foo; Foo->VERSION; print "ok"', 'ok');

    I tried this, with the fresh_perl_is in t/test.pl. However, it turns out
    that the program aborts during exit, so

    fresh_perl_is ('";" =~ /(?<a>(?|(?<b>;)))/ and print $&', ';')

    thinks everything went well - the program prints out ';' before aborting,
    and while _fresh_perl() takes notice of $?, it only uses this in a diagnostic
    message if the output was unexpected.

    Short of writing my own _fresh_perl that does check whether $? isn't set,
    is there something better I can use?


    Abigail
  • Michael G Schwern at Oct 12, 2008 at 1:34 am

    Abigail wrote:
    But for segfaulting tests there is fresh_perl_is() and fresh_perl_like() which
    executes the code as a one-liner and checks the output. For example, in
    t/op/universal.t

    # This segfaulted in a blead.
    fresh_perl_is('package Foo; Foo->VERSION; print "ok"', 'ok');
    I tried this, with the fresh_perl_is in t/test.pl. However, it turns out
    that the program aborts during exit, so

    fresh_perl_is ('";" =~ /(?<a>(?|(?<b>;)))/ and print $&', ';')

    thinks everything went well - the program prints out ';' before aborting,
    and while _fresh_perl() takes notice of $?, it only uses this in a diagnostic
    message if the output was unexpected.

    Short of writing my own _fresh_perl that does check whether $? isn't set,
    is there something better I can use?
    You could alter _fresh_perl() to pass a hash of information to the $resolve
    function so it can include both the text results and the exit status, then
    expose it as a public function so you can write your own custom condition.

    Or you can just check $? which should be pretty safe.

    fresh_perl_is('";" =~ /(?<a>(?|(?<b>;)))/ and print $&', ';');
    is( $?, 0 );


    --
    package Outer::Space; use Test::More tests => 9;

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupperl5-porters @
categoriesperl
postedOct 8, '08 at 10:16p
activeOct 12, '08 at 1:34a
posts5
users3
websiteperl.org

People

Translate

site design / logo © 2022 Grokbase