Paul Fenwick (p...@perltraining.com.au)

G'day Melb.pm and p5p,

Here's the long overdue report on the great Melbourne Perl Mongers bug triage.

We managed to properly triage an estimated 68 tickets at Melbourne Perl
Mongers (many thanks to Myf White for calculating this), along with about 14
which would have been completely triaged if I had all the meta-tickets
properly set up.  That's about 82 tickets total, which is pretty darn
awesome for a single night and not the world's best wireless access[1].  A
huge thank-you to all the Melbourne.pm volunteers!

If you want to run your own triage event, then here's a few recommendations:

* Work in pairs or small groups.

* Make sure meta-tickets are well-known beforehand.
On http://rt.perl.org/rt3/ these currently are:

* RT 69710 - Perl 5.12.0 - Showstoppers
* RT 70369 - Perl 5.12.0 - Expert assessment required
* RT 70421 - Perl 5.12.0 - Non-critical
 * RT 70371 - Perl 5.005  - End of life
 * RT 70373 - Perl 5.6.x  - End of life

* Have someone on-hand to give people privileges.  Not everyone will have
  them sorted out beforehand.

* It's nice to have 5.10.1 floating around. A lot of old (5.6.x)
  bugs can just be closed if it's fixed in the most recent
  release.

* Break list of bugs into chunks and hand to each group.  This avoids
  double-triage of bugs.

* Go through an example or two as a group on a projector first, if you
  can.

All the best,

Paul

[1] Federation square wifi allows outgoing http, but nothing else.
    No easy hopping on IRC.

G'day p5p,

The other day Jesse asked me if I'd be willing to sort out some 1,500
outstanding bugs at http://rt.perl.org/rt3/Public . In particular, I've
been asked to look at them all to see if there's any show-stoppers for 5.12.

I've accepted this task, with the intention to personally inspect as few of
these bugs as possible.  Instead, I hope to make people feel loved if they
triage the bugs for me, most likely by cajoling Perl Mongers groups and my
existing volunteer base.  I'm hoping to parcel up bugs into chunks of about
20 or so for individual volunteers to examine.

If we're optimising to locate 5.12 showstoppers, then I believe we generally
want to examine tickets with the following guidelines:

  * Tickets that have a (fatal|High) priority first.  Let's face it, a bug
    submitted with "low" priority is unlikely to be a showstopper.

  * Recent tickets over old tickets.  A recent ticket is more likely to
    represent a current bug.  Eight year old tickets are much less likely
    to still be current, or have such limited impact that they're not likely
    to be showstoppers.

With this in mind, while it's *nice* to triage all 1,500 bugs, we're likely
to get most of our hits on our primary goal (5.12 showstoppers) relatively
quickly.

Of course, if we're examining tickets for 5.12 showstoppers, it would be
lovely to do a bit more clean-up while we're at it.  Fixed bugs get closed.
 Bugs with patches get those patches looked at.  Eight year old bugs on Perl
5.004 on some obscure platform that's already end-of-lifed get closed (please!).

Volunteers who have RT access can make tweaks directly, but even if
*everyone* has RT access, we still need a way to record bugs as having been
triaged, so we don't duplicate effort.  And some of our users won't have RT
access[1], although I believe that *anyone* can add to RT via the e-mail
interface.

So, the plan is to have a template.  It records the template version (useful
for automated tools), and any relevant information, such as if it's a
showstopper (yes/no/maybe), if the bug should be closed, if it can be
reproduced, and so on.

My questions for you, dear reader:

* Is there something more suitable than a template?  Keep in mind that I'm
  going to be dealing with a wide variety of volunteers.  Simplicity is key.

* What constitutes a "showstopper"?  I imagine some things will fall into a
  "maybe" category, which indicates it requires further review.

* What information should be on the template?  I expect practically all
  fields will be optional, but should include:

* 5.12 showstopper? (Yes/no/maybe)
* Can/Can't be reproduced (version, platform)
* Hey look at this patch!
* Needs attention (and reason why).

  For people without RT access:
* Should be closed? (Fixed/not a bug/not supported/etc)
* Merge with...
* Change status to...
* Adjust priority/etc/other fields.

  Templates will, of course, also support free-form comments.

* Do you have any other great ideas that I should be thinking about?

I'm planning to kick off the first bug triage this Wednesday evening
(GMT+11) at Melbourne Perl Mongers.

Suggestions, ideas, comments, criticisms and beer all welcome.

Cheerio,

Paul

[1] If I can hand out RT access en masse and with little work, let me know!
But for the moment, I'm assuming that's non-trivial.

G'day Dave / All,

Warning: After a week of OSCON, and a large sleep debt, I'm practically out
of mana.  I apologise for any lack of clarity that may result.

Dave Mitchell wrote:

> Paul, I just pulled in autodie 2.06_01 from CPAN into blead and maint.
> AFAIKT, 2.06_01 is a sync of blead plus the addition of the eval_error
> method, so I'm assuming this is good for 5.10.1.

Ah, the 2.06_01 was pushed to the CPAN from OSCON to make sure my slides
worked on my Friday talk.  I marked it as a dev release because I hadn't put
any work into design or testing of eval_error(), so I best do that now.

Some modules (eg, Text::Balanced) may set $@ on failure, but not actually
throw an exception.  If one tries to apply autodie (with hints) to one of
these modules, the actual error message itself becomes impossible to access.
By the time autodie is formatting error messages, the old $@ is gone,
because autodie has thrown its own exception.

To accommodate these modules, I added eval_error() to the autodie::exception
class.  It contains the contents of $@ at the time autodie threw its own
exception, which means that we can use autodie with modules like
Text::Balanced and still get good looking and informative errors.

The only thing I really want to check is:

  * Is ->eval_error() a sensible method name for this.  I picked it
    as the English name for $@ is $EVAL_ERROR.

I also need to write tests.  I'm exhausted at the moment, so I'd cheerfully
accept patches, but otherwise they'll probably get written somewhere on my
trip to YAPC::EU.

Cheerio,

Paul

Mark Mielke wrote:

> How does it trip them up? I deal with newbies all the time and in two
> decades, I only remember having to explain ' once due to it "tripping
> somebody up".

Beyond hello world, the first exercise our students write involves printing
their name and favourite colour.  At this stage they know basic scalars and
string interpolation, but look what happens:

 my $name   = "Paul";
my $colour = "Black";

say "$name's favourite colour is $colour";

  Name "name::s" used only once: possible typo at - line 7.

One's first experience with a new language should not be corrupted output
and a hard to understand warning.

Personally, I've had ' cause me many more headaches (always inside strings)
than I've found legitimate uses (always outside of strings, and usually in
Klingon[1]).

Cheerio,

Paul

[1] And even then Perl doesn't have proper Klingon support.  A trailing
apostrophe causes all sorts of distress.

All the time, both for myself personally, and for newbies, and working with
newbies is my job.

I agree with Yves that its usage in strings at least should be considered
for deprecation.  Despite me having code written in Klingon, I'd be happy to
see it gone entirely.

Cheerio,

Paul

--------------030003040503090501030405
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

G'day p5p,

Craig A. Berry discovered that autodie's tests which attempt to detect
flock() support can cause bogus test failures on older VMS systems:

https://rt.cpan.org/Ticket/Display.html?id=47812

Attached is a trivial patch from Craig that fixes this issue.  It's already
been applied to my upstream repository.

Dave - I have no idea if this would be an issue for people building Perl
5.10.1 on old VMS systems.  You may wish to consider it for inclusion into
maint-5.10 just in case.

All the very best,

Paul

G'day Stevan / Moosers,

Stevan Little wrote:

> You cannot attached coercions to parameterized types directly. Try this ..
>
> class_type 'DateTime';
>
> # alias the Maybe[DateTime] type ...
> subtype 'MightBeADateTimeObjectOrMightNot' as 'Maybe[DateTime]';

Thank-you very much!  That works perfectly.  I can now suck down my
guitar-hero stats into a Moose object.  ;)

Cheerio,

Paul

G'day Moosers,

I'm writing a piece of code where I'm making use of the Maybe[DateTime]
construct.  An event may not have occurred (in which case we want undef), or
it may have (in which case we want a DateTime object).  I'm sucking in a
hunk of XML which represents the dates, and I have code that can convert a
hunk of XML (represented in a hash structure) into a DateTime object or undef:

    use Moose::Util::TypeConstraints;
    use DateTime;
    use DateTime::Format::Natural;

    class_type 'DateTime';

    coerce 'Maybe[DateTime]' => from 'HashRef' => via {
        warn "\n\n\n*** COERCING DATETIME ***\n\n\n";
        if ($_->{nil} eq "true") { return undef; }
return DateTime::Format::Natural->new->parse_datetime($_->{content})
    };

When trying to run this code, Moose is sad that it doesn't know about the
Maybe[DateTime] type:

    Cannot find type 'Maybe[DateTime]', perhaps you forgot to load it at
    /usr/lib/perl5/site_perl/5.10/Moose/Util/TypeConstraints.pm line 506

If I simply try to coerce DateTime, then I can't legitimately return undef,
because that fails the type constraint.  If I declare `class_type
'Maybe[DateTime]'` then the coerce code runs, but Moose still complains that
the type constraint fails:

    Attribute (last_blog_post_at) does not pass the type constraint because:
    Validation failed for 'Maybe[DateTime]' failed with value undef (not isa
    Maybe[DateTime]) at
    /usr/lib/perl5/site_perl/5.10/Moose/Meta/Attribute.pm line 753

FWIW, the attribute in question looks like this:

   has last_blog_post_at => ( isa => 'Maybe[DateTime]', coerce => 1 );

I'm using Moose 0.87.

Any advice would be appreciated.

Many thanks,

Paul

Many thanks to Craig Berry for tracking this down.
---
 lib/autodie/t/hints.t |   18 ++++++++++++++----
1 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/lib/autodie/t/hints.t b/lib/autodie/t/hints.t
index c21fde5..b508fee 100755
--- a/lib/autodie/t/hints.t
+++ b/lib/autodie/t/hints.t
@@ -15,6 +15,16 @@ use constant NO_SUCH_FILE2 => "this_file_had_better_not_exist_xyzzy";

 use constant PERL510  => ( $] >= 5.0100 );
use constant PERL5101 => ( $] >= 5.0101 );
+use constant PERL5102 => ( $] >= 5.0102 );
+
+# File::Copy states that all subroutines return '0' on failure.
+# However both Windows and VMS may return other false values
+# (notably empty-string) on failure.  This constant indicates
+# whether we should skip some tests because the return values
+# from File::Copy may not be what's in the documentation.
+
+use constant WEIRDO_FILE_COPY =>
+    ( ! PERL5102 and ( $^O eq "MSWin32" or $^O eq "VMS" ));

use Hints_test qw(
     fail_on_empty fail_on_false fail_on_undef
@@ -63,8 +73,8 @@ isa_ok($@, "autodie::exception");
is($@->function, "File::Copy::copy", "Function should be original name");

SKIP: {
-    skip("File::Copy is weird on Win32 before 5.10.1", 1)
-        if ( ! PERL5101 and $^O eq "MSWin32" );
+    skip("File::Copy is weird on Win32/VMS before 5.10.1", 1)
+        if WEIRDO_FILE_COPY;

     is($@->return, 0, "File::Copy returns zero on failure");
}
@@ -85,8 +95,8 @@ isa_ok($@, "autodie::exception");
is($@->function, "File::Copy::copy", "Function should be original name");

SKIP: {
-    skip("File::Copy is weird on Win32 before 5.10.1", 1)
-        if ( ! PERL5101 and $^O eq "MSWin32" );
+    skip("File::Copy is weird on Win32/VMS before 5.10.1", 1)
+        if WEIRDO_FILE_COPY;

     is_deeply($@->return, [0], "File::Copy returns zero on failure");
}

G'day Craig/Dave/p5p,

Attached is a patch to blead which skips File::Copy return
tests on 5.10.0 and 5.10.1 when running under VMS and Windows,
as previously discussed.

This doesn't come with a new upstream release of autodie, but
the next autodie release on the CPAN will include these changes.

Many thanks for Craig for tracking this down, and providing code
upon which the patch was based.

All the best,

    Paul

G'day Dave/Craig/p5p,

( This e-mail was written on a tram in Melbourne, without Internet access. )

Dave Mitchell wrote:

>> Thanks, Paul. That would be ok with me for 5.10.1, and I think would
>> just mean adding a clause to the Windows skippage that's already there.
>
> For 5.10.1, that sounds like the lowest risk option.

Done on my local autodie repo, with pushes to github and patches to Perl
soon.  I can push a new release of autodie to the CPAN if that's likely to
make VMS folks happy.

However, it should be noted that the skip-list explicitly tests for 5.10.0
and earlier.  My assumption is that this will be fixed in 5.10.1 because the
fix is *so* simple, and it's already in blead.  From 079cb8cc:

--- a/lib/File/Copy.pm
+++ b/lib/File/Copy.pm
@@ -213,7 +213,7 @@ sub copy {
             }
         }

-        return syscopy($from, $copy_to);
+        return syscopy($from, $copy_to) || 0;
     }

There's no need to go mucking around in XS land, since we can easily convert
merely false values to zeros.  This fixes it for both Windows, VMS, and any
other systems that have syscopy functions that don't adhere to the docs.  It
also comes with a bundle of tests to make sure those four extra characters
do what they're supposed to.

If for any reason we don't go with the fix which is currently in blead, then
I think we should consider fixing the docs so that rather than File::Copy
saying it returns zero, it instead says that it returns false.

Cheerio,

Paul

G'day Craig / All,

Craig A. Berry wrote:

> So likely we have an incompatible exit value on File::Copy, but I don't
> know offhand why autodie cares about that. Needs looking into.

It doesn't *really* care, those tests are verifying that File::Copy returns
what we expect, so that if any of the later tests fail, we know that
something is wrong.

However the documentation for File::Copy states for "RETURN":

       All functions return 1 on success, 0 on failure.

So it *should* be returning 0 if we're to be following what the docs say.

I can patch autodie to skip this test on VMS, or we can patch File::Copy to
return 0 rather than "" on failure.  I believe that cherry-picking
079cb8cc5abf40c0b016f9f878493b4d192d85d3 into maint-5.10 should do this.

For the CPAN version of autodie, I'll get it to always skip this test on
releases before 5.10.1.

Cheerio,

Paul

I think that translates to: "Paul forgot to do a git pull, but I figured it
out anyway."

Many thanks!

Paul

---
 Porting/Maintainers.pl |    6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/Porting/Maintainers.pl b/Porting/Maintainers.pl
index 31b0e86..3ff68f2 100755
--- a/Porting/Maintainers.pl
+++ b/Porting/Maintainers.pl
@@ -208,6 +208,12 @@ package Maintainers;
  'DISTRIBUTION' => 'PJF/autodie-2.04.tar.gz',
  'FILES'  => q[lib/Fatal.pm lib/autodie.pm lib/autodie],
  'EXCLUDED' => [ qr{^inc/Module/},
+
+                             # All these tests depend upon external
+                             # modules that don't exist when we're
+                             # building the core.  Hence, they can
+                             # never run, and should not be merged.
+
         qw(
     t/boilerplate.t
     t/critic.t

Hear hear!

Rafael, thank-you for guiding me when I was a complete newbie on p5p.
Thank-you for questioning my patches which sucked, and accepting my patches
that didn't.  Thank-you for saying that autodie sounded like a good idea for
core[1], which inspired me to continue working on it when I really didn't
want to.  Thank-you for putting up with me questioning smart-match
endlessly, even when I didn't participate in the original discussions.
Thank-you for changing smart-match based upon those questions.

Thank-you in particular for 5.10.0, which inspired me to write a very
popular conference talk, and a very popular pragma, and fulfil my lifetime
goal of claiming a Star Trek uniform as a tax deduction.

In particular, thank-you for being one of the nicest, friendliest, helpful,
and most knowledgeable people I know.

Yours most sincerely,

Paul

[1] Even though autodie is built on an accidental feature, built on an
enigma, built on a lie, built on a subspace rift near Cardiff.

Thanks to nothingmuch for assistance in the explanations.
---
 lib/Moose/Manual/BestPractices.pod |    7 +++++--
1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/lib/Moose/Manual/BestPractices.pod b/lib/Moose/Manual/BestPractices.pod
index 30c5673..9349718 100644
--- a/lib/Moose/Manual/BestPractices.pod
+++ b/lib/Moose/Manual/BestPractices.pod
@@ -32,8 +32,11 @@ Moose sugar and making your class immutable.

   1;

-The C<no Moose> bit is simply good code hygiene, and making classes
-immutable speeds up a lot of things, most notably object construction.
+The C<no Moose> bit simply good code hygiene, as it removes all the
+Moose keywords that are no longer needed once your class has been
+built.  C<make_immutable> relinquishes your right to make further
+changes to your class, and allows Moose to speed up a lot of things,
+most notably object construction.

=head2 Never override C<new>

G'day Dave,

Dave Mitchell wrote:

> Okay I think you've convinced me. So basically you'll release a 2.06 which
> is 2.05 + bug documentation; 2.06 goes in 5.10.1; then the bug fix goes in
> 2.07?

Sounds perfect.  I should have a documentation and tests patch to you within
24 hours.

Thank-you, and all the best,

Paul

- --
Paul Fenwick <pjf@perltraining.com.au> | http://perltraining.com.au/
Director of Training                   | Ph:  +61 3 9354 6001
Perl Training Australia                | Fax: +61 3 9354 2681

G'day Aristotle,

Aristotle Pagaltzis wrote:

> Your defensiveness is not becoming.

My apologies, you are quite right.  The proposal of removing autodie had
taken me somewhat by surprise.  Clearly my arei'mnu[1] is poor.  ;)

> Next on the agenda: an actual discussion about the needs of
> autodie and similar modules.

Agreed.  I'll save this for a separate message, not least because despite
being a heavy user of such an API, I haven't given much thought into what it
would actually look like.  At the very least, we need a way of saying "at
the end of this lexical scope (at compile time), run this".

All the very best,

Paul

[1] Vulcan for "emotional control".

G'day Nicholas,

Nicholas Clark wrote:

> If I understand you correctly, the problem isn't that autodie needs to do
> more than %^H currently does. You don't just need something *lexical* to hang
> state in that can be introspected at compile time.
>
> You also need some mechanism to run a *dynamic* end-of-scope hook at compile
> time?

Yes, this is exactly what autodie (and anything else using
B::Hooks::EndOfScope) needs.

> It was not reported at any time in the 3 years between when it was added to
> blead and when blead shipped as 5.10
>
> This report of a possible bug is a bit late.

I agree, it's very late!  Until now, I don't think anyone noticed this was
actually a problem.  What worked in 5.8.x continued to work in 5.10.x.
String evals are, well, not always the right way of doing things.  As such,
the amount of code that used lexical triggers via %^H next to string evals
were very rare.

Luckily...

> This behaviour was never a documented part of the implementation.
> It IS NOT NOW.

The 'perlpragma.pod' file, which talks about %^H in depth, says one
shouldn't store anything but simple strings and integers in %^H, because
anything else will get stringified.  So the stringification *is* documented,
it's just the timing that isn't.

While I agree that using %^H in this way is an accidental feature, we can
fix the problem that has just been encountered by:

   1) Upon string eval, we shallow copy %^H.  We already do this.
   2) Immediately stringify the values in the copied %^H and store them.
      We currently appear to do this much later (just before run-time?)

With these changes, code relying upon ref-counts in %^H to trigger magic
will continue to work as they did in 5.8.x.  Code using %^H to store
string/integer hints will also continue to work.  Lexical pragmas (using
both the accidental and intentional features) can continue to access %^H
within string evals.  I can't think of any immediate downsides.

AFAIK inside string evals the version of %^H which is available contains the
stringified values anyway.  It's just that we currently stringify them
rather late.

Going forward, finding a better solution that doesn't depend upon accidental
features is going to be preferable.  I honestly don't know what that
solution should be.  It could involve turning %^H's behaviour into an
intentional feature.  It could involve providing a different hook entirely.
Luckily, most of the code using %^H for this purpose uses
B::Hooks::EndOfScope for the heavy lifting, so most uses can be updated by
changing a single module.

All the best,

Paul

G'day Dave / p5p,

Looks like I've opened an interesting discussion here, and I certainly
didn't expect it to move to "let's boot autodie from the core" territory.

I don't want to see autodie get pulled from the core, so let's for a moment
examine it as it currently stands: version 2.05.  We now have a known bug:
it can leak if used near string evals.

Firstly, it should be noted that this bug has existed since at least the
15th June 2008, so it's almost a year old.  During that time, none of
autodie's tests (of which there are more than 400) detected it.  Nobody
submitted a bug report.  Nobody came to me complaining of autodie leaking
scope.  Nobody's reported seeing this in the wild.   Instead, the bug came
to me because rafl had discovered it in other code, and advised me that
autodie may be affected.

So, the bug is rare, but that doesn't mean it isn't serious.  It is.
However it also has a workaround in its current form.  In fact, it has two
workarounds:

1) Don't use string evals when autodie is in scope.
2) Use an explicit "no autodie" to plug the leak.

A known bug with workarounds is much better than an unknown bug.  However
one can still argue that it's a big enough defect to pull autodie from core.
But if we pull autodie from core, then what does that leave us with?

It leaves us with Fatal.  Fatal lacks the leaking bug, because by its very
nature it leaks everywhere.  It has package-wide scope.  There's no way to
restrict it to a single block, a single subroutine, or a single file.  Turn
it on, and it's on for your whole package.

And what does Fatal give you?  It gives you error messages that are famous
for their ugliness:

     $ perl -MFatal=open -e'open(my $fh, "<", "xyzzy");'
     Can't open(GLOB(0x1243928), <, xyzzy): No such file or directory at
    (eval 3) line 4
        main::__ANON__('GLOB(0x1243928)', '<', 'xyzzy') called at -e line 1

Oh yes, they're strings, too!  No exception objects here, if you want to
know what went wrong, you have to resort to regexpes and guessing.

What else does Fatal give us?  Oh yeah, I remember... bugs!  A big raft of
them.  Almost a dozen[1] of Perl's built-ins can legitimately return a false
value when successful, but Fatal can't handle that.  But it doesn't know
that, so it lets you wrap the built-in, but figuring out why that's not
working as you'd expect is left as an exercise for the developer.

In fact, for *everything* that Fatal does, autodie does it better.  I can
tear out the lexical handler for autodie, force users to install it with
package-wide scope, and it's still going to be better than Fatal in every way.

Throwing out autodie due to one bug that has never been seen in the wild is
like throwing out the baby with the bathwater.

I'm happy to document the bug and its workarounds, but I would ask you to
consider whether dropping autodie from 5.10.1 would actually improve Perl as
a whole.

Yours sincerely,

Paul

[1] fork, recv, send, open, fileno, read, readlink, sysread, syswrite,
    sysseek and umask can all return false (zero) values on some successes.

- --
Paul Fenwick <pjf@perltraining.com.au> | http://perltraining.com.au/
Director of Training                   | Ph:  +61 3 9354 6001
Perl Training Australia                | Fax: +61 3 9354 2681