Re: pjf: dual life modules (autodie)

G'day Dave,

Dave Mitchell wrote:

>  bleed, maint currently at 1.999.
> There was talk of a 2.00. Is this likely to appear any time soon,
> and if so, will it be suitable for 5.10.1; if not, is 1.999
>  suitable for 5.10.1?

There's autodie 1.999_01 on the CPAN.  It supports the hinting interface,
has tests, and is passing on all the platforms on which I expect it to pass.

The work to turn 1.999_01 into 2.000 is going through and making sure the
documentation matches the code, since a few things changed during
development.  There's also a few TODOs in the code itself to examine, most
of which have to do with minimising technical debt (eg, making sure all the
diagnostics messages are well-formed and documented).

The most likely time for these changes to occur is on my Australia -> Dubai
-> Saudi Arabia flights tomorrow.  I'm fond of conserving battery life on my
laptop, and coding tends to use a lot less power than anything else.  ;)

> Also, IIRC, you said that the new Fatal.pm was suitable for maint
> but there was an internal i/f that has changed; which you were
>  planning to fix. Has this been done?

autodie 1.999 doesn't keep compatibility with classical Fatal on the
internal interfaces.

For autodie 1.999_01 the fill_protos(), one_invocation() and
write_invocation() calls are backward compatible with classical Fatal.
There are test cases to ensure things stay this way.

The _make_fatal() subroutine (note the leading underscore) is not backwards
compatible, as it now expects to be called as a class method.  If you
really, really want, I can potentially fix that, although the code will look
even less pretty than it is now.

The public interface is completely backwards compatible with Fatal.

Cheerio,

Paul

G'day Dave,

Autodie 2.00 has finally been released, and is making its way through the
CPAN.  It has more backwardly compatible internal interfaces, the new
hinting interface, and a collection of more minor bugfixes.

I should have a patch ready for blead within the next 24 hours.  Full
changes are detailed below.

All the best,

Paul

2.00  Mon Jun 29 01:24:49 AUSEST 2009

        * FEATURE: autodie can now accept hints regarding how
          user and module subroutines should be handled.  See
          autodie::hints for more information.

        * INTERFACE: The calls to the internal subroutines
          one_invocation() and write_invocation() have changed.
          An additional argument (the user subroutine reference) is
          passed as the second-last argument.  This may break code
          that previously tried to call these subroutines directly.

        * BUGFIX: Calls to subroutines to File::Copy should now
          correctly throw exceptions when called in a list context.

        * BUGFIX: An internal error where autodie could potentially
          fail to correctly report a dying function's name has been
          fixed.

        * BUGFIX: autodie will no longer clobber package scalars when
          a format has the same name as an autodying function.
          (Thanks to Ben Morrow)

        * INTERFACE: The internal interfaces for fill_protos(),
          one_invocation(), write_invocation() are now once again
          backward compatible with legacy versions of Fatal.  It is
          still strongly recommended these interfaces are NOT called
          directly.  The _make_fatal() subroutine is not backwards
          compatible.

        * TEST: Added internal-backcompat.t to test backwards
          compatibility of internal interfaces.

        * DOCUMENTATION: Expanded documentation regarding how
          autodie changes calls to system(), and how this must be
          explicitly enabled.

        * BUILD: Upgraded to Module::Install 0.91

        * BUGFIX: A situation where certain compile-time diagnostics
          and errors from autodie would not be displayed has been
          fixed.

On Mon, 29 Jun 2009 12:21:01 +1000, Paul Fenwick
<pjf@perltraining.com.au> wrote:

> G'day Dave / p5p,
>
> Attached is a patch that brings blead up to autodie 2.0, providing the new
> hinting interface, and matching what's out there on the CPAN.

Thanks, patch successfully applied as 9b657a623c3928518095c706c37ba6315469a48e

I have two failures in bleadperl:
../lib/autodie/t/hints_pod_examples.t         (Wstat: 1280 Tests: 80 Failed: 5)
  Failed tests:  5, 14, 21, 37, 60
  Non-zero exit status: 5
../lib/autodie/t/hints.t                      (Wstat: 512 Tests: 21 Failed: 2)
  Failed tests:  2, 4
  Non-zero exit status: 2

Rafael Garcia-Suarez wrote:

>> Thanks, patch successfully applied as 9b657a623c3928518095c706c37ba6315469a48e
>
> I have two failures in bleadperl:

I can also reproduce these under cygwin/blead.  That'll teach me to post
patches before I've finished rebuilding Perl.

> ../lib/autodie/t/hints_pod_examples.t (Wstat: 1280 Tests: 80 Failed: 5)
>   Failed tests:  5, 14, 21, 37, 60
>   Non-zero exit status: 5

These look like they're the results of smart-match changes.  I'm going to
have to dig deeper to see what's going on here.  Investigating now.

> ../lib/autodie/t/hints.t (Wstat: 512 Tests: 21 Failed: 2)
>   Failed tests:  2, 4

Bah, these are red herrings.  The autodie::hints interface uses
Sub::Identify (or code taken from it, if not available) to determine the
original name of a subroutine.  Under 5.10.0 and 5.8.x, File::Copy's cp/mv
subroutines were aliases for copy and move.  Under blead, they're not.

All the best,

Paul

G'day Rafael / p5p,

For autodie, hints.t is fixed in my git repository.  Those tests were
failing due to a restructure of File::Copy, and the new tests handle these
correctly.  Of more interest, is:

Paul Fenwick wrote:
> Rafael Garcia-Suarez wrote:
>> ../lib/autodie/t/hints_pod_examples.t (Wstat: 1280 Tests: 80 Failed: 5)
>>   Failed tests:  5, 14, 21, 37, 60
>>   Non-zero exit status: 5
>
> These look like they're the results of smart-match changes. I'm going to
> have to dig deeper to see what's going on here. Investigating now.

These all seem to relate to how 5.10.1 smart-match handles undef.  In
particular:

 undef ~~ 0;  # True! (undef is coerced to 0)
 undef ~~ "";  # True! (undef is coerced to "")

Consequently, this also means the following prints "I've got a zero".

my $x = undef;

given ($x) {
  when (0) { say "I've got a zero!"        }  # printed
  default  { say "I've got something else" }
}

This means that for 5.10.1, any given/when block is going to need a separate
case for undef before any string/numeric tests if we want to avoid treating
it like 0/"".  I don't know about everyone else, but I *never* want to treat
undef as if it's defined.  If I did, I have now have the new // (defined-or)
operator to make that easy.  As such, it's a *big* surprise to have
smart-match go coercing undef so readily.

When comparing scalars, to me it makes the most sense to have undef only
smart-match against itself.  It certainly means I can have more succinct
given/when blocks, and less coercion warnings.

I'm also a little perplexed because:

 undef ~~ 0   # True
 undef ~~ [0]   # False!

Here no coercion is happening when comparing against an array reference!  My
understanding (after consulting the docs) is that the smart-match is
distributed over the array elements in the form `$_ ~~ $a`, resulting in the
undef on the RHS.  But then I'd expect `0 ~~ [undef]` to distribute to
`undef ~~ 0`, which then coerces undef to be a number, and hence be true,
but it's not...  So coercion only happens when comparing two scalars?  I'm
confused.

Unfortunately, this means that the new smart-matches don't fit into my head;
at least not these bits of it.  Using the principle of least surprise (to
pjf), I'd like to see:

 undef ~~ undef  # True
 0 ~~ undef  # False
 undef ~~ 0  # False (currently true)
[undef] ~~ undef # True (currently false!)
undef ~~ [undef] # True
 0 ~~ [undef]  # False
 [undef] ~~ 0  # False

I'm really sorry to be bringing up smart-match again.  Really, really sorry.
I don't want to start another long thread about how it should/shouldn't
work, but I fear I'm not going to be able to sleep soundly until I finally
grok it (or it groks me).

All the best,

Paul

I think no matter what choice will be made (undef is separate from 0/"";
undef is coerced to 0; undef is coerced to ""), you will encounter
situations where you wished the other choice was made. From experience,
sometimes I want 'undef' to be different from 0 and/or "". Often I want
'undef' and "" to be the same, but different from 0 (// isn't helping
there - a 'length' which doesn't warn on undef would help). Other times,
I want any false scalar value to be lumped together.

Therefore, I have no opinion on whether

    undef ~~ 0

and/or

    undef ~~ ""

should be true or false. Any choice will be surprising for some, and obvious
for others.

As long as there's some consistency.



Abigail

That's an inconsistency, and therefore a bug.

I think that for backward compatibility sake, and possibly for undef
commutativity sake, we should make undef ~~ 0 / "" not match. I'll do it
as soon as possible.

> Unfortunately, this means that the new smart-matches don't fit into my head;
> at least not these bits of it.  Using the principle of least surprise (to
> pjf), I'd like to see:
>
>        undef ~~ undef          # True
>        0 ~~ undef              # False
>        undef ~~ 0              # False (currently true)
>        [undef] ~~ undef        # True (currently false!)

There, I don't get you. How can an array (a non-empty array) match undef ?
The other cases I agree with.

>        undef ~~ [undef]        # True
>        0 ~~ [undef]            # False
>        [undef] ~~ 0            # False
>
> I'm really sorry to be bringing up smart-match again.  Really, really sorry.
>  I don't want to start another long thread about how it should/shouldn't
> work, but I fear I'm not going to be able to sleep soundly until I finally
> grok it (or it groks me).

Don't be sorry. You're one of the about 2 people who gave real-life,
valuable and argumented feedback before 5.10.1 is out.

2009/7/1 I:
> I think that for backward compatibility sake, and possibly for undef
> commutativity sake, we should make undef ~~ 0 / "" not match. I'll do it
> as soon as possible.

commit fb51372e8e462d7f3320e8a1b91a913f976aae12
Author: Rafael Garcia-Suarez <rgarciasuarez@gmail.com>
Date:   Wed Jul 1 12:28:04 2009 +0200

Make C<undef ~~ 0> and C<undef ~~ ""> not match (like in 5.10.0)

    This makes ~~ commutative with regard to undef, and fixes an
    inconsistency, since C<undef ~~ [0]> was not matching, and ~~
    should be distributive in this case.

Dave, can you consider this for inclusion in maint ?

No perldelta addition necessary, since we're actually restoring
behaviour that was in 5.10.0.

G'day Rafael,

Rafael Garcia-Suarez wrote:

>>        undef ~~ 0                      # True
>> undef ~~ [0] # False!
>
> That's an inconsistency, and therefore a bug.

Oh good.

> I think that for backward compatibility sake, and possibly for undef
> commutativity sake, we should make undef ~~ 0 / "" not match. I'll do it
> as soon as possible.

Thank-you very much!  I've just rebuild blead, and it looks like now I don't
need to change any of the pod_examples.t tests at all.  New autodie patches
coming soon.

>> [undef] ~~ undef # True (currently false!)

> There, I don't get you. How can an array (a non-empty array) match undef ?
> The other cases I agree with.

Sorry, this is my 5.10.0 brain not fully grokking the new smart-match.  I
look at `[undef] ~~ undef` and think, "Aha!  We must go searching for undef
inside that array."

Of course, ~~ is no longer commutative, but I keep having a really hard time
forgetting that.  Moritz said I should try a strong drink, or a gold brick
wrapped around a lemon, but unfortunately I'm all out of Arcturan Mega-gin.

> Don't be sorry. You're one of the about 2 people who gave real-life,
> valuable and argumented feedback before 5.10.1 is out.

This actually has me worried.  I've been picking at smart-match when it's
intersected with me doing something with autodie, and I've done this purely
out of laziness; I don't want to have to change how autodie works.  I
haven't poked around with the rest of smart-match, although I promised I would.

So, to help fix that, I've written (as I think I promised to long ago), a
test case to find any surprises.  It's based upon the smart-match Perl tip
at http://perltraining.com.au/tips/2008-04-18.html . I'll send it as a
patch in a separate e-mail.  Currently, it gives me two surprises:

my @foo = qw(x y z xyzzy ninja);
 ok( @foo ~~ 'ninja' );  # Fails

my $array_ref = [5, 10, 15];
ok ( $array_ref ~~ 10 ); # Fails

Looking down the table in blead/perlsyn.pod, I see the first one has a
string on the RHS, and the RHS determines what we do.  It looks like that
falls through to the very last rule (Any/Any), which means we're doing
string equality.  That doesn't seem very useful; because unless we've
specially crafted our string, it's never going to be true.

The same seems to apply for the second one.  I assume we're triggering
Any/Num, which is doing a straight numerical equality.  Again, that doesn't
seem very useful.

If that's what really what these are triggering, then can I respectfully
suggest they're changed to search the list instead (as they would do if they
were reversed)?

All the best,

Paul

--------------040205000207010802040601
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

G'day Rafael / p5p,

As per my previous post, here's a patch that adds a smartmatch_tip.t file to
the test suite, which are tests based upon our previously published Perl Tip at:

http://perltraining.com.au/tips/2008-04-18.html

I apologise the tests are terribly simple, I didn't actually check to see
what test.pl gives me in terms of functionality.

All the tests pass under 5.10.0, but there are two failures under blead.
They are:

my @foo = qw(x y z xyzzy ninja);
 ok( @foo ~~ 'ninja' );  # Fails, line 41

my $array_ref = [5, 10, 15];
ok ( $array_ref ~~ 10 ); # Fails, line 45

My previous mail discusses more about what I think is going on here, and
queries weather it's the right thing to do.

All the best,

Paul

* Rafael Garcia-Suarez <rgarciasuarez@gmail.com> [2009-07-01T06:31:35]
> Make C<undef ~~ 0> and C<undef ~~ ""> not match (like in 5.10.0)
>
> This makes ~~ commutative with regard to undef, and fixes an
> inconsistency, since C<undef ~~ [0]> was not matching, and ~~
>     should be distributive in this case.

Waaaaah?  I am so confused.

This is more special casing.  $x~~$str means $x eq $str.  undef eq ''.

I guess we'll just say: $x~~$str === defined $x and $x eq $str.  I find this
sort of bizarre, but maybe I'll get over it.

(and I am led to wonder what the // version of && looks like; I would need it
for: use autobox; $x = 0 §§ 0->ordinal)

I was wondering the same thing. The previous behavior made sense to me  
because it was consistent with `undef eq ''` and `undef == 0`.  
However, `[0] != undef`.

Best,

David

2009/7/1 Paul Fenwick <pjf@perltraining.com.au>:
> So, to help fix that, I've written (as I think I promised to long ago), a
> test case to find any surprises. =A0It's based upon the smart-match Perl =
tip
> at http://perltraining.com.au/tips/2008-04-18.html . =A0I'll send it as a
> patch in a separate e-mail. =A0Currently, it gives me two surprises:
>
> =A0 =A0 =A0 =A0my @foo =3D qw(x y z xyzzy ninja);
> =A0 =A0 =A0 =A0ok( @foo ~~ 'ninja' ); =A0 =A0 =A0 =A0 =A0# Fails
>
> =A0 =A0 =A0 =A0my $array_ref =3D [5, 10, 15];
> =A0 =A0 =A0 =A0ok ( $array_ref ~~ 10 ); =A0 =A0 =A0 =A0# Fails
>
> Looking down the table in blead/perlsyn.pod, I see the first one has a
> string on the RHS, and the RHS determines what we do. =A0It looks like th=
at
> falls through to the very last rule (Any/Any), which means we're doing
> string equality. =A0That doesn't seem very useful; because unless we've
> specially crafted our string, it's never going to be true.
>
> The same seems to apply for the second one. =A0I assume we're triggering
> Any/Num, which is doing a straight numerical equality. =A0Again, that doe=
sn't
> seem very useful.
>
> If that's what really what these are triggering, then can I respectfully
> suggest they're changed to search the list instead (as they would do if t=
hey
> were reversed)?

No, because if we have
    given ($foo) { when (undef) {...} }
we sort of expect the code in the when clause to be executed when $foo
is not defined.
That's precisely the same reason why in Perl 6 ~~ isn't commutative any
more either.

G'day Rafael / p5p,

Rafael Garcia-Suarez wrote:

>>        my @foo = qw(x y z xyzzy ninja);
>> ok( @foo ~~ 'ninja' ); # Fails
>>
>>        my $array_ref = [5, 10, 15];
>> ok ( $array_ref ~~ 10 ); # Fails

> No, because if we have
>     given ($foo) { when (undef) {...} }
> we sort of expect the code in the when clause to be executed when $foo
> is not defined.

I *think* I get it...  If `$array_ref ~~ 10` is allowed to search
@$array_ref for 10, then `$foo ~~ undef` could be searching @$foo for undef,
rather than checking if $foo is undefined.

This means that when we see:

@foo ~~ $x
%bar ~~ $x

they're both obvious mistakes, right, since they're practically never going
to succeed[1].  Is there scope for a warning or error to be emitted when we
encounter such?  I'm certain I won't be the only person who is going to
write `if (%hash ~~ $key)` or `if (@array ~~ $element)` to check for
containership, especially since this worked fine in 5.10.0, and has since
stuck in my head.  A compile-time warning/error would save me a huge amount
of grief, as well as being very valuable to anyone upgrading 5.10.0 code to
5.10.1.

I'm of mixed opinion if `$array_ref ~~ $x`  or `$hash_ref ~~ $x` should
generate a run-time warning.  One one hand, they're obviously incorrect, but
on the other hand I can see them being used as legitimate tests (eg, when
the LHS was passed in as a subroutine argument).

I guess I should update our Perl-tip and course notes, to make sure they're
still correct for when 5.10.1 hits the streets.

All the best,

Paul

* Paul Fenwick <pjf@perltraining.com.au> [2009-07-01T22:08:21]
> This means that when we see:
>
>  @foo ~~ $x
>  %bar ~~ $x
>
> they're both obvious mistakes, right, since they're practically never going
> to succeed[1].

That footnote appears to have been lost during transmission.

What do you suggest is in $x?  A simple number or integer?  In that case, they
may never succeed, but they are not obvious mistakes.  I suppose that this is
an probable mistake:

  if (@foo ~~ 5) { ... }

That is, the /literal/ 5 on the right hand side and an @-sigiled variable on
the left.  I guess we could look for that, if we thought it was worth it.

So many other cases will be in the form:

  given ($input) {
    when ($config) { ... }
    ..
  }

Where $input might be [...] and $config might be 8.  We don't to emit a warning
there.  It might just not match and a later clause might be relevant.

> A compile-time warning/error would save me a huge amount of grief, as well as
> being very valuable to anyone upgrading 5.10.0 code to 5.10.1.

I feel like this adds more code than value; how many people have much 5.10.0
code, without tests, that uses the @a~~5 or %h~~"foo" syntax -- with the
compile-time-determinable syntax?

Note my verb, though: this is only a feeling.  I have not performed science.

Frankly, every thread about ~~ makes me lean toward my initial position: it's
just too clever, even though it is clearly going to be wildly useful in
numerous cases.  I'm really divided on it.

It was going to be something about specially formatted strings/numbers that
matched stringified addresses.  I then thought better of it, and forgot to
remove the footnote marker.

> What do you suggest is in $x? A simple number or integer? In that case, they
> may never succeed, but they are not obvious mistakes. I suppose that this is
> an probable mistake:
>
>   if (@foo ~~ 5) { ... }

Crap.  For some reason I was only thinking of $x as being a simple scalar
(string/number/undef), and not a reference.  Your example *is* an obvious,
probable mistake.  My example *may* be a probable mistake, or may be just
fine if $x contains a reference, or the developer may not care that it fails
because $x contains a simple scalar.  As you've rightly mentioned, we can't
tell if that's an actual mistake or not without reading a human's mind.

I'm happy with `@foo ~~ 5` producing a warning, but I think most occurrences
of this particular mistake won't be lucky enough to have a literal on the
RHS.  I agree that we're probably adding more code than value if this is all
we can detect and warn against.

> Frankly, every thread about ~~ makes me lean toward my initial position: it's
> just too clever, even though it is clearly going to be wildly useful in
> numerous cases.  I'm really divided on it.

I'm in complete agreement here.  I loved the old (5.10.0) smart-match,
because it was communicative, and all the simple uses made sense.  I could
use it to check to see if a key exists, an element exists, a pattern
matches, or if two things looked the same.

It was also easy to explain to people new to Perl.  That's massively
important to me, since explaining Perl is my job.  A simple thing like
`@array ~~ $element` being the same as `$element ~~ @array` meaning "can I
find $element in @array?` is very valuable.  Add given/when, and it becomes
positively delightful.  I've seen old and new programmers alike dance for
joy when shown this simple construct.

However the new smart-match is just too clever for me!  I have to remember
ordering; if I get it wrong, the test means something completely different,
or means nothing at all (@foo ~~ 5)!  I find myself shying away from using
smart-match, because debugging (@foo ~~ $x) when I actually meant ($x ~~
@foo) is going to take much longer than debugging the alternatives, even
though smart-match is faster, and *should* be clearer.

Heck, even if I *can* remember it's supposed to be `$x ~~ @foo`, smart-match
now distributes itself over arrays.  If @foo contains a subroutine
reference, then suddenly I'm running code!  I don't want that, I really
really don't.  I just want to check to see if an element exists.
Smart-match should make simple things easy, Perl already makes hard things
possible.

Maybe my feelings will change with time.  Maybe my dev-brain is too stuck on
the old smart-match that it refuses to learn the new one.  Or maybe, just
maybe, smart-match really is trying to be too freakishly clever, and we're
going to be cursed by maintainers everywhere for our folly.

I don't know what to do about this.  For the moment, given the choice
between the old and new smart-matches for 5.10.1, I'd pick the old
not-so-smart-match in a heartbeat.

Paul

Paul Fenwick wrote:

A whole lot of stuff with his vocab hashing to the wrong entries.  Please
consider:

s/communicative/commutative/g;

Also consider me to be deeply ashamed.

* Paul Fenwick <pjf@perltraining.com.au> [2009-07-03T01:28:51]
> > Frankly, every thread about ~~ makes me lean toward my initial position:
> > it's just too clever, even though it is clearly going to be wildly useful
> > in numerous cases. I'm really divided on it.
>
> I'm in complete agreement here. I loved the old (5.10.0) smart-match,
> because it was communicative, and all the simple uses made sense. I could
> use it to check to see if a key exists, an element exists, a pattern
> matches, or if two things looked the same.

It's funny, because I find the new behavior *much* more straightforward and
easy to explain than the old.

I'd rather really have distributive smart matching (yes, run the code in my
array on the RHS), but I can imagine reasons to not.

I think in the end, this will happen:

  * I won't use smart match.
  * Someone (maybe me, if I buck up) will write 'no smartmatch' which will
    disable ~~ where the rhs is not an object overloading ~~
  * sugary ~~-overloading object generators will rule: $x ~~ any(5,6,7);
  * (hm, except that's just $x == any(5,6,7) with junctions)
  * well, anyway, ~~ will be useful with object rhs, and predictable
  * also, we'll finally get =~ overloading as often discussed
  * this wasn't much of a bullet list, was it?