Grokbase › Groups › Perl › perl5-porters › March 2014
FAQ

On Wed Jan 15 22:56:35 2014, andreas.koenig.7os6vvqr@franz.ak.mind.de wrote:
git
---
commit b127e37e51c21b0a36755dcd19811be931a03d83
Author: Karl Williamson <public@khwilliamson.com>
Date: Mon May 13 07:35:35 2013 -0600

PATCH: [perl #108378] [perl #115800]

diagnostics
-----------
http://www.cpantesters.org/cpan/report/d2dd3370-d826-11e2-bc6f-
95dabcbf7481

Thanks to Slaven Rezić for finding this.
The test that fails in 5.19 is expecting a floating point number to be returned by JSON to be enclosed in double quotes, which it used to be, but after the blamed commit, no longer is. I have tracked this down to the following statement in JSON::PP (JSON:XS has a similar logic):

  return $value # as is
   if $flags & ( B::SVp_IOK | B::SVp_NOK ) and !( $flags & B::SVp_POK ); # SvTYPE is IV or NV?

The blamed commit changed things so that some scalars that are NVs no longer are also PVs. In the past this statement would not return and code further down would enclose it in double quotes, hence the old behavior, and the new.

But the logic here seems wrong to me. I don't understand the possible nuances and gotchas, but it seems to me that if something is an NV, it should be returned as an NV, without regard to if there is a stringified version available or not, which is what SVp_POK indicates.

My guess is that the tests for POK are to detect cases where something has a numeric value, but its string is different from that value and needs to be preserved, such as perhaps "00" (I don't know the internals here, so am just guessing.) But there is no restriction on somebody's deciding to make a numeric scalar POK. It can be done for good reasons, and it could be done for invalid ones. The modules shouldn't depend on this.

(In this case, prior to the blamed commit, a floating point number was made POK when its string form was needed, in order to avoid having to recalculate it each time. But this was found to create bugs where the decimal point varies in a program between, say, a comma and a dot because of locale differences. Making it POK cemented the decimal point to what it was at the time it was first stringified, not what it should be the next time a stringified version is needed. So it isn't made POK and is recalculated each time)

I think that both JSON::PP and JSON::XS should add logic to better detect whether something's string is different from its numeric value





--
Karl Williamson

---
via perlbug: queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=121013

Search Discussions

  • Marc Lehmann at Mar 6, 2014 at 5:06 am ⇧

    On Wed, Mar 05, 2014 at 11:09:47AM -0800, Karl Williamson via RT wrote:
    The test that fails in 5.19 is expecting a floating point number to be returned by JSON to be enclosed in double quotes, which it used to be, but after the blamed commit, no longer is. I have tracked this down to the following statement in JSON::PP (JSON:XS has a similar logic):
    Without seeing the test code nor the patch (both weren't easily
    available), I can only make some general comments, however, there is a
    good chance that the test itself is buggy.

    Perl scalars are (for the most part) typeless - scalars are not NVs or PVs,
    not even internally.

    JSON::XS (and JSON::PP, which copied the logic) attempts the impossible
    "and guesses" the type - the rules for that are described in the section
    "MAPPING" in the JSON::XS documentation.

    The short version is that the last use/modification counts, and the string
    form is preferred when in doubt, so if you printed your NV, or stringified
    it, it will serialise as a string.
    But the logic here seems wrong to me. I don't understand the possible nuances and gotchas, but it seems to me that if something is an NV, it should be returned as an NV, without regard to if there is a stringified version available or not, which is what SVp_POK indicates.
    In your case, this is true, in other cases, this isn't. If you want to
    make sure that your number is serialised as number, then you need to +=0
    it for example.

    The the treatment of "types" varies considerably within perl versions, and
    the test as-is is the one that works reliably for perl versions since 5.8,
    and errs on the safe side.
    My guess is that the tests for POK are to detect cases where something
    has a numeric value, but its string is different from that value and
    needs to be preserved, such as perhaps "00" (I don't know the internals
    here, so am just guessing.) But there is no restriction on somebody's
    deciding to make a numeric scalar POK. It can be done for good reasons,
    and it could be done for invalid ones. The modules shouldn't depend on
    this.
    The test si simply there because in JSON, the number 5 is different than
    the string "5", while in Perl, there is no semantic difference between
    them.
    (In this case, prior to the blamed commit, a floating point number was
    made POK when its string form was needed, in order to avoid having to
    recalculate it each time. But this was found to create bugs where
    the decimal point varies in a program between, say, a comma and a dot
    because of locale differences.
    The locale changes recently introduced in 5.19 cause a great deal of
    breakage (probably again a case where the new p5ps don't care for
    backwards compatibility). I hope these are sorted out before it gets
    released.
    I think that both JSON::PP and JSON::XS should add logic to better detect whether something's string is different from its numeric value
    The problem is that it's not clear what to do in that case. JSON::XS has to
    decide on either string or number.

    If the += 0 "trick" no longer works because perl stringifies the NV, then we
    are indeed in trouble, as that makes it impossible to even semi-reliably
    guess types.

    However, such a change would break an enourmous number of programs (most
    sql drivers use a similar form of guessing), and one would hope that p5p
    would not introduce such a big change (although in the past, they didn't
    seem to be reluctant to break things for no demonstrable gain).

    --
                     The choice of a Deliantra, the free code+content MORPG
           -----==- _GNU_ http://www.deliantra.net
           ----==-- _ generation
           ---==---(_)__ __ ____ __ Marc Lehmann
           --==---/ / _ \/ // /\ \/ / schmorp@schmorp.de
           -=====/_/_//_/\_,_/ /_/\_\
  • Karl Williamson at Mar 10, 2014 at 11:44 pm ⇧

    On 3/5/2014 10:06 PM, Marc Lehmann wrote:
    The short version is that the last use/modification counts, and the string
    form is preferred when in doubt, so if you printed your NV, or stringified
    it, it will serialise as a string.
    But the logic here seems wrong to me. I don't understand the possible nuances and gotchas, but it seems to me that if something is an NV, it should be returned as an NV, without regard to if there is a stringified version available or not, which is what SVp_POK indicates.
    In your case, this is true, in other cases, this isn't. If you want to
    make sure that your number is serialised as number, then you need to +=0
    it for example.

    The the treatment of "types" varies considerably within perl versions, and
    the test as-is is the one that works reliably for perl versions since 5.8,
    and errs on the safe side.
    What if the code in JOSN::PP instead did something like this:

        if ($flags & ( B::SVp_IOK | B::SVp_NOK ) { # SvTYPE is IV or NV?
             return $value if ! $flags & B::SVp_POK; # as is
      my $numeric = $value + 0;
      return $value if "$numeric" eq $value; # stringified is identical to
    numeric
        }

    Would that also work reliably? And insulate the code from us cretins on
    p5p?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupperl5-porters @
categoriesperl
postedMar 5, '14 at 7:09p
activeMar 10, '14 at 11:44p
posts3
users3
websiteperl.org

3 users in discussion

Karl Williamson: 1 post Marc Lehmann: 1 post Karl Williamson via RT: 1 post

Content

People

Support

Translate

site design / logo © 2017 Grokbase