FAQ
06. Data structures, not YAML

Proposal:

The META spec should be defined in terms of (Perl) data structures, and not
in terms of YAML. (Slaven Rezic)

Comments:

* This does not mean that I want to replace YAML by a Perl data dump (or
maybe yes, but see below). It is just about the specification document:
it should not talk about "META.yml", but the META spec, and should not
use the YAML data type names (mapping, sequence), but Perl data type
names (hash, array).

* The actual serialization proposals could go into an appendix of the spec,
or even in a new document. So whether it will be JSON, or YAML::Tiny, or
YAML 1.1 could be decided later.

* In my opinion I would vote for *two* serialization formats: normal YAML
as it is done currently (or even YAML 1.1), which would be used for
statistical analysis (here we can force people to install the latest and
greatest YAML parser), and a Perl data dump which is used by CPAN.pm etc.
for configure_requires or similar stuff (here the user has probably a
very bare perl installation, the minimum we can expect is the ability to
parse Perl). Something similar is done with distroprefs handling in
CPAN.pm: if .yml is available, fine, otherwise it's possible to fallback
to .dd, a perl data dump file.

* I am still fine that the examples in the spec are still expressed in
YAML, because I think that YAML is a quite nice human-readable format.

Search Discussions

  • Ricardo Signes at Oct 9, 2009 at 12:14 pm
    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)
    Agreed, but the spec should be very clear (perhaps, as said, in an appendix)
    how the data should be serialized and included in dists.

    --
    rjbs
  • Graham Barr at Oct 9, 2009 at 1:32 pm

    On Oct 9, 2009, at 7:14 AM, Ricardo Signes wrote:

    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures,
    and not
    in terms of YAML. (Slaven Rezic)
    Agreed, but the spec should be very clear (perhaps, as said, in an
    appendix)
    how the data should be serialized and included in dists.
    +1

    Graham.
  • Steffen Mueller at Oct 9, 2009 at 2:09 pm

    Ricardo Signes wrote:
    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)
    Agreed, but the spec should be very clear (perhaps, as said, in an appendix)
    how the data should be serialized and included in dists.
    I have no strong feelings about this except for what Ricardo wrote
    above: The serialization method needs to be prominently featured.

    Steffen
  • Jarkko Hietaniemi at Oct 9, 2009 at 2:11 pm
    I really don't think we should have Perl data structures in files
    (that means Perl code, right?), because that indicates doing an eval,
    and I don't want to eval any more random code off the 'net than
    necessary.

    On Fri, Oct 9, 2009 at 8:14 AM, Ricardo Signes
    wrote:
    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)
    Agreed, but the spec should be very clear (perhaps, as said, in an appendix)
    how the data should be serialized and included in dists.

    --
    rjbs


    --
    There is this special biologist word we use for 'stable'. It is
    'dead'. -- Jack Cohen
  • Ricardo Signes at Oct 9, 2009 at 2:58 pm
    * Jarkko Hietaniemi [2009-10-09T10:11:35]
    I really don't think we should have Perl data structures in files
    (that means Perl code, right?), because that indicates doing an eval,
    and I don't want to eval any more random code off the 'net than
    necessary.
    Right. We definitely want a proper serialization layer. (JSON!) I think if
    read as a proposal to describe the data serialized in terms of Perl structures,
    rather than the serialization system's forms, this is great.

    ("author is a string or arrayref" not "author is a character-vector or
    sequence")

    --
    rjbs
  • Graham Barr at Oct 9, 2009 at 5:52 pm

    On Oct 9, 2009, at 9:11 AM, Jarkko Hietaniemi wrote:

    I really don't think we should have Perl data structures in files
    (that means Perl code, right?), because that indicates doing an eval,
    and I don't want to eval any more random code off the 'net than
    necessary.
    I strongly agree that we should not be having perl code in the META
    file for
    security reasons. Although most people would use Safe to read it,
    there would be
    those that would not and could get caught out

    But the spec should also not be biased to a particular format, IMO. So
    describing
    what goes into the META data in terms of perl data types seems
    reasonable.

    The spec should contain a separate section which describes how the
    data is serialized

    Graham.
  • David E. Wheeler at Oct 9, 2009 at 10:06 pm

    On Oct 9, 2009, at 10:52 AM, Graham Barr wrote:

    I strongly agree that we should not be having perl code in the META
    file for
    security reasons. Although most people would use Safe to read it,
    there would be
    those that would not and could get caught out

    But the spec should also not be biased to a particular format, IMO.
    So describing
    what goes into the META data in terms of perl data types seems
    reasonable.

    The spec should contain a separate section which describes how the
    data is serialized
    +1

    David
  • Barbie at Oct 31, 2009 at 4:03 pm

    On Fri, Oct 09, 2009 at 03:05:47PM -0700, David E. Wheeler wrote:
    On Oct 9, 2009, at 10:52 AM, Graham Barr wrote:

    I strongly agree that we should not be having perl code in the META
    file for
    security reasons. Although most people would use Safe to read it,
    there would be
    those that would not and could get caught out

    But the spec should also not be biased to a particular format, IMO. So
    describing
    what goes into the META data in terms of perl data types seems
    reasonable.

    The spec should contain a separate section which describes how the
    data is serialized
    +1
    +1 from me too :)

    Cheers,
    Barbie.
    --
    Birmingham Perl Mongers <http://birmingham.pm.org>
    Memoirs Of A Roadie <http://barbie.missbarbell.co.uk>
    CPAN Testers Blog <http://blog.cpantesters.org>
    YAPC Conference Surveys <http://yapc-surveys.org>
  • Zefram at Oct 10, 2009 at 8:08 pm

    Jarkko Hietaniemi wrote:
    I really don't think we should have Perl data structures in files
    (that means Perl code, right?), because that indicates doing an eval,
    Not necessarily. Working in *a defined subset of* Perl syntax would mean
    that readers have both options. Evaling would probably be acceptable
    in situations where you're installing the module described by the META
    file, so this could be a convenient option in some small installations.
    Stats gatherers, who don't want to run code from the module, can use a
    safe parser as they do now.

    It so happens that I have a suitable subset of Perl syntax already
    defined. I needed to implement it a while ago for work purposes, and now
    it's on CPAN, as Data::Pond. It's similar in spirit to JSON, which of
    course gives the same kind of parsing choice for the JavaScript language.

    -zefram
  • Slaven Rezic at Nov 1, 2009 at 8:51 am

    Jarkko Hietaniemi wrote:
    I really don't think we should have Perl data structures in files
    (that means Perl code, right?), because that indicates doing an eval,
    and I don't want to eval any more random code off the 'net than
    necessary.
    Consumers of META.yml may be separated into two groups:
    - those who really don't want to eval perl code (i.e. search.cpan.org,
    cpants, cpan deps etc.)
    - those who don't care to eval perl code, because they do it anyway in
    the same process (i.e. when installing a CPAN distribution with
    CPAN/CPANPLUS)

    Therefore I think the current approach of dumping to a yaml file should
    be preserved for the first group, and dumping to a perl file added for
    the second group. This has also the benefit that there does not need to
    be any discussions whether to add YAML::Tiny, YAML, or JSON or whatever
    deserializer to the Perl core.

    Regards,
    Slaven
    On Fri, Oct 9, 2009 at 8:14 AM, Ricardo Signes
    wrote:
    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)
    Agreed, but the spec should be very clear (perhaps, as said, in an appendix)
    how the data should be serialized and included in dists.

    --
    rjbs
  • David Golden at Nov 1, 2009 at 2:02 pm

    On Sun, Nov 1, 2009 at 3:51 AM, Slaven Rezic wrote:
    - those who don't care to eval perl code, because they do it anyway in the
    same process (i.e. when installing a CPAN distribution with CPAN/CPANPLUS)
    The second group doesn't really want to do it either in all cases.
  • Slaven Rezic at Nov 1, 2009 at 3:51 pm

    David Golden wrote:
    On Sun, Nov 1, 2009 at 3:51 AM, Slaven Rezic wrote:
    - those who don't care to eval perl code, because they do it anyway in the
    same process (i.e. when installing a CPAN distribution with CPAN/CPANPLUS)
    The second group doesn't really want to do it either in all cases.
    Why not? A few CPU cycles later "perl Makefile.PL" or "perl Build.PL"
    happens.
  • David Golden at Nov 1, 2009 at 8:53 pm

    On Sun, Nov 1, 2009 at 10:51 AM, Slaven Rezic wrote:
    Why not? A few CPU cycles later "perl Makefile.PL" or "perl Build.PL"
    happens.
    Yes, but in a subprocess. And YAML/JSON can be validated statically
    much more easily than Perl.

    -- David
  • Slaven Rezic at Nov 1, 2009 at 9:12 pm

    David Golden wrote:
    On Sun, Nov 1, 2009 at 10:51 AM, Slaven Rezic wrote:
    Why not? A few CPU cycles later "perl Makefile.PL" or "perl Build.PL"
    happens.
    Yes, but in a subprocess.
    Where's the difference? There's no extra security CPAN.pm provided when
    executing Makefile.PL.
    And YAML/JSON can be validated statically
    much more easily than Perl.
    There's no difference in simplicity between

    Kwalify::validate($meta_schema, Safe->new->rdo($meta_dd))

    and

    Kwalify::validate($meta_schema, YAML::LoadFile($meta_yml))

    Regards,
    Slaven
  • David Golden at Oct 9, 2009 at 1:56 pm

    On Fri, Oct 9, 2009 at 7:44 AM, David Golden wrote:
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)

    Comments:

    * This does not mean that I want to replace YAML by a Perl data dump (or
    maybe yes, but see below). It is just about the specification document:
    it should not talk about "META.yml", but the META spec, and should not
    use the YAML data type names (mapping, sequence), but Perl data type
    names (hash, array). Agreed.
    * The actual serialization proposals could go into an appendix of the spec,
    or even in a new document. So whether it will be JSON, or YAML::Tiny, or
    YAML 1.1 could be decided later.
    I like the idea of different sections for "data", "serialization", and
    "usage" (the latter being how clients use it.)
    * In my opinion I would vote for *two* serialization formats: normal YAML
    as it is done currently (or even YAML 1.1), which would be used for
    statistical analysis (here we can force people to install the latest and
    greatest YAML parser), and a Perl data dump which is used by CPAN.pm etc.
    for configure_requires or similar stuff (here the user has probably a
    very bare perl installation, the minimum we can expect is the ability to
    parse Perl). Something similar is done with distroprefs handling in
    CPAN.pm: if .yml is available, fine, otherwise it's possible to fallback
    to .dd, a perl data dump file.
    I don't like having .dd as an option. We want to minimize evaluated code.
    * I am still fine that the examples in the spec are still expressed in
    YAML, because I think that YAML is a quite nice human-readable format.
    I'm be in favor of replacing all examples with Perl data structures,
    since they will map more clearly to how they will be generated and how
    they will exist after deserializing. We're talking about a spec for
    Perl programmers, so I think that's probably sufficiently
    "human-readable".

    -- David
  • Ricardo Signes at Oct 9, 2009 at 11:22 pm
    * David Golden [2009-10-09T07:44:43]
    06. Data structures, not YAML

    Proposal:

    The META spec should be defined in terms of (Perl) data structures, and not
    in terms of YAML. (Slaven Rezic)
    I have made this change in a branch, here:

    http://github.com/rjbs/cpan-meta-spec/tree/restructure

    The branch changes the meat of the spec to discuss things in Perl terms and
    moves the existing discussion of YAML format to the end of the document. Any
    change in meaning is entirely accidental.

    I hope that this change can be accepted immediately so that I may begin making
    tentative branches from it for each of the seemingly-likely suggested changes.

    While most of them should be possible as branches, this change affects the spec
    enough overall that I'd like to get it agreed upon as a branching point.

    --
    rjbs
  • David E. Wheeler at Oct 10, 2009 at 2:05 am

    On Oct 9, 2009, at 4:22 PM, Ricardo Signes wrote:

    I have made this change in a branch, here:

    http://github.com/rjbs/cpan-meta-spec/tree/restructure

    The branch changes the meat of the spec to discuss things in Perl
    terms and
    moves the existing discussion of YAML format to the end of the
    document. Any
    change in meaning is entirely accidental.
    Awesome. Now it looks just like a Module::Build script. I like!

    David

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcpan-workers @
categoriesperl
postedOct 9, '09 at 11:45a
activeNov 1, '09 at 9:12p
posts18
users9
websitecpan.org

People

Translate

site design / logo © 2018 Grokbase