FAQ
Hello.

I think I'm being an idiot (not for the first time), but given the following hiera paragraph:

directorymap:
www.blah.com:
- source: dodgy-uploaded-jpegs
target: www.blah.com/wp-content/uploads
- source: disturbing-home-videos
target: www.blah.com/wp-content/filth
www.nice-site.co.uk:
- source: kitten-pics
- target: www.nice-site.co.uk/site-content/user-data

What's the least-worst way of extracting that, such that puppet pseudo-code along the lines of

'ln -s ${site}/${source} ${site}{$target}' would work?

There is likely a Better Way To Do It...

--
John Hawkes-Reed
Systems Administrator. Future Publishing. x 2526

--
Future Publishing Limited (registered company number 2008885) is a wholly owned subsidiary of Future plc (registered company number 3757874), both of which are incorporated in England and Wales and share the same registered address at Beauford Court, 30 Monmouth Street, Bath BA1 2BW.

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error please reply to this email and then delete it. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Future.

The recipient should check this email and any attachments for the presence of viruses. Future accepts no liability for any damage caused by any virus transmitted by this email.

Future may regularly and randomly monitor outgoing and incoming emails and other telecommunications on its email and telecommunications systems. By replying to this email you give your consent to such monitoring.

*****
Save resources: think before you print.


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Jcbollinger at Sep 21, 2012 at 2:48 pm

    On Friday, September 21, 2012 6:04:37 AM UTC-5, John Hawkes-Reed wrote:
    Hello.

    I think I'm being an idiot (not for the first time), but given the
    following hiera paragraph:

    directorymap:
    www.blah.com:
    - source: dodgy-uploaded-jpegs
    target: www.blah.com/wp-content/uploads
    - source: disturbing-home-videos
    target: www.blah.com/wp-content/filth
    www.nice-site.co.uk:
    - source: kitten-pics
    - target: www.nice-site.co.uk/site-content/user-data

    What's the least-worst way of extracting that, such that puppet
    pseudo-code along the lines of

    'ln -s ${site}/${source} ${site}{$target}' would work?

    So, it looks like that would expand to commands such as

    ln -s www.nice-site.co.uk/kitten-pics www.nice-site.co.uk
    www.nice-site.co.uk/site-content/user-data

    I suspect the duplication of the site on the target side is not what you
    want, but it follows from your data. Nevertheless, I guess your real
    question is about how to make Puppet digest your nested data structure.

    (Note also that your YAML data structure under "nice-site" is inconsistent
    with its cousin under "blah"; it appears that the structure under "blah" is
    what you intended. Not so interested in kitten pics? :-) )

    There is likely a Better Way To Do It...
    In fact, although they appear to make sense from a data design point of
    view, the arrays at the second nested level of your data structure are
    going to cause you trouble in Puppet. Puppet does understand arrays, but
    not iteration, so you would probably need to resort to Ruby DSL to do
    anything useful with them. If you change them to hashes, however, then you
    may have an easier time. I.e.:

    directorymap:
    www.blah.com:
    www.blah.com0:
    source: dodgy-uploaded-jpegs
    target: www.blah.com/wp-content/uploads
    www.blah.com1:
    source: disturbing-home-videos
    target: www.blah.com/wp-content/filth
    www.nice-site.co.uk:
    www.nice-site.co.uk0:
    source: kitten-pics
    target: www.nice-site.co.uk/site-content/user-data

    That duplicates data even more, however. You could improve it further
    (IMO) by flattening it a bit and pushing down the site info, such as:

    directorymap:
    artificial-id-0:
    source: dodgy-uploaded-jpegs
    site: www.blah.com
    target: wp-content/uploads <http://www.blah.com/wp-content/uploads>
    artificial-id-1:
    source: disturbing-home-videos
    site: www.blah.com
    target: wp-content/filth <http://www.blah.com/wp-content/filth>
    artificial-id-2:
    source: kitten-pics
    site: www.nice-site.co.uk
    target: site-content/user-data<http://www.nice-site.co.uk/site-content/user-data>

    Note that now the data structure is a hash whose keys are suitable
    identifiers for resources you want to manage, and whose values are hashes
    of the desired properties of those resources. That's exactly what you need
    for the create_resources() function.

    To use create_resources(), you also need a resource type to accept those
    parameters, and since there is no built-in type that does what you want,
    your best bet would be a defined type:

    define content::link($source, $site, target) {
    file { "${content::base_directory}/${site}/${target}":
    ensure => link,
    target => "${site}/${source}"
    }
    }

    Note that variable $content::base_directory must be defined elsewhere (in a
    separate class "content", in this case). This is essential because the
    base directory is not included in your data, yet is needed to correctly
    identify the link's full path.

    With those pieces in place, you can tie it all together as simply as this:

    create_resources('content::link', hiera('directorymap'))


    If you don't like the revised form of your data that I suggested, or for
    future problems, there are at least a few other tools you should have in
    your toolbox:

    1) A Puppet function to extract an array of keys from a hash. You can
    write your own if you like (it's easy), but there is one ready-made in
    Puppetlabs's useful 'stdlib' module
    (https://github.com/puppetlabs/puppetlabs-stdlib).

    2) Defined types. Learn them, live them, love them. And especially, know
    that inside a defined type body, you can refer to instances' given titles
    via the automatic $name and/or $title variables.

    3) Multiple resource declaration syntax. If the title of a resource
    declaration is an array -- either a literal one, or a variable with an
    array value -- then it means one resource for each element of the array,
    with the corresponding element as resource title. For instance,

    file { [ '/tmp/foo', '/tmp/bar' ]: ensure => present }

    is equivalent to

    file { '/tmp/foo': ensure => present }
    file { '/tmp/bar': ensure => present }

    That can be combined in useful ways with (1) and (2). In particular, you
    can extract an array of hash keys, declare instances of a defined type with
    those titles, and in the body of the defined type use the title to extract
    and work with one element of the original hash. So, if you wanted to avoid
    the create_resources() function (but use the same data structure) then you
    could do something like this instead:

    $keys = keys($content::directories)

    content_link2 { $keys: }

    define content::link2() {
    $source = $content::directories[$name]['source']
    $site = $content::directories[$name]['site']
    $target = $content::directories[$name]['target']

    file { "${content::base_directory}/${site}/${target}":
    ensure => link,
    target => "${site}/${source}"
    }
    }



    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/9eHSlGxYMKYJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • John Hawkes-Reed at Sep 21, 2012 at 4:47 pm

    On 21 Sep 2012, at 15:42, jcbollinger wrote:



    On Friday, September 21, 2012 6:04:37 AM UTC-5, John Hawkes-Reed wrote:
    Hello.

    I think I'm being an idiot (not for the first time), but given the following hiera paragraph:

    directorymap:
    www.blah.com:
    - source: dodgy-uploaded-jpegs
    target: www.blah.com/wp-content/uploads
    - source: disturbing-home-videos
    target: www.blah.com/wp-content/filth
    www.nice-site.co.uk:
    - source: kitten-pics
    - target: www.nice-site.co.uk/site-content/user-data

    What's the least-worst way of extracting that, such that puppet pseudo-code along the lines of

    'ln -s ${site}/${source} ${site}{$target}' would work?


    So, it looks like that would expand to commands such as

    ln -s www.nice-site.co.uk/kitten-pics www.nice-site.co.ukwww.nice-site.co.uk/site-content/user-data

    I suspect the duplication of the site on the target side is not what you want, but it follows from your data. Nevertheless, I guess your real question is about how to make Puppet digest your nested data structure.
    Yep. I've been staring at the problem slightly too long...
    (Note also that your YAML data structure under "nice-site" is inconsistent with its cousin under "blah"; it appears that the structure under "blah" is what you intended. Not so interested in kitten pics? :-) ) :p

    There is likely a Better Way To Do It...


    In fact, although they appear to make sense from a data design point of view, the arrays at the second nested level of your data structure are going to cause you trouble in Puppet. Puppet does understand arrays, but not iteration, so you would probably need to resort to Ruby DSL to do anything useful with them. If you change them to hashes, however, then you may have an easier time. I.e.:

    directorymap:
    www.blah.com:
    www.blah.com0:
    source: dodgy-uploaded-jpegs
    target: www.blah.com/wp-content/uploads
    www.blah.com1:
    source: disturbing-home-videos
    target: www.blah.com/wp-content/filth
    www.nice-site.co.uk:
    www.nice-site.co.uk0:
    source: kitten-pics
    target: www.nice-site.co.uk/site-content/user-data

    That duplicates data even more, however. You could improve it further (IMO) by flattening it a bit and pushing down the site info, such as:

    directorymap:
    artificial-id-0:
    source: dodgy-uploaded-jpegs
    site: www.blah.com
    target: wp-content/uploads
    artificial-id-1:
    source: disturbing-home-videos
    site: www.blah.com
    target: wp-content/filth
    artificial-id-2:
    source: kitten-pics
    site: www.nice-site.co.uk
    target: site-content/user-data

    Note that now the data structure is a hash whose keys are suitable identifiers for resources you want to manage, and whose values are hashes of the desired properties of those resources. That's exactly what you need for the create_resources() function.

    To use create_resources(), you also need a resource type to accept those parameters, and since there is no built-in type that does what you want, your best bet would be a defined type:

    define content::link($source, $site, target) {
    file { "${content::base_directory}/${site}/${target}":
    ensure => link,
    target => "${site}/${source}"
    }
    }

    Note that variable $content::base_directory must be defined elsewhere (in a separate class "content", in this case). This is essential because the base directory is not included in your data, yet is needed to correctly identify the link's full path.

    With those pieces in place, you can tie it all together as simply as this:

    create_resources('content::link', hiera('directorymap'))


    If you don't like the revised form of your data that I suggested, or for future problems, there are at least a few other tools you should have in your toolbox:

    1) A Puppet function to extract an array of keys from a hash. You can write your own if you like (it's easy), but there is one ready-made in Puppetlabs's useful 'stdlib' module (https://github.com/puppetlabs/puppetlabs-stdlib).

    2) Defined types. Learn them, live them, love them. And especially, know that inside a defined type body, you can refer to instances' given titles via the automatic $name and/or $title variables.

    3) Multiple resource declaration syntax. If the title of a resource declaration is an array -- either a literal one, or a variable with an array value -- then it means one resource for each element of the array, with the corresponding element as resource title. For instance,

    file { [ '/tmp/foo', '/tmp/bar' ]: ensure => present }

    is equivalent to

    file { '/tmp/foo': ensure => present }
    file { '/tmp/bar': ensure => present }

    That can be combined in useful ways with (1) and (2). In particular, you can extract an array of hash keys, declare instances of a defined type with those titles, and in the body of the defined type use the title to extract and work with one element of the original hash. So, if you wanted to avoid the create_resources() function (but use the same data structure) then you could do something like this instead:

    $keys = keys($content::directories)

    content_link2 { $keys: }

    define content::link2() {
    $source = $content::directories[$name]['source']
    $site = $content::directories[$name]['site']
    $target = $content::directories[$name]['target']

    file { "${content::base_directory}/${site}/${target}":
    ensure => link,
    target => "${site}/${source}"
    }
    }
    That's really quite wonderful. Thank you.

    As a team, we're in the middle of re-factoring a lot of old (in Puppet terms) code, which means Barberising the modules, lots of Jenkins runs with puppet-lint... And now discovering that we can throw away circa half the cruft if we use hiera/create_resources right and spend some time looking carefully at the design of the yaml in the hiera tree.


    --
    John Hawkes-Reed
    Systems Administrator. Future Publishing. x 2526

    --
    Future Publishing Limited (registered company number 2008885) is a wholly owned subsidiary of Future plc (registered company number 3757874), both of which are incorporated in England and Wales and share the same registered address at Beauford Court, 30 Monmouth Street, Bath BA1 2BW.

    This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error please reply to this email and then delete it. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Future.

    The recipient should check this email and any attachments for the presence of viruses. Future accepts no liability for any damage caused by any virus transmitted by this email.

    Future may regularly and randomly monitor outgoing and incoming emails and other telecommunications on its email and telecommunications systems. By replying to this email you give your consent to such monitoring.

    *****
    Save resources: think before you print.


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedSep 21, '12 at 11:04a
activeSep 21, '12 at 4:47p
posts3
users2
websitepuppetlabs.com

2 users in discussion

John Hawkes-Reed: 2 posts Jcbollinger: 1 post

People

Translate

site design / logo © 2022 Grokbase