[Puppet Users] Wood/trees/hiera discrimination problem

On Friday, September 21, 2012 6:04:37 AM UTC-5, John Hawkes-Reed wrote:
Hello.

I think I'm being an idiot (not for the first time), but given the
following hiera paragraph:

directorymap:
www.blah.com:
- source: dodgy-uploaded-jpegs
target: www.blah.com/wp-content/uploads
- source: disturbing-home-videos
target: www.blah.com/wp-content/filth
www.nice-site.co.uk:
- source: kitten-pics
- target: www.nice-site.co.uk/site-content/user-data

What's the least-worst way of extracting that, such that puppet
pseudo-code along the lines of

'ln -s ${site}/${source} ${site}{$target}' would work?

On 21 Sep 2012, at 15:42, jcbollinger wrote:



On Friday, September 21, 2012 6:04:37 AM UTC-5, John Hawkes-Reed wrote:
Hello.

I think I'm being an idiot (not for the first time), but given the following hiera paragraph:

directorymap:
www.blah.com:
- source: dodgy-uploaded-jpegs
target: www.blah.com/wp-content/uploads
- source: disturbing-home-videos
target: www.blah.com/wp-content/filth
www.nice-site.co.uk:
- source: kitten-pics
- target: www.nice-site.co.uk/site-content/user-data

What's the least-worst way of extracting that, such that puppet pseudo-code along the lines of

'ln -s ${site}/${source} ${site}{$target}' would work?


So, it looks like that would expand to commands such as

ln -s www.nice-site.co.uk/kitten-pics www.nice-site.co.ukwww.nice-site.co.uk/site-content/user-data

I suspect the duplication of the site on the target side is not what you want, but it follows from your data. Nevertheless, I guess your real question is about how to make Puppet digest your nested data structure.
Yep. I've been staring at the problem slightly too long...
(Note also that your YAML data structure under "nice-site" is inconsistent with its cousin under "blah"; it appears that the structure under "blah" is what you intended. Not so interested in kitten pics? :-) ) :p

There is likely a Better Way To Do It...


In fact, although they appear to make sense from a data design point of view, the arrays at the second nested level of your data structure are going to cause you trouble in Puppet. Puppet does understand arrays, but not iteration, so you would probably need to resort to Ruby DSL to do anything useful with them. If you change them to hashes, however, then you may have an easier time. I.e.:

directorymap:
www.blah.com:
www.blah.com0:
source: dodgy-uploaded-jpegs
target: www.blah.com/wp-content/uploads
www.blah.com1:
source: disturbing-home-videos
target: www.blah.com/wp-content/filth
www.nice-site.co.uk:
www.nice-site.co.uk0:
source: kitten-pics
target: www.nice-site.co.uk/site-content/user-data

That duplicates data even more, however. You could improve it further (IMO) by flattening it a bit and pushing down the site info, such as:

directorymap:
artificial-id-0:
source: dodgy-uploaded-jpegs
site: www.blah.com
target: wp-content/uploads
artificial-id-1:
source: disturbing-home-videos
site: www.blah.com
target: wp-content/filth
artificial-id-2:
source: kitten-pics
site: www.nice-site.co.uk
target: site-content/user-data

Note that now the data structure is a hash whose keys are suitable identifiers for resources you want to manage, and whose values are hashes of the desired properties of those resources. That's exactly what you need for the create_resources() function.

To use create_resources(), you also need a resource type to accept those parameters, and since there is no built-in type that does what you want, your best bet would be a defined type:

define content::link($source, $site, target) {
file { "${content::base_directory}/${site}/${target}":
ensure => link,
target => "${site}/${source}"
}
}

Note that variable $content::base_directory must be defined elsewhere (in a separate class "content", in this case). This is essential because the base directory is not included in your data, yet is needed to correctly identify the link's full path.

With those pieces in place, you can tie it all together as simply as this:

create_resources('content::link', hiera('directorymap'))


If you don't like the revised form of your data that I suggested, or for future problems, there are at least a few other tools you should have in your toolbox:

1) A Puppet function to extract an array of keys from a hash. You can write your own if you like (it's easy), but there is one ready-made in Puppetlabs's useful 'stdlib' module (https://github.com/puppetlabs/puppetlabs-stdlib).

2) Defined types. Learn them, live them, love them. And especially, know that inside a defined type body, you can refer to instances' given titles via the automatic $name and/or $title variables.

3) Multiple resource declaration syntax. If the title of a resource declaration is an array -- either a literal one, or a variable with an array value -- then it means one resource for each element of the array, with the corresponding element as resource title. For instance,

file { [ '/tmp/foo', '/tmp/bar' ]: ensure => present }

is equivalent to

file { '/tmp/foo': ensure => present }
file { '/tmp/bar': ensure => present }

That can be combined in useful ways with (1) and (2). In particular, you can extract an array of hash keys, declare instances of a defined type with those titles, and in the body of the defined type use the title to extract and work with one element of the original hash. So, if you wanted to avoid the create_resources() function (but use the same data structure) then you could do something like this instead:

$keys = keys($content::directories)

content_link2 { $keys: }

define content::link2() {
$source = $content::directories[$name]['source']
$site = $content::directories[$name]['site']
$target = $content::directories[$name]['target']

file { "${content::base_directory}/${site}/${target}":
ensure => link,
target => "${site}/${source}"
}
}