FAQ
Hi,
This follows on a bit from the previous thread 'trouble with hiera and
puppet defines' [1]

Up to now I've had a large file of virtual resources and then enabled them
on demand
on various services. The very standard.

@metric{'1234:
one => 1
two => [1,2]
}
@metric{'abcd':
one => a,
two => [b,c]
}

and then somewhere realize potentially with an override <| title == 1234 |>
{ one => 2 }

metric is something like
define metric ($one, $two) {
# do stuff.
}

Attempting to move the data of virtual resources above into hiera now and
a few questions.

metric1234:
one: 1
two:
-'1'
-'2'

metricabcd:
one: a
two:
-'b'
-'c'

I accept to have to use that hierarchy rather than the perhaps more obvious
metric:
1234: ...
abcd: ...

since hiera can only select on the first level as the key?, i.e
hiera(metic[1234]) or something is not posible is it?

To realize in a manifest the following works:

$h = hiera_hash('metric4102')
$g = { '4102' => $h }
create_resources('lemon::metric',$g)

however what I would like to do is get the defaults from hiera within the
defined resource so it becomes
like the following which is incorrect but hopefully explains it.

metric{4102: one=> 'override'}

where metric is defined as

define metric (hiera("lemon${title}") {
# do stuff.
}

I can understand why that does not make sense, the define is only going to
evaluated once and badly.

Only option I can think of is to override the paramters inside the define
if they are non-sensical
defaults.

define metric(one=>'UNSET', two=>'UNSET) {

$h = hiera("lemon${title}")
if $one == 'UNSET' {
$one = $h['one']
}
if $two == 'UNSET' {
$two = $['two']
}
}

but this is it a bit bizarre.

I am looking for a better way for a defined resource to get its defaults
dynamically from hiera?

The motivation is that I probably only realize 20 of a potential > 1000
resources
on a particular host so dropping that yaml file data in a database backend
to hiera would
make it easier.

I realize I can override with an earlier priority yaml file and hiera_hash
but would like
the paramatized declaration to work also.

Steve.

[1] https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Ou9F3KdzqeE

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/dtpmuIJ-9h8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Jcbollinger at May 15, 2012 at 2:48 pm

    On May 14, 2:27 pm, Steve Traylen wrote:
    Hi,
    This follows on a bit from the previous thread 'trouble with hiera and
    puppet defines' [1]

    Up to now I've had a large file of virtual resources and then enabled them
    on demand
    on various services. The very standard.

    @metric{'1234:
    one => 1
    two => [1,2]}

    @metric{'abcd':
    one => a,
    two => [b,c]

    }

    and then somewhere realize potentially with an override <| title == 1234 |>
    { one => 2  }

    Overriding parameter values at realization time is mildly evil. You
    need at least to understand that doing this prevents you from
    realizing the overridden resource in more than one place, which
    otherwise is safe and sometimes useful.

    metric is something like
    define metric ($one, $two) {
    # do stuff.

    }

    Attempting to move the data of virtual resources above into hiera now  and
    a few questions.

    metric1234:
    one: 1
    two:
    -'1'
    -'2'

    metricabcd:
    one: a
    two:
    -'b'
    -'c'

    I accept  to have to use that hierarchy rather than the perhaps more obvious
    metric:
    1234: ...
    abcd: ...

    since hiera can only select on the first level as the key?, i.e
    hiera(metic[1234]) or something is not posible is it?

    Not exactly, but similar should be possible. Hiera can return hashes
    as values, so you ought to be able to do something like

    $metric_data = hiera('metric')
    $metric_1234 = $metric_data['1234']

    That sort of structure might also play well with Puppet's
    create_resources() function, which I see you already know about.

    To  realize in a manifest the following works:

    $h = hiera_hash('metric4102')
    $g = { '4102' => $h }
    create_resources('lemon::metric',$g)

    Ok. You need to know, if you don't already, that the hiera() function
    will return a hash if that's the type of value associated with the
    specified key. Hiera_hash() also returns a hash, but its main purpose
    is to collect and merge data from your whole hierarchy, instead of
    from just the highest-priority level where the key is found. I don't
    see any evidence from your example that the latter is what you're
    really after in that case.

    however what I would like to do is get the defaults from hiera within the
    defined resource so it becomes
    like the following which is incorrect but hopefully explains it.

    metric{4102: one=> 'override'}

    where metric is defined as

    define metric (hiera("lemon${title}") {
    # do stuff.

    }

    I can understand why that does not make sense, the define is only going to
    evaluated once and badly.

    What does not make sense is "default values" that vary with the
    resource instance you are declaring. At least, that doesn't match up
    with what Puppet means by that term.

    Only option I can think of is to override the paramters inside the define
    if they are non-sensical
    defaults.

    define metric(one=>'UNSET', two=>'UNSET) {

    $h = hiera("lemon${title}")
    if $one == 'UNSET' {
    $one = $h['one']
    }
    if $two == 'UNSET' {
    $two = $['two']
    }

    }

    but this is it a bit bizarre.

    And it wouldn't work as written, since you cannot redefine variables
    in Puppet, but something similar could be done. You could do
    something a little cleaner and clearer with the aid of a couple of
    functions. For example,

    define metric($one => 'UNSET', $two => 'UNSET') {
    $passed_params = remove_value({ 'one' => $one, 'two' => $two },
    'UNSET')
    $standard_params = hiera("metric_${title}")
    $resolved_params = merge($standard_params, $passed_params)
    # do something with $resolved_params['one']
    # do something with $resolved_params['two']
    }

    There is just such a merge() function in the Puppetlabs stdlib
    module. You would have to write remove_values() yourself, but it
    ought to be pretty simple.

    I am looking for a better way for a defined resource to get its defaults
    dynamically from hiera?

    If the "default" values were associated with top-level keys in your
    data store, then perhaps you could do something like this:

    define metric($one => hiera("metric${title}_one"), $two =>
    hiera("metric${title}_one")) {
    # ...
    }

    Again, however, part of the problem is that you are trying to use
    Puppet's default value facility in a way that was not intended.
    Generally speaking, default values are defaults for the *type*, not
    separate defaults for each instance.

    The motivation is that I probably only realize 20 of a potential > 1000
    resources
    on a particular host so dropping that yaml file data in a database backend
    to hiera would
    make it easier.

    I realize I can override with an earlier priority yaml file and hiera_hash
    but would like
    the paramatized declaration to work also.

    I think you're creating a mess for yourself, especially with parameter
    overrides at realization time. You would be better off moving all the
    way to hiera. In fact, you could leverage hiera further to pull those
    1000 virtual resource declarations out of your manifests altogether.
    For example:

    class metric::instances {
    # Retrieve an array of the names of the metric
    # instances wanted for this node
    $metrics = hiera('metrics')

    # Declare all the wanted 'metric' instances (concretely).
    # The definition relies on hiera for all needed data,
    # likely by merging data from two or more levels via
    # hiera_hash()
    metric::metric { $metrics: }
    }

    Isn't that better than thousands of lines of virtual resource
    declarations? It will be a lot lighter on your puppetmaster, too.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Steve Traylen at May 30, 2012 at 8:37 pm

    On Tuesday, 15 May 2012 16:48:48 UTC+2, jcbollinger wrote:
    On May 14, 2:27 pm, Steve Traylen wrote:
    Up to now I've had a large file of virtual resources and then enabled them
    on demand
    on various services. The very standard.

    @metric{'1234:
    one => 1
    two => [1,2]}

    @metric{'abcd':
    one => a,
    two => [b,c]
    }
    and then somewhere realize potentially with an override <| title == 1234
    { one => 2 }
    Overriding parameter values at realization time is mildly evil. You
    need at least to understand that doing this prevents you from
    realizing the overridden resource in more than one place, which
    otherwise is safe and sometimes useful.
    Agreed, i need to set paramaters at creation time, the parameters just may
    differ in place to place at creation time sometimes.
    metric is something like
    define metric ($one, $two) {
    # do stuff.

    }

    Attempting to move the data of virtual resources above into hiera now and
    a few questions.

    metric1234:
    one: 1
    two:
    -'1'
    -'2'

    metricabcd:
    one: a
    two:
    -'b'
    -'c'

    I accept to have to use that hierarchy rather than the perhaps more obvious
    metric:
    1234: ...
    abcd: ...

    since hiera can only select on the first level as the key?, i.e
    hiera(metic[1234]) or something is not posible is it?
    Not exactly, but similar should be possible. Hiera can return hashes
    as values, so you ought to be able to do something like

    $metric_data = hiera('metric')
    $metric_1234 = $metric_data['1234']

    That sort of structure might also play well with Puppet's
    create_resources() function, which I see you already know about.
    To realize in a manifest the following works:

    $h = hiera_hash('metric4102')
    $g = { '4102' => $h }
    create_resources('lemon::metric',$g)
    Ok. You need to know, if you don't already, that the hiera() function
    will return a hash if that's the type of value associated with the
    specified key. Hiera_hash() also returns a hash, but its main purpose
    is to collect and merge data from your whole hierarchy, instead of
    from just the highest-priority level where the key is found. I don't
    see any evidence from your example that the latter is what you're
    really after in that case.
    however what I would like to do is get the defaults from hiera within the
    defined resource so it becomes
    like the following which is incorrect but hopefully explains it.

    metric{4102: one=> 'override'}

    where metric is defined as

    define metric (hiera("lemon${title}") {
    # do stuff.

    }

    I can understand why that does not make sense, the define is only going to
    evaluated once and badly.
    What does not make sense is "default values" that vary with the
    resource instance you are declaring. At least, that doesn't match up
    with what Puppet means by that term.
    Okay, I guess i am just using hiera as a data store because its easy to
    access that
    and let it deal with the complexity.
    Only option I can think of is to override the paramters inside the define
    if they are non-sensical
    defaults.

    define metric(one=>'UNSET', two=>'UNSET) {

    $h = hiera("lemon${title}")
    if $one == 'UNSET' {
    $one = $h['one']
    }
    if $two == 'UNSET' {
    $two = $['two']
    }

    }

    but this is it a bit bizarre.
    And it wouldn't work as written, since you cannot redefine variables
    in Puppet, but something similar could be done. You could do
    something a little cleaner and clearer with the aid of a couple of
    functions. For example,

    define metric($one => 'UNSET', $two => 'UNSET') {
    $passed_params = remove_value({ 'one' => $one, 'two' => $two },
    'UNSET')
    $standard_params = hiera("metric_${title}")
    $resolved_params = merge($standard_params, $passed_params)
    # do something with $resolved_params['one']
    # do something with $resolved_params['two']
    }

    There is just such a merge() function in the Puppetlabs stdlib
    module. You would have to write remove_values() yourself, but it
    ought to be pretty simple.
    I am looking for a better way for a defined resource to get its defaults
    dynamically from hiera?
    If the "default" values were associated with top-level keys in your
    data store, then perhaps you could do something like this:

    define metric($one => hiera("metric${title}_one"), $two =>
    hiera("metric${title}_one")) {
    # ...
    }

    Again, however, part of the problem is that you are trying to use
    Puppet's default value facility in a way that was not intended.
    Generally speaking, default values are defaults for the *type*, not
    separate defaults for each instance.
    What I went for in end which was a merge of some of your comments was

    define metric( $param => undef ) {
    $metricid = $title
    $metrichash = hiera("lemon_${metricid}")

    # Set $my_param to first passed parameter, then hiera value then
    default value.
    if ( $param != undef )
    $my_param = $param
    } elsif ( has_key($metrichas,'param')
    $my_param = $metrichash['hash']
    } else {
    $my_param = undef
    }

    And then use $my_param in the erb template.

    The motivation is that I probably only realize 20 of a potential > 1000
    resources
    on a particular host so dropping that yaml file data in a database backend
    to hiera would
    make it easier.

    I realize I can override with an earlier priority yaml file and
    hiera_hash
    but would like
    the paramatized declaration to work also.

    I think you're creating a mess for yourself, especially with parameter
    overrides at realization time. You would be better off moving all the
    way to hiera. In fact, you could leverage hiera further to pull those
    1000 virtual resource declarations out of your manifests altogether.
    For example:
    I now have no virtual resources, I just load the bits of data from hiera as
    and when needed.

    Thanks for the input , it helped me a lot with my understanding of the
    finer points.

    I am at the moment fairly happy with the result simply because it allows
    one person to maintain
    the library of metrics in YAML and other people to enable what they want.
    .. And they can still
    do override of the data which is especially useful when you want to
    override based
    on a fact for instance which can only be set in a manifest.

    For reference: https://github.com/CERN-Ops/puppet-lemon

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/2m8so2NH2lkJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 14, '12 at 7:28p
activeMay 30, '12 at 8:37p
posts3
users2
websitepuppetlabs.com

2 users in discussion

Steve Traylen: 2 posts Jcbollinger: 1 post

People

Translate

site design / logo © 2022 Grokbase