FAQ
Hi folks,

In my common.yaml file I have the following hiera hashes:

system::packages:
     sshd:
         ensure: 'present'
     ...
...
more stuff here
...
system::packages:
     ntpd:
         ensure: 'present'

I naively assumed that Hiera would merge those 2 hashes together, just as
it does if they were present in different layers of the hierarchy. It
appears, though, that the latter simply overrides the former; I have an
sshd service with a dependency on the sshd package, but with the above
hiera data, the catalog fails to compile as the sshd package isn't in the
catalog.

Am I just missing some YAML syntax to stop the hash from being overwritten,
or am I perhaps misunderstanding the behaviour I'm seeing here? I did test
manually merging the hashes in the YAML file and that did indeed 'fix' the
problem.

If that behaviour is simply a fact of life, that's fine too; I'll fix the
data up, although there was an aesthetic/organisational reason for having
it structured as it is currently.

Thanks,

Matt.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKUTv3%2BDODmoGaO2rW%3D5DKUFFedZ-R8RgeXkAZFGZNqWGsXWWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Felix Frank at Apr 1, 2014 at 4:17 pm
    Hi,

    long story short: This is what the hiera_hash() function is for (as
    opposed to hiera()).

    HTH,
    Felix
    On 04/01/2014 06:15 PM, Matthew Burgess wrote:
    Hi folks,

    In my common.yaml file I have the following hiera hashes:

    system::packages:
    sshd:
    ensure: 'present'
    ...
    ...
    more stuff here
    ...
    system::packages:
    ntpd:
    ensure: 'present'

    I naively assumed that Hiera would merge those 2 hashes together, just
    as it does if they were present in different layers of the hierarchy.
    It appears, though, that the latter simply overrides the former; I have
    an sshd service with a dependency on the sshd package, but with the
    above hiera data, the catalog fails to compile as the sshd package isn't
    in the catalog.

    Am I just missing some YAML syntax to stop the hash from being
    overwritten, or am I perhaps misunderstanding the behaviour I'm seeing
    here? I did test manually merging the hashes in the YAML file and that
    did indeed 'fix' the problem.

    If that behaviour is simply a fact of life, that's fine too; I'll fix
    the data up, although there was an aesthetic/organisational reason for
    having it structured as it is currently.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/533AE688.3070702%40alumni.tu-berlin.de.
    For more options, visit https://groups.google.com/d/optout.
  • Matthew Burgess at Apr 1, 2014 at 4:35 pm
    Thanks Felix.

    I'm using the system module (https://forge.puppetlabs.com/erwbgy/system)
    and the system::packages module *is* using hiera_hash() (
    https://github.com/erwbgy/puppet-system/blob/master/manifests/packages.pp)
    (as is system::services which is also causing me the same problem).

    I assume hiera_hash()'s behaviour is default, i.e. I don't need to set
    :merge_behaviour in hiera.yaml?

    In addition, running 'hiera' to debug this shows the same behaviour; I
    assume there isn't a native hiera equivalent to Puppet's hiera_hash()
    function?

    Cheers,

    Matt.

    On 1 April 2014 17:17, Felix Frank wrote:

    Hi,

    long story short: This is what the hiera_hash() function is for (as
    opposed to hiera()).

    HTH,
    Felix
    On 04/01/2014 06:15 PM, Matthew Burgess wrote:
    Hi folks,

    In my common.yaml file I have the following hiera hashes:

    system::packages:
    sshd:
    ensure: 'present'
    ...
    ...
    more stuff here
    ...
    system::packages:
    ntpd:
    ensure: 'present'

    I naively assumed that Hiera would merge those 2 hashes together, just
    as it does if they were present in different layers of the hierarchy.
    It appears, though, that the latter simply overrides the former; I have
    an sshd service with a dependency on the sshd package, but with the
    above hiera data, the catalog fails to compile as the sshd package isn't
    in the catalog.

    Am I just missing some YAML syntax to stop the hash from being
    overwritten, or am I perhaps misunderstanding the behaviour I'm seeing
    here? I did test manually merging the hashes in the YAML file and that
    did indeed 'fix' the problem.

    If that behaviour is simply a fact of life, that's fine too; I'll fix
    the data up, although there was an aesthetic/organisational reason for
    having it structured as it is currently.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/533AE688.3070702%40alumni.tu-berlin.de
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKUTv3KDQA%3Dnw9vRnL6DL5dFujJ_s5pdQCKdmKJ7FmK9Hkk_GA%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matthew Burgess at Apr 1, 2014 at 4:51 pm
    Apologies for that last email; hiera_hash() is obviously doing *nearly* the
    right thing, as it *does* merge hashes across different levels of the
    hierarchy. It *doesn't* seem to be merging hashes within the same level of
    the hierarchy.

    Thanks,

    Matt.

    On 1 April 2014 17:35, Matthew Burgess wrote:

    Thanks Felix.

    I'm using the system module (https://forge.puppetlabs.com/erwbgy/system)
    and the system::packages module *is* using hiera_hash() (
    https://github.com/erwbgy/puppet-system/blob/master/manifests/packages.pp)
    (as is system::services which is also causing me the same problem).

    I assume hiera_hash()'s behaviour is default, i.e. I don't need to set
    :merge_behaviour in hiera.yaml?

    In addition, running 'hiera' to debug this shows the same behaviour; I
    assume there isn't a native hiera equivalent to Puppet's hiera_hash()
    function?

    Cheers,

    Matt.

    On 1 April 2014 17:17, Felix Frank wrote:

    Hi,

    long story short: This is what the hiera_hash() function is for (as
    opposed to hiera()).

    HTH,
    Felix
    On 04/01/2014 06:15 PM, Matthew Burgess wrote:
    Hi folks,

    In my common.yaml file I have the following hiera hashes:

    system::packages:
    sshd:
    ensure: 'present'
    ...
    ...
    more stuff here
    ...
    system::packages:
    ntpd:
    ensure: 'present'

    I naively assumed that Hiera would merge those 2 hashes together, just
    as it does if they were present in different layers of the hierarchy.
    It appears, though, that the latter simply overrides the former; I have
    an sshd service with a dependency on the sshd package, but with the
    above hiera data, the catalog fails to compile as the sshd package isn't
    in the catalog.

    Am I just missing some YAML syntax to stop the hash from being
    overwritten, or am I perhaps misunderstanding the behaviour I'm seeing
    here? I did test manually merging the hashes in the YAML file and that
    did indeed 'fix' the problem.

    If that behaviour is simply a fact of life, that's fine too; I'll fix
    the data up, although there was an aesthetic/organisational reason for
    having it structured as it is currently.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/533AE688.3070702%40alumni.tu-berlin.de
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKUTv3JLCw_FWJTnf8ZCJ_RtWuLKvwG6afa5oECxB1R%3DxKdjHQ%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Felix Frank at Apr 1, 2014 at 4:59 pm
    Hmm, so you have the same key multiple times in one yaml file? Why?
    On 04/01/2014 06:51 PM, Matthew Burgess wrote:
    Apologies for that last email; hiera_hash() is obviously doing *nearly*
    the right thing, as it *does* merge hashes across different levels of
    the hierarchy. It *doesn't* seem to be merging hashes within the same
    level of the hierarchy.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/533AF084.9030509%40alumni.tu-berlin.de.
    For more options, visit https://groups.google.com/d/optout.
  • Matthew Burgess at Apr 1, 2014 at 5:19 pm
    Like I said, it was purely for cosmetic/aesthetic purposes; I wanted to
    write the YAML file in the same order as things were defined in our design
    document so that it was easier to review that the implementation matched
    the design. Section 1 of that doc might cover something like NTP
    configuraiton, which requires a package, config file and service; Section 2
    might cover sshd, again with another package, config file and service. As
    it doesn't look like having multiple keys with the same name in the same
    file is supported, I'll get over my obsession with trying to lay the file
    out to match the order things appear in the design :-)

    Perhaps I need to look at this another way; I could lay my tests out (I'm
    sure I have some somewhere!) in the same order as the design doc, then I
    get the implementation review for free by executing the tests!

    Thanks for your help,

    Matt.

    On 1 April 2014 17:59, Felix Frank wrote:

    Hmm, so you have the same key multiple times in one yaml file? Why?
    On 04/01/2014 06:51 PM, Matthew Burgess wrote:
    Apologies for that last email; hiera_hash() is obviously doing *nearly*
    the right thing, as it *does* merge hashes across different levels of
    the hierarchy. It *doesn't* seem to be merging hashes within the same
    level of the hierarchy.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/533AF084.9030509%40alumni.tu-berlin.de
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKUTv3K9mgAMWKS13XwqGzid3CDVQoKxUdY3Ugyp0FvHS%2BJW-w%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Felix Frank at Apr 2, 2014 at 7:47 am
    Yeah, sounds about right.

    Pardon my prejudice, but from your description I get the feeling that
    you are not employing hiera to the height of its strengths. You want
    your hierarchy to contain *data* for your nodes. While it's possible to
    map all your resources to YAML and make heavy use of create_resources(),
    it is certainly not beneficial in all cases.

    I suggest at least a cursory glance at the roles/profiles pattern (or
    what I lovingly refer to as the hiera ENC) to see if it fits you better.
    An approach that has served me well is to make my manifests as
    pseudo-intelligent as possible (allowing puppet to do the right thing
    without us copy-pasting all the time) and sprinkle in hiera lookups only
    where customization is needed or I cannot whip up a logic based on facts
    alone.

    Note that either way, you won't solve your issue of spacing out hiera
    values per your given design structure. If you rely on manifests more,
    you can get much closer to that. (If it's a sound goal at all is yet
    another good question.)

    Regards,
    Felix
    On 04/01/2014 07:10 PM, Matthew Burgess wrote:
    Like I said, it was purely for cosmetic/aesthetic purposes; I wanted to
    write the YAML file in the same order as things were defined in our
    design document so that it was easier to review that the implementation
    matched the design. Section 1 of that doc might cover something like
    NTP configuraiton, which requires a package, config file and service;
    Section 2 might cover sshd, again with another package, config file and
    service. As it doesn't look like having multiple keys with the same
    name in the same file is supported, I'll get over my obsession with
    trying to lay the file out to match the order things appear in the
    design :-)

    Perhaps I need to look at this another way; I could lay my tests out
    (I'm sure I have some somewhere!) in the same order as the design doc,
    then I get the implementation review for free by executing the tests!

    Thanks for your help,

    Matt.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/533BC087.1010907%40alumni.tu-berlin.de.
    For more options, visit https://groups.google.com/d/optout.
  • Matthew Burgess at Apr 2, 2014 at 11:22 am

    On 2 April 2014 08:47, Felix Frank wrote:

    Yeah, sounds about right.

    Pardon my prejudice, but from your description I get the feeling that
    you are not employing hiera to the height of its strengths. You want
    your hierarchy to contain *data* for your nodes. While it's possible to
    map all your resources to YAML and make heavy use of create_resources(),
    it is certainly not beneficial in all cases.

    I suggest at least a cursory glance at the roles/profiles pattern (or
    what I lovingly refer to as the hiera ENC) to see if it fits you better.
    ​We're actually using the roles/profiles pattern too.

    However, rather than create a separate module for our custom stuff​

    ​which is largely going to be install a package, a config file and a
    service, we're utilising the aforementioned 'system' module to prevent
    module-proliferation.

    Our hiera structure mirrors the roles part of the roles/profiles pattern to
    enable us to customise the packages and services available for each unique
    role. Our common.yaml provides the data for our base profile.

    An approach that has served me well is to make my manifests as
    pseudo-intelligent as possible (allowing puppet to do the right thing
    without us copy-pasting all the time) and sprinkle in hiera lookups only
    where customization is needed or I cannot whip up a logic based on facts
    alone.
    ​It was that copy-pasting we were trying to get away from by utilising the
    system module. I also really like the fact that we've now got pretty close
    to 'infrastructure as configuration data' as opposed to 'infrastructure as
    code' (obviously there'll always be some custom modules required, but our
    goal is to keep those to a minimum).


    Note that either way, you won't solve your issue of spacing out hiera
    values per your given design structure. If you rely on manifests more,
    you can get much closer to that. (If it's a sound goal at all is yet
    another good question.)
    ​Thanks for your comments; it's always good to hear other points of view.

    Regards,

    Matt.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAKUTv3KearKAnraJ3t2yu0VU8hFPPrHL1SVy1hEkh%3D6_AFejMg%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Robin Bowes at Apr 1, 2014 at 5:00 pm
    I suspect duplicate keys are not valid YAML.

    R.
    On 1 Apr 2014 17:51, "Matthew Burgess" wrote:

    Apologies for that last email; hiera_hash() is obviously doing *nearly*
    the right thing, as it *does* merge hashes across different levels of the
    hierarchy. It *doesn't* seem to be merging hashes within the same level of
    the hierarchy.

    Thanks,

    Matt.

    On 1 April 2014 17:35, Matthew Burgess wrote:

    Thanks Felix.

    I'm using the system module (https://forge.puppetlabs.com/erwbgy/system)
    and the system::packages module *is* using hiera_hash() (
    https://github.com/erwbgy/puppet-system/blob/master/manifests/packages.pp)
    (as is system::services which is also causing me the same problem).

    I assume hiera_hash()'s behaviour is default, i.e. I don't need to set
    :merge_behaviour in hiera.yaml?

    In addition, running 'hiera' to debug this shows the same behaviour; I
    assume there isn't a native hiera equivalent to Puppet's hiera_hash()
    function?

    Cheers,

    Matt.

    On 1 April 2014 17:17, Felix Frank wrote:

    Hi,

    long story short: This is what the hiera_hash() function is for (as
    opposed to hiera()).

    HTH,
    Felix
    On 04/01/2014 06:15 PM, Matthew Burgess wrote:
    Hi folks,

    In my common.yaml file I have the following hiera hashes:

    system::packages:
    sshd:
    ensure: 'present'
    ...
    ...
    more stuff here
    ...
    system::packages:
    ntpd:
    ensure: 'present'

    I naively assumed that Hiera would merge those 2 hashes together, just
    as it does if they were present in different layers of the hierarchy.
    It appears, though, that the latter simply overrides the former; I have
    an sshd service with a dependency on the sshd package, but with the
    above hiera data, the catalog fails to compile as the sshd package isn't
    in the catalog.

    Am I just missing some YAML syntax to stop the hash from being
    overwritten, or am I perhaps misunderstanding the behaviour I'm seeing
    here? I did test manually merging the hashes in the YAML file and that
    did indeed 'fix' the problem.

    If that behaviour is simply a fact of life, that's fine too; I'll fix
    the data up, although there was an aesthetic/organisational reason for
    having it structured as it is currently.

    Thanks,

    Matt.
    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/533AE688.3070702%40alumni.tu-berlin.de
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit
    https://groups.google.com/d/msgid/puppet-users/CAKUTv3JLCw_FWJTnf8ZCJ_RtWuLKvwG6afa5oECxB1R%3DxKdjHQ%40mail.gmail.com<https://groups.google.com/d/msgid/puppet-users/CAKUTv3JLCw_FWJTnf8ZCJ_RtWuLKvwG6afa5oECxB1R%3DxKdjHQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
    .
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJGKfwDhE7t-0qK9TpDHrZ6j_ePM6_%2BwK-K%2BUFfBRw0C3qJdow%40mail.gmail.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jcbollinger at Apr 2, 2014 at 1:56 pm

    On Tuesday, April 1, 2014 12:00:45 PM UTC-5, Robin Bowes wrote:
    I suspect duplicate keys are not valid YAML.
    Indeed they are not. The keys of a YAML mapping are required to be unique (
    http://www.yaml.org/spec/1.2/spec.html#id2764044).

    How a particular YAML processor deals with duplicate mapping keys is
    implementation-defined, but the most likely behaviors are an error message
    and/or choosing one the two values presented for the duplicate key. I
    don't see how merging the values is even viable.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/202c4e9b-1846-4c7a-885b-ce1138c208dd%40googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedApr 1, '14 at 4:15p
activeApr 2, '14 at 1:56p
posts10
users4
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase