FAQ
In a posting a few days ago was this hiera.yaml source listing:
---------------------------------
:backends: - json
:hierarchy: - %{fqdn}
- %{lmax_role}_role
- %{lmax_env}_server
- %{pop}.tradefair
- common
:json:
:datadir: - /etc/puppet/private/
- /etc/puppet/environments/%{environment}/hiera_data_store/
- /etc/puppet/environments/%{environment}/rebirth_data_store/
- /etc/puppet/environments/%{environment}/satellite_system_groups/
---------------------------------

I am curious how one might utilize this configuration.

Can the same key value appear in more than one datadir ?
Or does one use the multiple datadir's to group keys in some fashion ?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Luke Bigum at May 18, 2012 at 8:24 am

    On 18/05/12 04:04, Dan White wrote:
    In a posting a few days ago was this hiera.yaml source listing:
    ---------------------------------
    :backends: - json
    :hierarchy: - %{fqdn}
    - %{lmax_role}_role
    - %{lmax_env}_server
    - %{pop}.tradefair
    - common
    :json:
    :datadir: - /etc/puppet/private/
    - /etc/puppet/environments/%{environment}/hiera_data_store/
    - /etc/puppet/environments/%{environment}/rebirth_data_store/
    - /etc/puppet/environments/%{environment}/satellite_system_groups/
    ---------------------------------

    I am curious how one might utilize this configuration.
    Hi Dan,

    That's my own terrible modifications to Hiera to support multiple data
    stores of the same type. Code is here, but I warn you, I hacked in the
    quickest way to get the functionality I wanted. I only modify
    hiera-json, would be simple to do any others though:

    https://github.com/lukebigum/hiera
    https://github.com/lukebigum/hiera-json

    As for use cases, a more elegant description is in this feature request:
    http://projects.puppetlabs.com/issues/13954
    Can the same key value appear in more than one datadir ?
    Yes, but precedence is taken just like Hiera's natural hierarchy, so
    data stores at the start of the array have higher priority. In my case
    for example, "hiera_data_store" overrides "satellite_system_groups".
    Or does one use the multiple datadir's to group keys in some fashion ?
    That was the the original idea of the hack - I had data coming from
    several places - some JSON written by hand, some generated by scripts
    (the 'satellite_system_groups' store is a dump of our old Red Hat
    Satellite System Groups and 'rebirth_data_store' is procedurally
    generated). I could have modified the generating scripts to read any
    existing JSON and write out a merged file, but I was rushing and I
    didn't think of that at the time ;-)

    It has managed to keep a nice logical separation of types of Hiera keys
    though and you can put your data stores in different locations, like
    having a 'private' dir for password hashes that's not under revision
    control. This leads to giving our Development team a Git repo that's the
    lowest priority data store and allow them to specify their own keys to
    configure their Dev servers. Any Admin still has the ability to
    overwrite any key in an data store above it, so it stops them doing
    anything silly like changing passwords, LDAP servers, etc, as these keys
    will already be set by us (Admins).

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Dan White at May 18, 2012 at 1:00 pm

    ----- Luke Bigum wrote:
    On 18/05/12 04:04, Dan White wrote:
    In a posting a few days ago was this hiera.yaml source listing:
    ---------------------------------
    :backends: - json
    :hierarchy: - %{fqdn}
    - %{lmax_role}_role
    - %{lmax_env}_server
    - %{pop}.tradefair
    - common
    :json:
    :datadir: - /etc/puppet/private/
    - /etc/puppet/environments/%{environment}/hiera_data_store/
    - /etc/puppet/environments/%{environment}/rebirth_data_store/
    - /etc/puppet/environments/%{environment}/satellite_system_groups/
    ---------------------------------

    I am curious how one might utilize this configuration.
    Hi Dan,

    That's my own terrible modifications to Hiera to support multiple data
    stores of the same type. Code is here, but I warn you, I hacked in the
    quickest way to get the functionality I wanted. I only modify
    hiera-json, would be simple to do any others though:

    https://github.com/lukebigum/hiera
    https://github.com/lukebigum/hiera-json

    As for use cases, a more elegant description is in this feature request:
    http://projects.puppetlabs.com/issues/13954
    Can the same key value appear in more than one datadir ?
    Yes, but precedence is taken just like Hiera's natural hierarchy, so
    data stores at the start of the array have higher priority. In my case
    for example, "hiera_data_store" overrides "satellite_system_groups".
    Or does one use the multiple datadir's to group keys in some fashion ?
    That was the the original idea of the hack - I had data coming from
    several places - some JSON written by hand, some generated by scripts
    (the 'satellite_system_groups' store is a dump of our old Red Hat
    Satellite System Groups and 'rebirth_data_store' is procedurally
    generated). I could have modified the generating scripts to read any
    existing JSON and write out a merged file, but I was rushing and I
    didn't think of that at the time ;-)

    It has managed to keep a nice logical separation of types of Hiera keys
    though and you can put your data stores in different locations, like
    having a 'private' dir for password hashes that's not under revision
    control. This leads to giving our Development team a Git repo that's the
    lowest priority data store and allow them to specify their own keys to
    configure their Dev servers. Any Admin still has the ability to
    overwrite any key in an data store above it, so it stops them doing
    anything silly like changing passwords, LDAP servers, etc, as these keys
    will already be set by us (Admins).

    -Luke
    Thanks for responding, Luke.

    This looks like a useful expansion of the hiera back-end, but as I am still fairly new to Puppet (and Python and Ruby for that matter), I think I will wait for this to be formally accepted and incorporated before trying it myself.

    One parting question: Is the :hierarchy: repeated in each mamber of the :datadir: ? Logically, it should repeat, but I'd rather ask and be certain.

    “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
    Bill Waterson (Calvin & Hobbes)

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Luke Bigum at May 21, 2012 at 8:16 am

    On 18/05/12 14:00, Dan White wrote:
    Thanks for responding, Luke.

    This looks like a useful expansion of the hiera back-end, but as I am still fairly new to Puppet (and Python and Ruby for that matter), I think I will wait for this to be formally accepted and incorporated before trying it myself.

    One parting question: Is the :hierarchy: repeated in each mamber of the :datadir: ? Logically, it should repeat, but I'd rather ask and be certain.

    “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
    Bill Waterson (Calvin& Hobbes)
    Hi Dan,

    Last week I was going to reply with "Sure does, here's an example
    debug..." and I released that my code didn't work they way I thought it
    did, I had the inner and outer loops around the wrong way ;-)

    So after much more butchering of RIP's code I can happily report that
    the :hierarchy: is searched as the outer loop, the :datadir: Array as
    the inner loop. This means that first the %{fqdn} files are searched in
    each member of :datadir:, then if nothing is found it moves on to
    '%{lmax_env}_role' and so on down the hierarchy.

    A (working!) example:

    [root@gs2puppet01 ~]# /opt/ruby-enterprise/bin/hiera -c
    /etc/puppet/hiera.yaml -y
    /var/lib/puppet/yaml/facts/test.example.com.yaml -a collective --debug

    DEBUG: Mon May 21 07:59:39 +0000 2012: Hiera JSON backend starting
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking up collective in JSON backend
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking for data source
    test.example.com
    DEBUG: Mon May 21 07:59:39 +0000 2012: Backend datadir for json is an
    Array, multiple data dirs to search
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/private/test.example.com.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found datafile
    /etc/puppet/environments/production/hiera_data_store/test.example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/rebirth_data_store/test.example.com.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/satellite_system_groups/test.example.com.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Attempting to read JSON from
    /etc/puppet/environments/production/hiera_data_store/test.example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking for data source _role
    DEBUG: Mon May 21 07:59:39 +0000 2012: Backend datadir for json is an
    Array, multiple data dirs to search
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/private/_role.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/hiera_data_store/_role.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/rebirth_data_store/_role.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/satellite_system_groups/_role.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking for data source _server
    DEBUG: Mon May 21 07:59:39 +0000 2012: Backend datadir for json is an
    Array, multiple data dirs to search
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/private/_server.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/hiera_data_store/_server.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/rebirth_data_store/_server.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/satellite_system_groups/_server.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking for data source example.com
    DEBUG: Mon May 21 07:59:39 +0000 2012: Backend datadir for json is an
    Array, multiple data dirs to search
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found datafile
    /etc/puppet/private/example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found datafile
    /etc/puppet/environments/production/hiera_data_store/example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/rebirth_data_store/example.com.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/satellite_system_groups/example.com.json,
    skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Attempting to read JSON from
    /etc/puppet/private/example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Attempting to read JSON from
    /etc/puppet/environments/production/hiera_data_store/example.com.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Looking for data source common
    DEBUG: Mon May 21 07:59:39 +0000 2012: Backend datadir for json is an
    Array, multiple data dirs to search
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/private/common.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found datafile
    /etc/puppet/environments/production/hiera_data_store/common.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Cannot find datafile
    /etc/puppet/environments/production/rebirth_data_store/common.json, skipping
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found datafile
    /etc/puppet/environments/production/satellite_system_groups/common.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Attempting to read JSON from
    /etc/puppet/environments/production/hiera_data_store/common.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Attempting to read JSON from
    /etc/puppet/environments/production/satellite_system_groups/common.json
    DEBUG: Mon May 21 07:59:39 +0000 2012: Found key collective in file
    /etc/puppet/environments/production/satellite_system_groups/common.json,
    appending to Array
    DEBUG: Mon May 21 07:59:39 +0000 2012: Lookup found answer ''
    [""]


    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 18, '12 at 3:04a
activeMay 21, '12 at 8:16a
posts4
users2
websitepuppetlabs.com

2 users in discussion

Dan White: 2 posts Luke Bigum: 2 posts

People

Translate

site design / logo © 2022 Grokbase