FAQ
So… I'm coming across a situation where I foresee myself having a collection of related k/v pairs in hiera. I want to create a thing… lets say a yumrepo for example's sake, but this same idea translates to a vhost, a web app configuration , a virtual domain, etc.

lets assume a hierarchy of:

- %{region}/%{dc}/%{env}/%{function}/%{fqdn}
- %{region}/%{dc}/%{env}/%{function}/common
- %{region}/%{dc}/%{env}/common
- %{region}/%{dc}/common
- %{region}/common

in the us-east region common yaml file, should I create the following two repos values:

repo_descr: eastcoast-pub
repo_baseurl: http://myfoo.com/eastcoastgoodtimes
repo_gpgkey: http://myfoo.com/eastcoastgoodtimes/key
repo_gpgcheck: 1
repo_enabled: 1
repo_enablegroups: 0

repo_descr: eastcoast-internal
repo_baseurl: http://myfoo.com/eastcoast
repo_gpgkey: http://myfoo.com/eastcoast/key
repo_gpgcheck: 1
repo_enabled: 1
repo_enablegroups: 0


in the foo DC common file I have a repo for that datacenter:

repo_descr: foo
repo_baseurl: http://myfoo.com/goodtimes
repo_gpgkey: http://myfoo.com/goodtimes/key
repo_gpgcheck: 1
repo_enabled: 1
repo_enablegroups: 0



in the 'qa' env common yaml file

repo_descr: fooqarepo
repo_baseurl: http://myfoo.com/qa_crispybacon/goodtimes
repo_gpgkey: http://myfoo.com/qa_crispybacon/goodtimes/key
repo_gpgcheck: 1
repo_enabled: 1
repo_enablegroups: 0



in the webservers function common yaml file, I have another repo:

repo_descr: webservers
repo_baseurl: http://myfoo.com/bacon/goodtimes
repo_gpgkey: http://myfoo.com/bacon/goodtimes/key
repo_gpgcheck: 1
repo_enabled: 1
repo_enablegroups: 0






so I want all the qa web servers in the foo DC in the east coast to have all five of those repo files defined.. except for web02, of course, because he's special… looking at it this way makes my head hurt. I don't see a good way to actualize these repos onto a node, or exclude them from a unique snowflake if necessary, which sort-of defeats the purpose of hiera altogether.

Do I not create them as separate keys, but as a single 'repo' array, and in my manifest use a template to iterate through the array and associate the values contained within the array with values needed for the addition of a repo?

What have people found works "best"?





________________________________

This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Jcbollinger at Jun 12, 2012 at 10:18 pm

    On Tuesday, June 12, 2012 1:48:09 PM UTC-5, Wolf Noble wrote:
    So… I'm coming across a situation where I foresee myself having a
    collection of related k/v pairs in hiera. I want to create a thing… lets
    say a yumrepo for example's sake, but this same idea translates to a
    vhost, a web app configuration , a virtual domain, etc.

    lets assume a hierarchy of:

    - %{region}/%{dc}/%{env}/%{function}/%{fqdn}
    - %{region}/%{dc}/%{env}/%{function}/common
    - %{region}/%{dc}/%{env}/common
    - %{region}/%{dc}/common
    - %{region}/common

    in the us-east region common yaml file, should I create the following two
    repos values:

    repo_descr: eastcoast-pub
    repo_baseurl: http://myfoo.com/eastcoastgoodtimes
    repo_gpgkey: http://myfoo.com/eastcoastgoodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0

    repo_descr: eastcoast-internal
    repo_baseurl: http://myfoo.com/eastcoast
    repo_gpgkey: http://myfoo.com/eastcoast/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0


    in the foo DC common file I have a repo for that datacenter:

    repo_descr: foo
    repo_baseurl: http://myfoo.com/goodtimes
    repo_gpgkey: http://myfoo.com/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0



    in the 'qa' env common yaml file

    repo_descr: fooqarepo
    repo_baseurl: http://myfoo.com/qa_crispybacon/goodtimes
    repo_gpgkey: http://myfoo.com/qa_crispybacon/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0



    in the webservers function common yaml file, I have another repo:

    repo_descr: webservers
    repo_baseurl: http://myfoo.com/bacon/goodtimes
    repo_gpgkey: http://myfoo.com/bacon/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0






    so I want all the qa web servers in the foo DC in the east coast to have
    all five of those repo files defined.. except for web02, of course, because
    he's special… looking at it this way makes my head hurt. I don't see a good
    way to actualize these repos onto a node, or exclude them from a unique
    snowflake if necessary, which sort-of defeats the purpose of hiera
    altogether.

    Do I not create them as separate keys, but as a single 'repo' array, and
    in my manifest use a template to iterate through the array and associate
    the values contained within the array with values needed for the addition
    of a repo?

    What have people found works "best"?
    For complex data you need to put some structure in your YAML and process it
    accordingly in your manifests. In particular, in this case it would be to
    your advantage to use hiera's (and YAML's) support for arrays and / or
    hashes as values. For example:

    in us-east/common.yaml:
    ======
    region_repos:
    eastcoast_pub_repo: {
    descr: eastcoast-pub,
    baseurl: http://myfoo.com/eastcoastgoodtimes,
    gpgkey: http://myfoo.com/eastcoastgoodtimes/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    },
    eastcoast_internal_repo: {
    descr: eastcoast-internal,
    baseurl: http://myfoo.com/eastcoast,
    gpgkey: http://myfoo.com/eastcoast/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    }
    ======

    That's a hash of hashes, associated with the key "region_repos". You could
    use that in your manifests like so:

    mymodule::repos {
    create_resources('yumrepo', hiera('region_repos'))
    # perhaps also create_resources('yumrepo', hiera('dc_repos')), etc.
    }

    If that looks simple, that's because I tailored the YAML data structure to
    the expectations of the create_resources() function.

    It's also because the structure assumes that you will explicitly load the
    data for each hierarchy level you want, instead of, for example, using
    hiera_array() or hiera_hash() to assemble an even more complex aggregate
    data structure from various levels of your hierarchy. I think that's
    reasonable for your use case, especially given that you anticipate certain
    machines will need exceptions. You're still getting indirection out of it
    because hiera is choosing the appropriate region-, dc-, and
    environment-specific data files.

    There are many other ways to do it, but that one has the advantage of being
    simple (relatively speaking) and fairly easy to understand.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Rug-iffTkLkJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Wolf Noble at Jun 12, 2012 at 10:51 pm
    Hi John,

    Thanks a bunch. This seems incredibly sensible.

    If I'm understanding your suggestion properly, I interpret the best thing to do as something like:


    mymodule::repos {
    $region_repos = hiera('region_repos' , '' )
    $dc_repos = hiera('dc_repos' , '' )

    if $region_repos {
    create_resources('yumrepo', $region_repos))
    }
    if $dc_repos {
    create_resources('yumrepo', $dc_repos))
    }
    #...
    }

    and then I could override them like

    us-east/foo/qa/webservers/web02.qa.myfoo.com.yaml

    region_repos: ""
    dc_repos:
    foodc_qa_repo: {
    descr: foodc-qa,
    baseurl: http://myfoo.com/foodc-repo,
    gpgkey: http://myfoo.com/foodcrepo/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    }


    with the known caveat that I'd have to explicitly redeclare the entire hash at the most relevant tier if I wanted to change any element of the hash already defined.


    Did I miss anything?

    On Jun 12, 2012, at 5:18 PM, jcbollinger wrote:



    On Tuesday, June 12, 2012 1:48:09 PM UTC-5, Wolf Noble wrote:
    So… I'm coming across a situation where I foresee myself having a collection of related k/v pairs in hiera. I want to create a thing… lets say a yumrepo for example's sake, but this same idea translates to a vhost, a web app configuration , a virtual domain, etc.

    lets assume a hierarchy of:

    - %{region}/%{dc}/%{env}/%{function}/%{fqdn}
    - %{region}/%{dc}/%{env}/%{function}/common
    - %{region}/%{dc}/%{env}/common
    - %{region}/%{dc}/common
    - %{region}/common

    in the us-east region common yaml file, should I create the following two repos values:

    repo_descr: eastcoast-pub
    repo_baseurl: http://myfoo.com/eastcoastgoodtimes
    repo_gpgkey: http://myfoo.com/eastcoastgoodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0

    repo_descr: eastcoast-internal
    repo_baseurl: http://myfoo.com/eastcoast
    repo_gpgkey: http://myfoo.com/eastcoast/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0


    in the foo DC common file I have a repo for that datacenter:

    repo_descr: foo
    repo_baseurl: http://myfoo.com/goodtimes
    repo_gpgkey: http://myfoo.com/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0



    in the 'qa' env common yaml file

    repo_descr: fooqarepo
    repo_baseurl: http://myfoo.com/qa_crispybacon/goodtimes
    repo_gpgkey: http://myfoo.com/qa_crispybacon/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0



    in the webservers function common yaml file, I have another repo:

    repo_descr: webservers
    repo_baseurl: http://myfoo.com/bacon/goodtimes
    repo_gpgkey: http://myfoo.com/bacon/goodtimes/key
    repo_gpgcheck: 1
    repo_enabled: 1
    repo_enablegroups: 0






    so I want all the qa web servers in the foo DC in the east coast to have all five of those repo files defined.. except for web02, of course, because he's special… looking at it this way makes my head hurt. I don't see a good way to actualize these repos onto a node, or exclude them from a unique snowflake if necessary, which sort-of defeats the purpose of hiera altogether.

    Do I not create them as separate keys, but as a single 'repo' array, and in my manifest use a template to iterate through the array and associate the values contained within the array with values needed for the addition of a repo?

    What have people found works "best"?

    For complex data you need to put some structure in your YAML and process it accordingly in your manifests. In particular, in this case it would be to your advantage to use hiera's (and YAML's) support for arrays and / or hashes as values. For example:

    in us-east/common.yaml:
    ======
    region_repos:
    eastcoast_pub_repo: {
    descr: eastcoast-pub,
    baseurl: http://myfoo.com/eastcoastgoodtimes,
    gpgkey: http://myfoo.com/eastcoastgoodtimes/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    },
    eastcoast_internal_repo: {
    descr: eastcoast-internal,
    baseurl: http://myfoo.com/eastcoast,
    gpgkey: http://myfoo.com/eastcoast/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    }
    ======

    That's a hash of hashes, associated with the key "region_repos". You could use that in your manifests like so:

    mymodule::repos {
    create_resources('yumrepo', hiera('region_repos'))
    # perhaps also create_resources('yumrepo', hiera('dc_repos')), etc.
    }

    If that looks simple, that's because I tailored the YAML data structure to the expectations of the create_resources() function.

    It's also because the structure assumes that you will explicitly load the data for each hierarchy level you want, instead of, for example, using hiera_array() or hiera_hash() to assemble an even more complex aggregate data structure from various levels of your hierarchy. I think that's reasonable for your use case, especially given that you anticipate certain machines will need exceptions. You're still getting indirection out of it because hiera is choosing the appropriate region-, dc-, and environment-specific data files.

    There are many other ways to do it, but that one has the advantage of being simple (relatively speaking) and fairly easy to understand.


    John


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Rug-iffTkLkJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    ________________________________

    This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jun 13, 2012 at 1:24 pm

    On Tuesday, June 12, 2012 5:51:06 PM UTC-5, Wolf Noble wrote:
    Hi John,

    Thanks a bunch. This seems incredibly sensible.

    If I'm understanding your suggestion properly, I interpret the best thing
    to do as something like:


    mymodule::repos {
    $region_repos = hiera('region_repos' , '' )
    $dc_repos = hiera('dc_repos' , '' )

    if $region_repos {
    create_resources('yumrepo', $region_repos))
    }
    if $dc_repos {
    create_resources('yumrepo', $dc_repos))
    }
    #...
    }

    and then I could override them like

    us-east/foo/qa/webservers/web02.qa.myfoo.com.yaml

    region_repos: ""
    dc_repos:
    foodc_qa_repo: {
    descr: foodc-qa,
    baseurl: http://myfoo.com/foodc-repo,
    gpgkey: http://myfoo.com/foodcrepo/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    }


    with the known caveat that I'd have to explicitly redeclare the entire
    hash at the most relevant tier if I wanted to change any element of the
    hash already defined.


    Did I miss anything?
    I don't see any obvious errors or omissions. In particular, you are
    correct that overriding repos declared at a lower level will involve
    declaring an entirely new hash (of hashes) at the appropriate higher
    level. That's one of the costs attending complex data. All in all, it
    looks nicely fleshed out.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Ts3RcypLci0J.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Wolf Noble at Jun 13, 2012 at 3:21 pm
    Awesome!

    I wonder about one other thing though, Is create_resources likely to balk if there are values in the hash that are irrelevant for the resource type being fed?

    ie: if I have a region_repos hash that has both yum values, and apt values… (assuming that a debian sibling to the yumrepo type comes along at some point in the future) and feed them both via the same key, is that likely going to cause me pain? or should the key be geared more to one or the other, i.e.:

    region_yumrepo:
    foo_repo: {
    ..
    }

    region_aptsource
    foo_aptsource: {

    }

    just trying to think about where the pointy/sharp parts of this might be.

    On Jun 13, 2012, at 8:24 AM, jcbollinger wrote:



    On Tuesday, June 12, 2012 5:51:06 PM UTC-5, Wolf Noble wrote:
    Hi John,

    Thanks a bunch. This seems incredibly sensible.

    If I'm understanding your suggestion properly, I interpret the best thing to do as something like:


    mymodule::repos {
    $region_repos = hiera('region_repos' , '' )
    $dc_repos = hiera('dc_repos' , '' )

    if $region_repos {
    create_resources('yumrepo', $region_repos))
    }
    if $dc_repos {
    create_resources('yumrepo', $dc_repos))
    }
    #...
    }

    and then I could override them like

    us-east/foo/qa/webservers/web02.qa.myfoo.com.yaml

    region_repos: ""
    dc_repos:
    foodc_qa_repo: {
    descr: foodc-qa,
    baseurl: http://myfoo.com/foodc-repo,
    gpgkey: http://myfoo.com/foodcrepo/key,
    gpgcheck: 1,
    enabled: 1,
    enablegroups: 0
    }


    with the known caveat that I'd have to explicitly redeclare the entire hash at the most relevant tier if I wanted to change any element of the hash already defined.


    Did I miss anything?

    I don't see any obvious errors or omissions. In particular, you are correct that overriding repos declared at a lower level will involve declaring an entirely new hash (of hashes) at the appropriate higher level. That's one of the costs attending complex data. All in all, it looks nicely fleshed out.


    John


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Ts3RcypLci0J.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    ________________________________

    This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jun 14, 2012 at 1:02 pm

    On Wednesday, June 13, 2012 10:21:19 AM UTC-5, Wolf Noble wrote:
    I wonder about one other thing though, Is create_resources likely to balk
    if there are values in the hash that are irrelevant for the resource type
    being fed?
    I don't know off-hand, but I wouldn't be at all surprised if
    create_resources() fails when its input contains mappings for properties
    that the target resource type does not actually have. If you put such
    additional data into your repository structure then you'll probably need to
    process the result to select the desired portions before feeding it to
    create_resources(). You might need to write a custom function for that
    purpose. Alternatively, you could associate data for different resource
    types with different hiera keys.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/sZuRX4_F_BYJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJun 12, '12 at 6:48p
activeJun 14, '12 at 1:02p
posts6
users2
websitepuppetlabs.com

2 users in discussion

Jcbollinger: 3 posts Wolf Noble: 3 posts

People

Translate

site design / logo © 2022 Grokbase