FAQ
Hello,

I've never looked at heira before. I'm looking into upgrading from 2.7 to
3, and it seems that heira is the recommended way to do things now. From
what I've read so far, it looks like one professed reason to use it is to
go from parameterized classes to hiera overrides, because I guess
parameterized classes are bad. So instead of:

class bar ($foo = 'blah') { ... }

One would have something like:

class bar { $foo = heira('bar::foo', 'blah') }

Is that correct?

I'm not yet sold on heira; so far it seems to just shift complexity outside
of the classes and add a little more in the process, with hierarchies and
stuff, and now I have data in multiple places... I still need case
statements to handle different OSes and stuff. Eh, we'll see. Maybe I just
haven't read something that explains the benefits well yet.

Josh

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Brian Lalor at May 5, 2013 at 1:54 am

    On May 4, 2013, at 9:50 PM, Josh wrote:

    So instead of:

    class bar ($foo = 'blah') { ... }

    One would have something like:

    class bar { $foo = heira('bar::foo', 'blah') }

    Is that correct?

    That's the hard way to do it. The easy way is to do
      class bar ($foo) { … }
    and Hiera will look up the key "bar::foo" from the configured data sources.

    --
    Brian Lalor
    blalor@bravo5.org


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ashley Penney at May 5, 2013 at 1:58 am

    On Sat, May 4, 2013 at 9:50 PM, Josh wrote:
    I'm not yet sold on heira; so far it seems to just shift complexity
    outside of the classes and add a little more in the process, with
    hierarchies and stuff, and now I have data in multiple places... I still
    need case statements to handle different OSes and stuff. Eh, we'll see.
    Maybe I just haven't read something that explains the benefits well yet.

    You should check out
    https://puppetlabs.com/blog/hiera-for-pouncers-and-stalkers/ as it's a
    great guide to the benefits of hiera. You really don't need case
    statements if you use heira, because you can make a hiera structure based
    on a fact like $::osfamily and then have:

    osfamily/Redhat.yaml
    osfamily/Debian.yaml

    This gives you the benefit that if you drop in a new .yaml file supporting
    a new distribution you only have to drop it into heira, and not modify
    multiple manifests. The article I linked probably does a better job of
    explaining it, but the goal is to remove all the data from your manifests
    leaving just variables behind, and then hiera fills in all the blanks. It
    takes some getting used to but I think you'll find it leads to better,
    cleaner, manifests in the long run.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Larry Fast at May 5, 2013 at 6:04 pm
    Hi Josh,
    Here's what we found. I would love to hear about other approaches because
    we're still debating and building out our service.

    1. Yes it's a great way to escape from parameterized classes. If you ever
    needed to 'include' a class in two places, parameterized classes makes this
    difficult or impossible.
    2. I don't recommend using automatic hiera lookup. There are some subtle
    gotchas in it especially when using hashes in hiera. In my office our
    recommended practice is as almost as you wrote it, class bar ($foo =
    hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
    an easier transition from parameterized classes so it may still be a good
    choice for you. As Ashley points out, read the latest docs!
    3. If you explicitly use the hiera() call, you'll get nice "variable not
    found" messages when you make a mistake. Clear error messages are rare in
    puppet so don't underestimate their value.
    4. Don't over complicate your hiera tree. It's not the place for
    everything. Pick your largest collection of system-to-system variation and
    put that in hiera.

    In our office we identified three variations that we *wanted* to put in
    hiera:
    1. Variation between servers *within* an instance of our service
    2. Variation between similar servers across different instances of our
    service (eg. Identity of the zookeeper in production, staging & QA)
    3. Version differences between instances of our service. IE. use hiera to
    manage the pipeline of releases.

    What we *actually* put into hiera was only #2. When #1 started creeping
    into hiera things got very messy. #1 is better handled by managing server
    Roles. #3 should never go into Hiera - but I'm still arguing this point
    with my co-workers.

    Caveate Emptor: We're still hemming and hawwing about all this.
    Cheers, Larry Fast



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Drew Michel at May 6, 2013 at 12:42 am
    Hi Larry,

    I'm curious to your opinion on point # 3, are you talking about OS packages
    or your organizations app version? If the latter, I was thinking of using
    hiera, maybe with a backend other than yaml such as redis, to store the
    version of the app, that way like you said it could be used in deploy
    pipeline. Why wouldn't you want to do this?


    In addition to what everyone else has said, one of the bigger win's of
    using hiera, is the flexibility it offers you with overriding variables.


    With the below hierarchy you can override based off an individual server,
    environment or datacenter. This could be powerful if you wanted to test
    some new JVM settings on just one server in production or an entire
    datacenter. You can use any facter fact you want to define your hierarchy.

    :hierarchy:
       - %{fqdn}
       - env/%{env}/common
       - datacenter/%{datacenter}
       - common



    Drew

    On Sunday, May 5, 2013 2:04:19 PM UTC-4, Larry Fast wrote:

    Hi Josh,
    Here's what we found. I would love to hear about other approaches because
    we're still debating and building out our service.

    1. Yes it's a great way to escape from parameterized classes. If you ever
    needed to 'include' a class in two places, parameterized classes makes this
    difficult or impossible.
    2. I don't recommend using automatic hiera lookup. There are some subtle
    gotchas in it especially when using hashes in hiera. In my office our
    recommended practice is as almost as you wrote it, class bar ($foo =
    hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
    an easier transition from parameterized classes so it may still be a good
    choice for you. As Ashley points out, read the latest docs!
    3. If you explicitly use the hiera() call, you'll get nice "variable not
    found" messages when you make a mistake. Clear error messages are rare in
    puppet so don't underestimate their value.
    4. Don't over complicate your hiera tree. It's not the place for
    everything. Pick your largest collection of system-to-system variation and
    put that in hiera.

    In our office we identified three variations that we *wanted* to put in
    hiera:
    1. Variation between servers *within* an instance of our service
    2. Variation between similar servers across different instances of our
    service (eg. Identity of the zookeeper in production, staging & QA)
    3. Version differences between instances of our service. IE. use hiera to
    manage the pipeline of releases.

    What we *actually* put into hiera was only #2. When #1 started creeping
    into hiera things got very messy. #1 is better handled by managing server
    Roles. #3 should never go into Hiera - but I'm still arguing this point
    with my co-workers.

    Caveate Emptor: We're still hemming and hawwing about all this.
    Cheers, Larry Fast
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Larry Fast at May 6, 2013 at 5:02 pm

    I'm curious to your opinion on point # 3, are you talking about OS
    packages or your organizations app version? If the latter, I was thinking
    of using hiera, maybe with a backend other than yaml such as redis, to
    store the version of the app, that way like you said it could be used in
    deploy pipeline. Why wouldn't you want to do this?
    Drew
    Hi Drew,
    I'm using a yaml backend so I can't speak about other options. So here's my
    disertation on why I (mostly) don't want version numbers in hiera. In my
    world view, our puppet code is just another component of our system. When
    I talk about our pipeline, I'm refering to deploying a version of our
    service into an environment. At the highest level, the service version
    defines the version of all components - including puppet code. BTW, it's
    not a big flat table, just the tip of the dependency tree.

    I get my religion from Continuous Delivery (Jez Humble, David Farley) and I
    can only hope that I'm not misreading it. Paraphrasing: You can break your
    service much faster with a configuration change than with a code change in
    one applications. So configuration should be version controlled.

    The direction I'm headed is to manage top level version control outside of
    our puppet code - with one caveate. We also have {%fqdn} as the first
    entry in our hiera tree and the ability to override the version setting via
    hiera. But it comes with the following warning:
    # FQDN should only be used for temporary overrides.

    Cheers,
    Larry

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Josh Endries at May 6, 2013 at 1:43 am
    Thanks for the replies!

    While I don't like the idea of "reaching in" to a class to set its
    variables, given the design flaws in Puppet with parameterized classes, I
    agree that this is a strength of Heira (so far, the only one, IMHO). Maybe
    this irks me because I'm also a developer who started with C++.

    I looked for docs and didn't find much beyond random blog posts, there are
    no "docs", really... I did read that article. I'm obviously a stalker. My
    (senior) co-worker that I'm training on Puppet is very much a pouncer. On
    Friday I had to say "no you actually need to read what it says" when trying
    to get him to use source instead of content (file type). He couldn't copy
    and paste, oh noes! I understand, though.

    I read a couple of the articles linked from that one, too, and there are
    lots of problems with them, and with the example Ashley gave (which the
    articles mention also). The NTP argument is pretty weak, IMHO. I don't
    think it's a great NTP module, I guess (I don't use it). =P No offense. =)

    I think the premise of supporting a new distro or OS (there is more than
    Linux out there, you know) with just variable flips almost silly. It
    practically requires modifying everything that will use that new OS. If all
    you do is flip a variable value, the module probably isn't very
    comprehensive. I almost always have functional differences for different
    OSes, like deploying an option file for a port, using .d files for some
    OSes and not for others, different templates, etc., not just a different
    package name.

    I completely agree with your finding #4. I keep reading about using Heira
    to include classes and it makes me cringe. That's such a terrible idea, and
    totally unnecessary, IMHO. It's trivial to include classes based on facts
    with Puppet itself, where all the rest of your structure is declared...
    Introducing a second area of declaring that stuff is just...ugh.

    I don't totally understand your variations...but I like the thought process
    and abstraction. It sounds like what I do all day long (on my "Puppet
    days"). I could see version numbers being useful in Heira: having a default
    and overriding it for dev or test. Instead of including instance::dev or
    instance::test classes, I guess, that do the same thing. We don't use
    version numbers at all in Puppet, though. =) This, however, is so far the
    only good use for Heira that I've found--overriding variables.

    I organize things into classes within a single module. I make high-level
    "functional" classes that group things together based on mostly
    non-technical stuff, like workflow or purpose, kind of like roles, and
    include those in nodes. These grouping are usually pretty well-defined,
    with limited scope. E.g. I have a few different support-group sub-classes
    (e.g.: support-group::research) that create different user accounts and set
    or override some parameters, or high level role classes like research-group
    sub-classes. I almost never include a class in a node manifest directly
    (usually just when testing). I have no node variables or top scope
    variables. We have lots of HPC clusters, from 1 or 2 to entire racks of
    machines, so putting everything into classes as a policy and habit really
    helps us. This eliminates almost all problems with overriding variables so
    far. I guess if we need to split something up (like different "types" of
    NTP server), I would create classes for them and include those where
    necessary. I'm not sure if/how Heira can help with this yet.

    The only real "problem" I have is user accounts integrating with various
    other things (like NFS/no NFS/root squash/pam_mount *and* AutoFS on the
    same box). We have a very big, very flexible user define with a layer on
    top of it for some corner cases. It's a lot of functional differences. I'm
    not yet sure if Heira could help with this, either, but it's something I'll
    be looking into...

    I might try to track my journey and make blog posts to help codify what I
    learn. Maybe someone will find it interesting or useful if they get into
    the same situation...

    GOodnight,
    Josh

    On Sun, May 5, 2013 at 2:04 PM, Larry Fast wrote:

    Hi Josh,
    Here's what we found. I would love to hear about other approaches because
    we're still debating and building out our service.

    1. Yes it's a great way to escape from parameterized classes. If you ever
    needed to 'include' a class in two places, parameterized classes makes this
    difficult or impossible.
    2. I don't recommend using automatic hiera lookup. There are some subtle
    gotchas in it especially when using hashes in hiera. In my office our
    recommended practice is as almost as you wrote it, class bar ($foo =
    hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
    an easier transition from parameterized classes so it may still be a good
    choice for you. As Ashley points out, read the latest docs!
    3. If you explicitly use the hiera() call, you'll get nice "variable not
    found" messages when you make a mistake. Clear error messages are rare in
    puppet so don't underestimate their value.
    4. Don't over complicate your hiera tree. It's not the place for
    everything. Pick your largest collection of system-to-system variation and
    put that in hiera.

    In our office we identified three variations that we *wanted* to put in
    hiera:
    1. Variation between servers *within* an instance of our service
    2. Variation between similar servers across different instances of our
    service (eg. Identity of the zookeeper in production, staging & QA)
    3. Version differences between instances of our service. IE. use hiera to
    manage the pipeline of releases.

    What we *actually* put into hiera was only #2. When #1 started creeping
    into hiera things got very messy. #1 is better handled by managing server
    Roles. #3 should never go into Hiera - but I'm still arguing this point
    with my co-workers.

    Caveate Emptor: We're still hemming and hawwing about all this.
    Cheers, Larry Fast

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/puppet-users/Z7GTiygw294/unsubscribe?hl=en
    .
    To unsubscribe from this group and all its topics, send an email to
    puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at May 6, 2013 at 4:34 pm

    On Sunday, May 5, 2013 8:43:52 PM UTC-5, Josh wrote:
    Thanks for the replies!

    Although it's a bit tardy, I'd like to add a word or two here.

    First, I think it's a mischaracterization to say "heira is the recommended
    way to do things now." Certainly, hiera is the favored interface with
    external data, and I have said before that hiera finally makes
    parameterized classes usable. That's not at all the same as hiera being
    recommended for *every*thing, however.

    You should not feel that hiera is being imposed on you. Its integration
    with the Puppet core is a big advance -- the most significant in Puppet 3
    -- but to the extent that you already have a manifest set that works in 2.7
    without hiera, you have no particular need to convert to using hiera.

    While I don't like the idea of "reaching in" to a class to set its
    variables,

    Do you mean "to *get* its variables"? Puppet variables can be set only in
    the scope to which they belong, and even then only once. It is not
    possible to reach in to a class to set its variables.

    On the other hand, a class can reach *out* to obtain values for its
    variables from an external source. Prevailing Puppet wisdom holds that it
    is good practice to separate data from classes.


    given the design flaws in Puppet with parameterized classes,

    Although I have never been a fan of parameterized classes, with Puppet 3 I
    no longer attribute the related flaws to parameterized classes themselves,
    but rather to the parameterized-class declaration syntax, and somewhat to
    the intentional blurring of the distinction between classes and resources.
    It's not a particular problem to single out certain variables that
    characterize the allowed variation in the state described by a given class
    -- i.e. parameters -- but it is a *fundamental* problem to bind data to
    parameters in a manner that depends on parse order. Now that there are
    alternatives to doing the latter, those are separable issues.

    I devote verbiage to this issue because the word "flaw" suggests something
    that can be fixed, but although class parameterization is basically ok (if
    overrated), the problems with binding data to classes via unrestricted DSL
    statements (parameterized-style class declarations) are not fixable. That
    does not reflect a design flaw so much as the nature of the problem.


    I agree that this is a strength of Heira (so far, the only one, IMHO).
    Maybe this irks me because I'm also a developer who started with C++.

    This is actually a hard place from which to come to Puppet. Puppet DSL
    uses some OO terms and has syntax and structure reminiscent of some OO
    languages, but it is essential to understand that it is NOT object-oriented
    in any but the most basic sense. Your developer instincts will send you
    the wrong signals when you see the words "class" or "inherit", for
    example. Coming from C/C++ may be especially hard, because "include" means
    something radically different in Puppet than in the C world, and "define"
    and defined types are very different beasts than C preprocessor macros.

    Also, the overall mindset attending development in imperative languages is
    just plain wrong for writing Puppet manifests. C++ (for example) focuses
    on actions, whereas Puppet DSL is almost entirely about state. Imperative
    programmers are typically inclined to ask the wrong question: "How do I
    make Puppet do <foo>?". The correct question is usually "How do I model
    <foo-result> in Puppet?"

    If you've overcome these issues then I offer my congratulations to you.

    I looked for docs and didn't find much beyond random blog posts, there are
    no "docs", really... I did read that article.

    Yes, this a significant weakness for hiera. I am sympathetic to your
    dissatisfaction there.


    I'm obviously a stalker. My (senior) co-worker that I'm training on Puppet
    is very much a pouncer. On Friday I had to say "no you actually need to
    read what it says" when trying to get him to use source instead of content
    (file type). He couldn't copy and paste, oh noes! I understand, though.

    I read a couple of the articles linked from that one, too, and there are
    lots of problems with them, and with the example Ashley gave (which the
    articles mention also). The NTP argument is pretty weak, IMHO. I don't
    think it's a great NTP module, I guess (I don't use it). =P No offense. =)

    I think the premise of supporting a new distro or OS (there is more than
    Linux out there, you know) with just variable flips almost silly. It
    practically requires modifying everything that will use that new OS. If all
    you do is flip a variable value, the module probably isn't very
    comprehensive. I almost always have functional differences for different
    OSes, like deploying an option file for a port, using .d files for some
    OSes and not for others, different templates, etc., not just a different
    package name.

    You're right, or course, that in many cases the differences from one OS to
    another requires more than just plugging in different package names, file
    names, etc.. Where you have deeper differences between configuration
    alternatives, it is entirely possible to use hiera to declare different
    classes for some nodes than for others, as appropriate to the node identity
    and fact pattern.


    I completely agree with your finding #4. I keep reading about using Heira
    to include classes and it makes me cringe. That's such a terrible idea, and
    totally unnecessary, IMHO. It's trivial to include classes based on facts
    with Puppet itself, where all the rest of your structure is declared...
    Introducing a second area of declaring that stuff is just...ugh.
    That's clearly a matter of taste. Perhaps there's a bit of subconscious
    anti-hiera bias there, too? Using hiera to identify classes to apply makes
    it serve essentially as an external node classifier. Or at least
    semi-external -- although hiera itself is integrated, the characteristic
    data are external. ENCs in general have a long history with Puppet and the
    concept is widely accepted. That doesn't necessarily make it right for
    you, but prevailing opinion doesn't agree with "ugh" to the basic idea.


    I don't totally understand your variations...but I like the thought
    process and abstraction. It sounds like what I do all day long (on my
    "Puppet days"). I could see version numbers being useful in Heira: having a
    default and overriding it for dev or test. Instead of including
    instance::dev or instance::test classes, I guess, that do the same thing.
    We don't use version numbers at all in Puppet, though. =) This, however, is
    so far the only good use for Heira that I've found--overriding variables.

    I organize things into classes within a single module. I make high-level
    "functional" classes that group things together based on mostly
    non-technical stuff, like workflow or purpose, kind of like roles, and
    include those in nodes. These grouping are usually pretty well-defined,
    with limited scope. E.g. I have a few different support-group sub-classes
    (e.g.: support-group::research) that create different user accounts and set
    or override some parameters, or high level role classes like research-group
    sub-classes. I almost never include a class in a node manifest directly
    (usually just when testing). I have no node variables or top scope
    variables. We have lots of HPC clusters, from 1 or 2 to entire racks of
    machines, so putting everything into classes as a policy and habit really
    helps us. This eliminates almost all problems with overriding variables so
    far. I guess if we need to split something up (like different "types" of
    NTP server), I would create classes for them and include those where
    necessary. I'm not sure if/how Heira can help with this yet.

    The only real "problem" I have is user accounts integrating with various
    other things (like NFS/no NFS/root squash/pam_mount *and* AutoFS on the
    same box). We have a very big, very flexible user define with a layer on
    top of it for some corner cases. It's a lot of functional differences. I'm
    not yet sure if Heira could help with this, either, but it's something I'll
    be looking into...

    I might try to track my journey and make blog posts to help codify what I
    learn. Maybe someone will find it interesting or useful if they get into
    the same situation...
    And that brings me back to the beginning: you should not feel that hiera is
    being imposed on you. You especially should not get the impression that
    your approach to designing modules and classes is challenged by the
    integration of hiera into Puppet 3. Your classes and modules do not need
    to be hiera-centric.

    If you are not sold on externalizing your data, then hiera will be of
    limited interest to you. And that's OK. If you also do not use
    parameterized classes then (i) good for you, and (ii) hiera may be of no
    interest to you at all. To the extent that you read external data or bind
    non-default data to class parameters, however, my technical advice to you
    is to do it via hiera.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 5, '13 at 1:50a
activeMay 6, '13 at 5:02p
posts8
users6
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase