FAQ
Hello

I'm in the progress of refactoring a lot of our old puppet manifests
to finally get rid of all this global variables (we where young and... )

Now some of the settings are used in more then one class.

Example for one domain in hiera:

sssd::ldap_servers:
   - 'ldap://ldap1.example.com/'
   - 'ldap://ldap2.example.com/'

openldap::client::servers:
   - 'ldap://ldap1.example.com/'
   - 'ldap://ldap2.example.com/'

which i could simplify with anchors:

ldap_servers: &ldap_servers
   - 'ldap://ldap1.example.com/'
   - 'ldap://ldap2.example.com/'

sssd::ldap_servers: *ldap_servers
openldap::client::servers: *ldap_servers

A bit less redundancy than before, but there is still another problem.
If i want to set something else in a different domain i have to copy
the whole thing.

So if i add a new class i have to think of all the domains and other
levels where i overwrote that stuff and add that class.

It would be really nice if there was a way to only overwrite the
anchor so i don't have to copy all the classes that use the value to
the other YAMLs. From what i see in the hiera code this is not
possible at the moment.

Or is this a bad idea in general?

would appreciate your thoughts.

Greetings
Andreas

- --
Andreas Zuber
Linux System-Ingenieur
Puzzle ITC GmbH
www.puzzle.ch

Telefon +41 31 370 22 00
Direkt +41 31 370 22 49
Mobile +41 79 766 25 51
Fax +41 31 370 22 01

Werfen Sie einen Blick in unseren Blog:
<http://www.puzzle.ch/blog>


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jcbollinger at May 15, 2013 at 5:52 pm

    On Wednesday, May 15, 2013 8:43:39 AM UTC-5, Andreas Zuber wrote:
    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hello

    I'm in the progress of refactoring a lot of our old puppet manifests
    to finally get rid of all this global variables (we where young and... )

    Now some of the settings are used in more then one class.

    Example for one domain in hiera:

    sssd::ldap_servers:
    - 'ldap://ldap1.example.com/'
    - 'ldap://ldap2.example.com/'

    openldap::client::servers:
    - 'ldap://ldap1.example.com/'
    - 'ldap://ldap2.example.com/'

    which i could simplify with anchors:

    ldap_servers: &ldap_servers
    - 'ldap://ldap1.example.com/'
    - 'ldap://ldap2.example.com/'

    sssd::ldap_servers: *ldap_servers
    openldap::client::servers: *ldap_servers

    A bit less redundancy than before, but there is still another problem.
    If i want to set something else in a different domain i have to copy
    the whole thing.

    So if i add a new class i have to think of all the domains and other
    levels where i overwrote that stuff and add that class.

    It would be really nice if there was a way to only overwrite the
    anchor so i don't have to copy all the classes that use the value to
    the other YAMLs. From what i see in the hiera code this is not
    possible at the moment.

    I'm not sure what you mean by "overwrite the anchor".


    Or is this a bad idea in general?
    I think the bad idea here is to present fundamentally the same data via
    different, parallel paths. There is no sense of an authoritative source,
    and the resulting muddle is a natural result.

    If your sssd class and your opernldap::client class inherently must have
    the same list of LDAP servers in order for a correct configuration to
    result, then that should be modeled in Puppet, not made a requirement for
    the data maintainer to ensure. In that case hiera should define a single
    (both physically and logically) LDAP server list to any given client. On
    the Puppet side, either all classes that need the data should load it via
    the same key(s) (because it is and must be the same data), or else one
    designated class should load the data and serve as its steward, and others
    that want it should get it from that class. Anchors simply don't enter
    into it there.

    On the other hand, if there is no inherent constraint that the two classes
    use the same LDAP server lists, then data duplication on the hiera side is
    coincidental and perfectly appropriate. In that case I would advise you to
    avoid giving a different impression, as using a combined physical
    representation through anchors might do.

    Note, too, that hiera data doesn't have to be accessed via automatic class
    parameter binding, and indeed that classes don't have to be parameterized.
    Since Puppet 3, I have relaxed my stance toward parameterized classes
    (which was once highly critical), but I still don't think they provide much
    of a win. I would rather load external data via explicit hiera() calls --
    or via hiera_array() or hiera_hash() calls where that is useful, which
    automated parameter binding cannot provide. In this case, doing so would
    remove the false perception that the same data need to be provided via
    different keys.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Andreas Zuber at May 16, 2013 at 12:06 pm
    Hi John

    Thank you for your thoughts on this.
    On 05/15/2013 07:52 PM, jcbollinger wrote:
    I'm not sure what you mean by "overwrite the anchor".
    In defaults.yaml I set

    bar: &bar 1
    foo1::bar: &bar
    foo2::bar: &bar

    And for examlpe in hosts/mymachine.yaml only

    bar: &bar 2

    So when it tries to set foo1::bar for mymachine it would use the
    anchor from mymachine.yaml and ignore the one in defaults.yaml.

    overwrite isn't probably the right word here yes.
    If your sssd class and your opernldap::client class inherently must
    have the same list of LDAP servers in order for a correct
    configuration to result, then that should be modeled in Puppet, not
    made a requirement for the data maintainer to ensure. In that case
    hiera should define a single (both physically and logically) LDAP
    server list to any given client. On the Puppet side, either all
    classes that need the data should load it via the same key(s)
    (because it is and must be the same data), or else one designated
    class should load the data and serve as its steward, and others
    that want it should get it from that class. Anchors simply don't
    enter into it there.
    I know what you mean, but this ends in more global variables or some
    sort of singleton construct (steward) and I have no way of telling in
    hiera what classes in Puppet get influenced when I change this global
    variable.

    I would have to grep the whole code after this variable or I would
    have to hand them over to class parameters in some central location,
    which is just another place for the same thing I try to do in hiera
    directly with anchors but with one more layer which adds to the
    complexity.

    On the other hand we already have a class which seams to be a good
    place for this and I was probably just too stupid not to see that :-)

    I just realized I did not make clear what I try to achieve with this.
    The goal is to improve the visibility about where the data comes from
    and where it is used.
    Note, too, that hiera data doesn't have to be accessed via
    automatic class parameter binding, and indeed that classes don't
    have to be parameterized. Since Puppet 3, I have relaxed my stance
    toward parameterized classes (which was once highly critical), but
    I still don't think they provide much of a win. I would rather
    load external data via explicit hiera() calls -- or via
    hiera_array() or hiera_hash() calls where that is useful, which
    automated parameter binding cannot provide. In this case, doing so
    would remove the false perception that the same data need to be
    provided via different keys.
    I'm aware about that possibility, that's how we used hiera in puppet
    until now. We have hiera calls everywhere, in the middle of classes,
    templates, defines. And it is really hard right now to find out what
    changes if I manipulate the data.

    I think using variables only trough class parameters has some benefits
    when I comes to the visibility I mentioned earlier.

    The anchor thing was one solution I tried to come up with. But I think
    you have convinced me already to drop it again.

    Thanks again for your insights.

    Greetings
    Andreas


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jcbollinger at May 16, 2013 at 1:57 pm

    On Thursday, May 16, 2013 7:06:02 AM UTC-5, Andreas Zuber wrote:
    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi John

    Thank you for your thoughts on this.
    On 05/15/2013 07:52 PM, jcbollinger wrote:
    I'm not sure what you mean by "overwrite the anchor".
    In defaults.yaml I set

    bar: &bar 1
    foo1::bar: &bar
    foo2::bar: &bar

    And for examlpe in hosts/mymachine.yaml only

    bar: &bar 2

    So when it tries to set foo1::bar for mymachine it would use the
    anchor from mymachine.yaml and ignore the one in defaults.yaml.

    overwrite isn't probably the right word here yes.

    So, another thing you may be missing is that Hiera's YAML back end parses
    each data file as an independent document. Anchor references in one
    cannot, therefore, be resolved against anchor tags in another. I can't
    tell whether that solves your problem or scuttles your whole plan.

    If your sssd class and your opernldap::client class inherently must
    have the same list of LDAP servers in order for a correct
    configuration to result, then that should be modeled in Puppet, not
    made a requirement for the data maintainer to ensure. In that case
    hiera should define a single (both physically and logically) LDAP
    server list to any given client. On the Puppet side, either all
    classes that need the data should load it via the same key(s)
    (because it is and must be the same data), or else one designated
    class should load the data and serve as its steward, and others
    that want it should get it from that class. Anchors simply don't
    enter into it there.
    I know what you mean, but this ends in more global variables or some
    sort of singleton construct (steward) and I have no way of telling in
    hiera what classes in Puppet get influenced when I change this global
    variable.
    All puppet classes are singletons already. It's a fundamental aspect of
    their nature. Class variables are a perfectly well-recognized and accepted
    means of providing data to other classes and modules. (Do not confuse
    classes with defined type instances, which are very different in this
    regard.) It does not create any more variables than you already have, but
    it may eliminate some because you currently have duplicate variables in the
    form of parameters of different classes that hold the same data.

    Furthermore, you anyway have no reliable way of telling in hiera what
    classes will be influenced by data changes. Any class can retrieve data
    via any key, and only by convention does the presence of a key in hiera
    imply a consumer of that key among your manifests. Moreover, if you rely
    on anchors to duplicate data by reference, then you create for yourself the
    same problem of needing to grep for references, only you need to search
    your data instead your manifests.


    I would have to grep the whole code after this variable


    Why? I mean, for what purpose do you anticipate wanting to identify all
    the users of the data? It's much more usual to ask the opposite question:
    "from where does the data I'm using come?" and if the answer is a class
    variable then that's very easy to trace back.

    Anyway, how would that be improved by games with anchors in your hiera
    data? It seems to me that it would make the problem worse, because you
    have to both identify which hiera keys correspond to the data of interest,
    and then also identify which classes use those data, directly *and
    indirectly*. You still need to examine your manifests because nothing
    prevents one class from accessing the variables (including the parameters)
    of another, and also because you need to be wary of a class passing the
    data on to another via a parameterized-style class declaration.


    or I would
    have to hand them over to class parameters in some central location,
    which is just another place for the same thing I try to do in hiera
    directly with anchors but with one more layer which adds to the
    complexity.

    I agree, don't involve class parameters in this mix, except possibly for
    binding LDAP server data to one steward class. Generally speaking, it is
    not needful to do so. If you use parameterized-style class declarations
    (related to, but not the same as parameterized classes themselves),
    however, then that can complicate the equation.


    On the other hand we already have a class which seams to be a good
    place for this and I was probably just too stupid not to see that :-)

    I just realized I did not make clear what I try to achieve with this.
    The goal is to improve the visibility about where the data comes from
    and where it is used.
    Within your Puppet manifests, that goal cannot be served better than by
    making all users draw the same data from the same source, whether that's
    via the same hiera key or via the same variable(s) of the same class.
    Games with anchors or data duplication on the hiera side obscure the matter
    of where the data come from for the puppet side. I urge you to focus on
    the Puppet side, and to make your hiera data serve that. I think that will
    yield the best result, both functionally and structurally, not only more
    clear but also more maintainable.


    Note, too, that hiera data doesn't have to be accessed via
    automatic class parameter binding, and indeed that classes don't
    have to be parameterized. Since Puppet 3, I have relaxed my stance
    toward parameterized classes (which was once highly critical), but
    I still don't think they provide much of a win. I would rather
    load external data via explicit hiera() calls -- or via
    hiera_array() or hiera_hash() calls where that is useful, which
    automated parameter binding cannot provide. In this case, doing so
    would remove the false perception that the same data need to be
    provided via different keys.
    I'm aware about that possibility, that's how we used hiera in puppet
    until now. We have hiera calls everywhere, in the middle of classes,
    templates, defines. And it is really hard right now to find out what
    changes if I manipulate the data.

    That's not going to be solved by using anchors. All those data consumers
    are going to continue to consume the same data and be affected by the same
    changes -- its only a question of where they get the data. Again, anchors
    would just complicate matters there, as then you need to look for multiple
    different keys among your manifests and templates.

    I think using variables only trough class parameters has some benefits
    when I comes to the visibility I mentioned earlier.

    If you want to establish a policy of using hiera only to bind data to class
    parameters, then I don't see any inherent problem with that. All of your
    other hiera calls then need to become references to the appropriate class
    variables (which happen to be parameters), so that's not very far off from
    the single steward class idea I promoted. The steward class goes further
    by eliminating data aliasing within your manifests, which I would think
    serves your stated goals fairly well.

    The anchor thing was one solution I tried to come up with. But I think
    you have convinced me already to drop it again.

    Thanks again for your insights.
    I'm pleased to be of service.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 15, '13 at 1:43p
activeMay 16, '13 at 1:57p
posts4
users2
websitepuppetlabs.com

2 users in discussion

Jcbollinger: 2 posts Andreas Zuber: 2 posts

People

Translate

site design / logo © 2022 Grokbase