FAQ
I'm fighting with a ticklish issue. We have some groups and users that only belong on some systems. So we made all users virtual and then realize them in classes specific to those system types. This works quite well for the users, but not for the groups. When you specify a user, you have to list all the groups they are in.
groups => ['support',ops','dev'],

Obviously some groups aren't realized on all systems, so this produces an error when usermod is run.
'/usr/sbin/usermod -G support,ops,dev jrhett' returned 6: usermod: unknown group dev
usermod: unknown group dev

So I tried to get smarter, and put logic to add the group to each member under the appropriate class
Class users::dev inherits users {
User['jrhett'] { groups +> ['dev'] }
}

This works… almost. It works for all instances where the user is only subclassed once. But if I do the same technique in multiple classes I get

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Parameter 'groups' is already set on User_and_key[jrhett] by #<Puppet::Resource::Type:0x7f4feed2d828> at /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at /etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com

So how can this be achieved, short of using an exec with an unless doing another exec to determine if the group exists?

--
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Denmat at Jul 12, 2012 at 7:52 am
    Puppet users and groups are fiddly. My current not implemented thinking is to use ldap and manage pam_groups via puppet on the hosts to get the granularity.

    More thinking out loud than anything else.

    Den
    On 12/07/2012, at 6:03, Jo Rhett wrote:

    I'm fighting with a ticklish issue. We have some groups and users that only belong on some systems. So we made all users virtual and then realize them in classes specific to those system types. This works quite well for the users, but not for the groups. When you specify a user, you have to list all the groups they are in.
    groups => ['support',ops','dev'],

    Obviously some groups aren't realized on all systems, so this produces an error when usermod is run.
    '/usr/sbin/usermod -G support,ops,dev jrhett' returned 6: usermod: unknown group dev
    usermod: unknown group dev

    So I tried to get smarter, and put logic to add the group to each member under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }

    This works… almost. It works for all instances where the user is only subclassed once. But if I do the same technique in multiple classes I get

    err: Could not retrieve catalog from remote server: Error 400 on SERVER: Parameter 'groups' is already set on User_and_key[jrhett] by #<Puppet::Resource::Type:0x7f4feed2d828> at /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at /etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com

    So how can this be achieved, short of using an exec with an unless doing another exec to determine if the group exists?

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Christopher Wood at Jul 12, 2012 at 7:57 am
    I use nss-pam-ldapd and pam_ldap depending on the system, using an ldap filter to allow only certain groups per system. I prefer nss-pam-ldapd.

    nss-pam-ldapd:

    CentOS 6
    Debian 6
    Ubuntu 10.04

    pam_ldap:

    CentOS 5
    FreeBSD 9

    (Solaris is more like pam_ldap in configuration, but fairly unique.)

    The manifests to deal with the above are essentially OS-specific.
    On Thu, Jul 12, 2012 at 05:52:24PM +1000, Denmat wrote:
    Puppet users and groups are fiddly. My current not implemented thinking is
    to use ldap and manage pam_groups via puppet on the hosts to get the
    granularity.
    More thinking out loud than anything else.
    Den

    On 12/07/2012, at 6:03, Jo Rhett wrote:

    I'm fighting with a ticklish issue.  We have some groups and users that
    only belong on some systems. So we made all users virtual and then
    realize them in classes specific to those system types.  This works
    quite well for the users, but not for the groups. When you specify a
    user, you have to list all the groups they are in.
    groups => ['support',ops','dev'],
    Obviously some groups aren't realized on all systems, so this produces
    an error when usermod is run.
    '/usr/sbin/usermod -G support,ops,dev jrhett' returned 6:
    usermod: unknown group dev
    usermod: unknown group dev
    So I tried to get smarter, and put logic to add the group to each member
    under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }
    This works� almost. It works for all instances where the user is only
    subclassed once. But if I do the same technique in multiple classes I
    get
    err: Could not retrieve catalog from remote server: Error 400 on SERVER:
    Parameter 'groups' is already set on User_and_key[jrhett] by
    #<Puppet::Resource::Type:0x7f4feed2d828> at
    /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
    /etc/puppet/modules/users/manifests/dev.pp:27 on node
    [2]s2-d1.company.com
    So how can this be achieved, short of using an exec with an unless doing
    another exec to determine if the group exists?
    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet
    projects.

    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To post to this group, send email to [3]puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    [4]puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    [5]http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    References

    Visible links
    1. mailto:jrhett@netconsonance.com
    2. http://s2-d1.company.com/
    3. mailto:puppet-users@googlegroups.com
    4. mailto:puppet-users+unsubscribe@googlegroups.com
    5. http://groups.google.com/group/puppet-users?hl=en
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 12, 2012 at 6:20 pm
    That's great if you have centralized and co-hosted infrastructure and are willing to accept the dependancy. Given that this is a small need for a small number of users on a very small amount of systems (like 3 out of hundreds) without a centralized backbone between them, implementing LDAP makes little sense.
    On Jul 12, 2012, at 12:52 AM, Denmat wrote:
    Puppet users and groups are fiddly. My current not implemented thinking is to use ldap and manage pam_groups via puppet on the hosts to get the granularity.

    More thinking out loud than anything else.

    Den
    On 12/07/2012, at 6:03, Jo Rhett wrote:

    I'm fighting with a ticklish issue. We have some groups and users that only belong on some systems. So we made all users virtual and then realize them in classes specific to those system types. This works quite well for the users, but not for the groups. When you specify a user, you have to list all the groups they are in.
    groups => ['support',ops','dev'],

    Obviously some groups aren't realized on all systems, so this produces an error when usermod is run.
    '/usr/sbin/usermod -G support,ops,dev jrhett' returned 6: usermod: unknown group dev
    usermod: unknown group dev

    So I tried to get smarter, and put logic to add the group to each member under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }

    This works… almost. It works for all instances where the user is only subclassed once. But if I do the same technique in multiple classes I get

    err: Could not retrieve catalog from remote server: Error 400 on SERVER: Parameter 'groups' is already set on User_and_key[jrhett] by #<Puppet::Resource::Type:0x7f4feed2d828> at /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at /etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com

    So how can this be achieved, short of using an exec with an unless doing another exec to determine if the group exists?

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.




    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 12, 2012 at 11:30 am
    Hi,
    On 07/11/2012 10:03 PM, Jo Rhett wrote:
    So I tried to get smarter, and put logic to add the group to each member
    under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }

    This works… almost. It works for all instances where the user is only
    subclassed once. But if I do the same technique in multiple classes I get
    sound approach, but I've hit this wall a couple of times as well.

    I've resorted to horrors that would add items to array variables that
    are declared in a central, well-known class, and use the final value for
    the resources in question. Depending on how much flexibility is
    required, this may not be feasible at all.

    Perhaps hiera can be used to do something clever here?

    Cheers,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 12, 2012 at 6:22 pm

    On Jul 12, 2012, at 4:30 AM, Felix Frank wrote:
    On 07/11/2012 10:03 PM, Jo Rhett wrote:
    So I tried to get smarter, and put logic to add the group to each member
    under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }

    This works… almost. It works for all instances where the user is only
    subclassed once. But if I do the same technique in multiple classes I get
    sound approach, but I've hit this wall a couple of times as well.

    I've resorted to horrors that would add items to array variables that
    are declared in a central, well-known class, and use the final value for
    the resources in question. Depending on how much flexibility is
    required, this may not be feasible at all.
    Hm. That might work, but seems even uglier :(
    Perhaps hiera can be used to do something clever here?

    This is actually something that hiera seems perfect for, but we simply don't have any backend dataset from which to derive hiera data at this time. That is going to change, and I'm looking forward to having hiera access at that point.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 12, 2012 at 1:46 pm

    On Wednesday, July 11, 2012 3:03:14 PM UTC-5, Jo wrote:
    I'm fighting with a ticklish issue. We have some groups and users that
    only belong on some systems. So we made all users virtual and then realize
    them in classes specific to those system types. This works quite well for
    the users, but not for the groups. When you specify a user, you have to
    list all the groups they are in.
    groups => ['support',ops','dev'],

    Obviously some groups aren't realized on all systems, so this produces an
    error when usermod is run.
    '/usr/sbin/usermod -G support,ops,dev jrhett' returned 6: usermod: unknown
    group dev
    usermod: unknown group dev

    So I tried to get smarter, and put logic to add the group to each member
    under the appropriate class
    Class users::dev inherits users {
    User['jrhett'] { groups +> ['dev'] }
    }

    This works… almost. It works for all instances where the user is only
    subclassed once. But if I do the same technique in multiple classes I get

    err: Could not retrieve catalog from remote server: Error 400 on SERVER:
    Parameter 'groups' is already set on User_and_key[jrhett] by
    #<Puppet::Resource::Type:0x7f4feed2d828> at
    /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
    /etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com

    So how can this be achieved, short of using an exec with an unless doing
    another exec to determine if the group exists?
    If it is the case that each user always has the same potential secondary
    groups, and you need to narrow the actual secondary groups to those that
    are actually present, then I think you could do it without too much pain.
    The main ingredients would be a list (array) of the groups that are
    supposed to be present, and a custom function that forms the intersection
    of two arrays. (Or you could use an inline template and split(), but yuck!)

    Hiera would probably provide a good means for building the list of
    available groups, which you could then use not only to filter user
    definitions but also to drive virtual group realization. Here's a skeleton
    of how it might work:

    class auth::constants {
    $available_groups = hiera('groups')
    }

    class auth::groups::virtual {
    # Virtual group declarations, such as
    @group { 'dev':
    gid => 4242,
    ensure => present
    }
    }

    define auth::concrete_group () {
    include 'auth::groups::virtual'
    realize Group[$name]
    }

    class auth::groups {
    include 'auth::constants'

    auth::concrete_group { $auth::constants::available_groups: }
    }

    class auth::users::virtual {
    include 'auth::constants'

    # Virtual user declarations, such as
    @user { 'jbolling':
    uid => 4200,
    gid => 4200,
    groups => intersect(['dev', 'support', 'ops'],
    $auth::constants::available_groups)
    }
    }

    A few bits are omitted, most notably user realization. The main concept is
    to declare what you want in the first place, rather than throwing up
    something and trying to tweak it afterward, or trying to build values
    incrementally. The latter two approaches tends to work poorly in Puppet
    (with certain caveats).

    Note also that the above is completely hypothetical. I think it would
    work, but it's not based on anything I have actually implemented.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/uo9sWOQTJyMJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 12, 2012 at 6:42 pm

    On Jul 12, 2012, at 6:46 AM, jcbollinger wrote:
    If it is the case that each user always has the same potential secondary groups, and you need to narrow the actual secondary groups to those that are actually present, then I think you could do it without too much pain. The main ingredients would be a list (array) of the groups that are supposed to be present, and a custom function that forms the intersection of two arrays. (Or you could use an inline template and split(), but yuck!)

    Hiera would probably provide a good means for building the list of available groups, which you could then use not only to filter user definitions but also to drive virtual group realization. Here's a skeleton of how it might work:

    class auth::constants {
    $available_groups = hiera('groups')
    }
    Interesting idea, but depends on an external datasource that tells us which groups are valid. Since all of these groups are already defined in puppet, I simply don't see the value of managing intersections of data between a hiera data source and puppet.
    # Virtual user declarations, such as
    @user { 'jbolling':
    uid => 4200,
    gid => 4200,
    groups => intersect(['dev', 'support', 'ops'], $auth::constants::available_groups)
    }
    }

    I think the intersect idea is valid, as long as I can find out if a parameter is realized or not. Basically, write a function that removes from the array any group which isn't realized. This removes any need for heira. However I'm poking around and the docs don't show any methods to determine if something has been realized or not.

    If I am reading this right, intersect is provided by stdlib, right? So I really just need to write a function to determine if something is realized or not. I suspect this is going to fall back to the same issues as defined() unless I can delay execution until the end.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 12, 2012 at 9:26 pm

    On Thursday, July 12, 2012 1:42:28 PM UTC-5, Jo wrote:
    On Jul 12, 2012, at 6:46 AM, jcbollinger wrote:

    If it is the case that each user always has the same potential secondary
    groups, and you need to narrow the actual secondary groups to those that
    are actually present, then I think you could do it without too much pain.
    The main ingredients would be a list (array) of the groups that are
    supposed to be present, and a custom function that forms the intersection
    of two arrays. (Or you could use an inline template and split(), but yuck!)

    Hiera would probably provide a good means for building the list of
    available groups, which you could then use not only to filter user
    definitions but also to drive virtual group realization. Here's a skeleton
    of how it might work:

    class auth::constants {
    $available_groups = hiera('groups')
    }


    Interesting idea, but depends on an external datasource that tells us
    which groups are valid. Since all of these groups are already defined in
    puppet, I simply don't see the value of managing intersections of data
    between a hiera data source and puppet.
    No, it doesn't depend on an external datasource; rather, It depends on
    up-front knowledge of which groups are supposed to be realized for the
    node. Although I proposed using an external datasource to provide that
    data, it could just as well be provided by an ENC or determined via DSL
    code based on conditionals, node facts, etc. Even class parameters.


    # Virtual user declarations, such as
    @user { 'jbolling':
    uid => 4200,
    gid => 4200,
    groups => intersect(['dev', 'support', 'ops'],
    $auth::constants::available_groups)
    }
    }


    I think the intersect idea is valid, as long as I can find out if a
    parameter is realized or not. Basically, write a function that removes
    from the array any group which isn't realized. This removes any need for
    heira. However I'm poking around and the docs don't show any methods to
    determine if something has been realized or not.

    If I am reading this right, intersect is provided by stdlib, right?
    If so, then I'm somehow overlooking it. My suggestion and expectation was
    that you would create it yourself, but it seems sufficiently
    general-purpose that you might find something suitable already made. You
    can also, of course, jerry-rig something based on inline_template().

    So I really just need to write a function to determine if something is
    realized or not. I suspect this is going to fall back to the same issues as
    defined() unless I can delay execution until the end.
    I would avoid that variation on this approach if at all possible. You
    would sidestep multiple pitfalls if you could determine up front, based on
    node name and facts, which groups are *supposed* to be present, instead of
    attempting to determine after the fact which were realized. Indeed, you
    might even find it convenient to use that information to drive group
    realization. If nothing else, doing so would ensure that users aren't
    assigned to secondary groups that don't get realized.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/tO-mgaYJ7-sJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 12, 2012 at 11:04 pm

    On Jul 12, 2012, at 2:26 PM, jcbollinger wrote:
    I would avoid that variation on this approach if at all possible. You would sidestep multiple pitfalls if you could determine up front, based on node name and facts, which groups are supposed to be present, instead of attempting to determine after the fact which were realized. Indeed, you might even find it convenient to use that information to drive group realization.
    If nothing else, doing so would ensure that users aren't assigned to secondary groups that don't get realized.
    This is what policy as expressed in the puppet manifests does. I don't see how to avoid the unrealized problem here.

    What's funny is that you are expressing exactly what puppet does today, but it appears you are suggesting that I need to create another data source and mirror the information out of puppet manifests into that for comparison purposes. Huh?

    I'm a bit baffled by the fairly constant suggestion by people here that I keep spreading out the places where information is stored. The point is to centralize the data, not provide more sources to grow inconsistent with each other.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 13, 2012 at 3:02 pm

    On Thursday, July 12, 2012 6:04:17 PM UTC-5, Jo wrote:
    On Jul 12, 2012, at 2:26 PM, jcbollinger wrote:

    I would avoid that variation on this approach if at all possible. You
    would sidestep multiple pitfalls if you could determine up front, based on
    node name and facts, which groups are *supposed* to be present, instead
    of attempting to determine after the fact which were realized. Indeed, you
    might even find it convenient to use that information to drive group
    realization.

    If nothing else, doing so would ensure that users aren't assigned to
    secondary groups that don't get realized.


    This is what policy as expressed in the puppet manifests does. I don't see
    how to avoid the unrealized problem here.

    What's funny is that you are expressing exactly what puppet does today,
    but it appears you are suggesting that I need to create another data source
    and mirror the information out of puppet manifests into that for comparison
    purposes. Huh?

    I'm a bit baffled by the fairly constant suggestion by people here that I
    keep spreading out the places where information is stored. The point is to
    centralize the data, not provide more sources to grow inconsistent with
    each other.
    Relying on a single source of information is *exactly* what what I have
    suggested you do, specifically by using an up-front group list both to
    filter users' declared secondary groups and to drive which groups get
    realized. I have described that three times now, and it's included in the
    example code I posted earlier. You can populate such a list by whatever
    means you want and from whatever source you want, and you can store it
    wherever you want, so long as you produce the entire list before any part
    of it is needed.

    So no, I'm not suggesting you mirror information from your puppet
    manifests. Rather, I am suggesting that you *move* implicit information
    out of your manifests to someplace more accessible. Study my example if
    you still don't understand what I mean by that. The "someplace" where the
    information lands could be an explicit expression elsewhere in your
    manifests, or it could be external, as seems best to you. The information
    implicitly encoded in the structure of your manifests and/or developed
    during compilation is inherently difficult to use from within the manifests
    themselves, and if you insist on using it anyway then you're choosing to be
    stuck in an uncomfortable position.

    More generally, people recommending various possible data sources to you --
    hiera, ENC, etc. -- are not implying that you should "spread out" your
    data. That's a function of your own manifest designs and how you use the
    data. You do a disservice to those volunteering their help to you by
    criticizing them for deficiencies in *your imagined applications* of their
    suggestions.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ldsUjX6CCy0J.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 13, 2012 at 3:17 pm

    On 07/13/2012 05:02 PM, jcbollinger wrote:
    More generally, people recommending various possible data sources to you
    -- hiera, ENC, etc. -- are not implying that you should "spread out"
    your data. That's a function of your own manifest designs and how you
    use the data. You do a disservice to those volunteering their help to
    you by criticizing them for deficiencies in /your imagined applications/
    of their suggestions.
    Though he did put it quite bluntly, I do believe that Jo has a point.

    The thing is, I generally want my manifests to be clever about some
    things. When I include my mysql development class, I may want to realize
    a couple of users and groups as a result (hypothetically speaking - I
    have no such class nor such requirements, but there are other things in
    this vein).

    I would tell hiera to have puppet include the mysql development class,
    not each single user and group. That would strike me as silly.

    So there *is* value in constructing information inside the manifest from
    outside information, or at least that is my firm belief.

    Configuring roles and other settings of larger granularity is what I
    want to do in hiera. Picking single user accounts - probably not so much.

    Cheers,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 13, 2012 at 4:38 pm

    On Friday, July 13, 2012 10:17:25 AM UTC-5, Felix.Frank wrote:
    On 07/13/2012 05:02 PM, jcbollinger wrote:
    More generally, people recommending various possible data sources to you
    -- hiera, ENC, etc. -- are not implying that you should "spread out"
    your data. That's a function of your own manifest designs and how you
    use the data. You do a disservice to those volunteering their help to
    you by criticizing them for deficiencies in /your imagined applications/
    of their suggestions.
    Though he did put it quite bluntly, I do believe that Jo has a point.
    Do you mean that it would be useful to have a reliable way for manifests to
    extract information about declarations appearing some unspecified
    elsewhere? I don't dispute that, but the fact is that Puppet does not have
    such a mechanism, and I don't see any likelihood that it will have one
    soon. The point is that that limitation does not create a need for data
    duplication, despite Jo's assertions to the contrary.

    The thing is, I generally want my manifests to be clever about some
    things. When I include my mysql development class, I may want to realize
    a couple of users and groups as a result (hypothetically speaking - I
    have no such class nor such requirements, but there are other things in
    this vein).
    I don't mean to suggest that Puppet already does the best it is possible to
    do in this area. Indeed, my comments to Jo are entirely about working
    within Puppet's current constraints, not about wishlist features.

    Nevertheless, the essential problem is a hard one: to determine what
    declarations satisfying certain criteria will have been made by the end of
    catalog compilation. That's the underlying problem with using defined(),
    using a hypothetical realized() function, building values cooperatively,
    and perhaps other potentially useful things. It would be convenient to be
    able to do those safely and reliably, but solving the key problem would
    likely require a fundamental change in the manifest compiler's architecture.

    I would tell hiera to have puppet include the mysql development class,
    not each single user and group. That would strike me as silly.
    Sure, but I'm not seeing how that relates. A more parallel situation would
    be if in addition, some unrelated class wanted to be able to determine
    which users and groups the mysql development class had declared. As Puppet
    now stands, the best way would be for the mysql development class to
    provide that data in class variables, or else to have obtained it from some
    shared source in the first place. The point is that neither of those
    options requires that data to be duplicated in the structure of the class.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/OvZbRRPlwpMJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 16, 2012 at 7:45 am
    John,
    On 07/13/2012 06:38 PM, jcbollinger wrote:

    I would tell hiera to have puppet include the mysql development class,
    not each single user and group. That would strike me as silly.


    Sure, but I'm not seeing how that relates. A more parallel situation
    would be if in addition, some unrelated class wanted to be able to
    determine which users and groups the mysql development class had
    declared. As Puppet now stands, the best way would be for the mysql
    development class to provide that data in class variables, or else to
    have obtained it from some shared source in the first place. The point
    is that neither of those options requires that data to be duplicated in
    the structure of the class.
    I'm not sure I concur with your conclusion, nor with what you boil the
    problem at hand down to.

    Puppet *does* have means to implement business logic that adds resources
    as implications of roles or aspects, and that is by the singleton nature
    of classes. You can add users via implication of different possible
    aspects of nodes by having the "aspect" class include the approprate
    user class(es) for example.

    Puppet even allows to make adjustments to such implied resources by
    means of subclasses. If a node has a very certain role, the respective
    class will include a subclass of some other feature to specialize its
    resources appropriately.

    All this is according to what puppet has so far been designed to do.

    The problem at hand can be handled to a degree with these language
    features, as the OP has successfully done already. The approach has
    scaling issues, so a workaround is currently needed. I don't see why the
    approach should be discarded as "not currently implemented" out of hand,
    seeing as the better part of it *is* in fact possible.

    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 16, 2012 at 2:14 pm
    Felix,
    On Monday, July 16, 2012 2:44:58 AM UTC-5, Felix.Frank wrote:

    John,
    On 07/13/2012 06:38 PM, jcbollinger wrote:

    I would tell hiera to have puppet include the mysql development class,
    not each single user and group. That would strike me as silly.


    Sure, but I'm not seeing how that relates. A more parallel situation
    would be if in addition, some unrelated class wanted to be able to
    determine which users and groups the mysql development class had
    declared. As Puppet now stands, the best way would be for the mysql
    development class to provide that data in class variables, or else to
    have obtained it from some shared source in the first place. The point
    is that neither of those options requires that data to be duplicated in
    the structure of the class.
    I'm not sure I concur with your conclusion, nor with what you boil the
    problem at hand down to.
    I didn't define the problem, Jo did. He defined at least two different
    problems, in fact, so perhaps that's where you and I have gotten out of
    sync. The first problem was to build the value of a resource property
    based on declarations in multiple unrelated classes. Jo then went on to
    object to some proposed alternative solutions on the basis that they
    introduced duplication of data already implicit in his class and resource
    declarations, making avoiding such duplication the second problem.

    Puppet *does* have means to implement business logic that adds resources
    as implications of roles or aspects, and that is by the singleton nature
    of classes. You can add users via implication of different possible
    aspects of nodes by having the "aspect" class include the approprate
    user class(es) for example.

    Puppet even allows to make adjustments to such implied resources by
    means of subclasses. If a node has a very certain role, the respective
    class will include a subclass of some other feature to specialize its
    resources appropriately.

    All this is according to what puppet has so far been designed to do.

    The problem at hand can be handled to a degree with these language
    features, as the OP has successfully done already. The approach has
    scaling issues, so a workaround is currently needed. I don't see why the
    approach should be discarded as "not currently implemented" out of hand,
    seeing as the better part of it *is* in fact possible.
    The fact that the approach doesn't work isn't sufficient? Until now, my
    focus has been on something that Jo could actually use, rather than on
    advocacy for new Puppet features. If you have a good way to make that
    approach work for Jo, with current Puppet, then I would be genuinely
    interested in hearing it. I suppose Jo would be even more so.

    I'm not suggesting that Puppet shouldn't have a feature that allowed Jo to
    do what he wants, approximately how he wants, without parse-order
    dependencies or other serious drawbacks. I would love it if there were
    such a feature. In fact, I think it would serve a lot of people well, and
    allow clearer, simpler manifests.

    That feature isn't available today, as far as I know, but I have an idea
    how it might look. In fact, I had it about six months ago. Remember
    resource constraints
    (https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?
    The idea arose as an approach to cross-module dependencies, but it would
    address this problem as well. In fact, this problem is pretty similar to a
    cross-module dependency issue, and might even be one.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/2A8pqYpNEeoJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 16, 2012 at 3:42 pm

    On 07/16/2012 04:14 PM, jcbollinger wrote:
    The fact that the approach doesn't work isn't sufficient? Until now, my
    focus has been on something that Jo could actually use, rather than on
    advocacy for new Puppet features. If you have a good way to make that
    approach work for Jo, with current Puppet, then I would be genuinely
    interested in hearing it. I suppose Jo would be even more so.
    I cannot, of course, but I do sympathize with Jo's notion that in order
    to solve the apparently small problem of making resource overrides
    scale, he is now required to rework most if not all of his manifests to
    play with a hiera based approach.
    I'm not saying there is a simpler solution right now. And we've already
    agreed there should be:
    I'm not suggesting that Puppet shouldn't have a feature that allowed Jo
    to do what he wants, approximately how he wants, without parse-order
    dependencies or other serious drawbacks. I would love it if there were
    such a feature. In fact, I think it would serve a lot of people well,
    and allow clearer, simpler manifests.

    That feature isn't available today, as far as I know, but I have an idea
    how it might look. In fact, I had it about six months ago. Remember
    resource constraints
    (https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?
    The idea arose as an approach to cross-module dependencies, but it would
    address this problem as well. In fact, this problem is pretty similar
    to a cross-module dependency issue, and might even be one.
    I do. Sometimes even wondered if that made it to some pin-board down at
    puppetlabs ;-)

    But I'm not sure they're the best solution for the problem at hand. My
    understanding is that you would have each class declare constraints on
    given users' group memberships, e.g. user jo needs to be in the devs group.
    The only real advantage I see here is that, yes, you can make the
    compilation fail if your hiera data is not consistent with your business
    logic. That doesn't really simplify the way to get to the desired catalog.

    Am I missing a wholly different implied intention of your's here?

    Cheers,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 16, 2012 at 6:19 pm

    On Jul 16, 2012, at 8:42 AM, Felix Frank wrote:
    I cannot, of course, but I do sympathize with Jo's notion that in order
    to solve the apparently small problem of making resource overrides
    scale, he is now required to rework most if not all of his manifests to
    play with a hiera based approach.

    Well, more matter of factly, that shifting to a hiera-based approach would require us to manage very carefully the balance of data between puppet and hiera, and manage by eyeball the dependencies between the two. There is considerable resistance to this idea here.

    If it was possible to put all the user and group information in hiera and then put the assignment/management of that information into puppet then we could probably manage that. But having to edit "this host gets the sql server info" in puppet and then "these users get put in mysql group on this host" in hiera is completely nonfunctional, and I've seen no examples of ways to bridge that gap.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Christopher Wood at Jul 16, 2012 at 6:56 pm
    (inline)
    On Mon, Jul 16, 2012 at 11:19:02AM -0700, Jo Rhett wrote:
    On Jul 16, 2012, at 8:42 AM, Felix Frank wrote:

    I cannot, of course, but I do sympathize with Jo's notion that in order
    to solve the apparently small problem of making resource overrides
    scale, he is now required to rework most if not all of his manifests to
    play with a hiera based approach.

    Well, more matter of factly, that shifting to a hiera-based approach would
    require us to manage very carefully the balance of data between puppet and
    hiera, and manage by eyeball the dependencies between the two. There is
    considerable resistance to this idea here.
    If it was possible to put all the user and group information in hiera and
    then put the assignment/management of that information into puppet then we
    could probably manage that. But having to edit "this host gets the sql
    server info" in puppet and then "these users get put in mysql group on
    this host" in hiera is completely nonfunctional, and I've seen no examples
    of ways to bridge that gap.
    Possibly something like the following pseudocode example? The main point being to only include a puppet class if there's a certain piece of data in hiera.

    node default {
    if hiera('usemysql') {
    include mysql::service
    }
    if hiera_array('users') {
    include users
    }
    }

    (I haven't tested the above myself. We're still not using hiera at work, more's the pity.)
    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet
    projects.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 16, 2012 at 8:29 pm

    On Jul 16, 2012, at 11:55 AM, Christopher Wood wrote:
    Possibly something like the following pseudocode example? The main point being to only include a puppet class if there's a certain piece of data in hiera.

    node default {
    if hiera('usemysql') {
    include mysql::service
    }
    if hiera_array('users') {
    include users
    }
    }
    I'm not sure how this would work. So you're now talking about putting all the if/then logic inside hiera?
    (I haven't tested the above myself. We're still not using hiera at work, more's the pity.)

    We aren't, because we have no external datasource for this stuff and every example we've seen (like yours above) indicates that we're going to have to put half of the logic engine of puppet inside the data source, which means it needs to be a very complex thing that enforces the structure and somehow ties it with the puppet logic. Our analysis so far is that to implement hiera we're going to have to write our own software platform which manages hiera data and writes out puppet policies on the fly when the data changes.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 17, 2012 at 8:02 am

    On 07/16/2012 10:29 PM, Jo Rhett wrote:
    We aren't, because we have no external datasource for this stuff and
    every example we've seen (like yours above) indicates that we're going
    to have to put half of the logic engine of puppet inside the data
    source, which means it needs to be a very complex thing that enforces
    the structure and somehow ties it with the puppet logic. Our analysis so
    far is that to implement hiera we're going to have to write our own
    software platform which manages hiera data and writes out puppet
    policies on the fly when the data changes.
    Not quite. I believe that the canonical approach is to move your node ->
    roles relation into hiera. This way you need little "individual"
    manifest code per node.
    You certainly need a means to manage hiera's datastores, but I don't
    think generating manifests is required.

    Anyway, as participants in this thread seem to agree, the approach is
    limited insofar that puppet has some shortcomings in the area of
    constructing complex derived data (e.g. groups for given users) from
    complex and disjoint pieces of input data.

    Since you're going to end up with some sort of kludge anyway (imho), I
    believe it's perfectly fine for you to forego the hiera backend at this
    time and apply the kludge inside your comfort zone, for what it's worth.

    Cheers,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 17, 2012 at 5:32 pm

    On Jul 17, 2012, at 12:54 AM, Felix Frank wrote:
    On 07/16/2012 10:29 PM, Jo Rhett wrote:
    We aren't, because we have no external datasource for this stuff and
    every example we've seen (like yours above) indicates that we're going
    to have to put half of the logic engine of puppet inside the data
    source, which means it needs to be a very complex thing that enforces
    the structure and somehow ties it with the puppet logic. Our analysis so
    far is that to implement hiera we're going to have to write our own
    software platform which manages hiera data and writes out puppet
    policies on the fly when the data changes.
    Not quite. I believe that the canonical approach is to move your node ->
    roles relation into hiera. This way you need little "individual"
    manifest code per node.
    You certainly need a means to manage hiera's datastores, but I don't
    think generating manifests is required.

    Is this not the epitome of diverse and redundant dependancies? I can't edit my hiera data without evaluating puppet manifests, I can't edit the puppet manifests without editing the hiera data…

    In short, I'll look at hiera as soon as I have time to build out a whole new infrastructure for data management. And trust me, free time is something I have lots of. (not)

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 17, 2012 at 1:30 pm

    On Monday, July 16, 2012 10:42:11 AM UTC-5, Felix.Frank wrote:
    On 07/16/2012 04:14 PM, jcbollinger wrote:
    That feature isn't available today, as far as I know, but I have an idea
    how it might look. In fact, I had it about six months ago. Remember
    resource constraints
    (
    https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?
    The idea arose as an approach to cross-module dependencies, but it would
    address this problem as well. In fact, this problem is pretty similar
    to a cross-module dependency issue, and might even be one.
    I do. Sometimes even wondered if that made it to some pin-board down at
    puppetlabs ;-)

    But I'm not sure they're the best solution for the problem at hand. My
    understanding is that you would have each class declare constraints on
    given users' group memberships, e.g. user jo needs to be in the devs
    group.
    The only real advantage I see here is that, yes, you can make the
    compilation fail if your hiera data is not consistent with your business
    logic. That doesn't really simplify the way to get to the desired catalog.

    Am I missing a wholly different implied intention of your's here?
    Apparently so. I don't want to drag this thread off into a rehash of the
    constraints idea, but one of the central ideas is that it allows
    cooperative specification of resource properties. Constraints -- as I
    envision them -- are not a dynamic validation feature, but rather an
    indirect, deferred declaration feature. In many cases, explicit resource
    declarations could be replaced by one or more constraints on the same
    resource, which could appear anywhere in the manifest set. Everything gets
    resolved after all resources are compiled.

    I'll say no more about that here, but if anyone wants to discuss it further
    then I'd be likely to respond to a new thread on that topic.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Q_fD1qLRUHwJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 17, 2012 at 5:35 pm

    On Jul 17, 2012, at 6:30 AM, jcbollinger wrote:
    Apparently so. I don't want to drag this thread off into a rehash of the constraints idea, but one of the central ideas is that it allows cooperative specification of resource properties. Constraints -- as I envision them -- are not a dynamic validation feature, but rather an indirect, deferred declaration feature. In many cases, explicit resource declarations could be replaced by one or more constraints on the same resource, which could appear anywhere in the manifest set. Everything gets resolved after all resources are compiled.
    Sounds like treating hiera data as virtualized to me (and sounds like a functional way to deal with the issues we are discussing). How would you implement this today?
    I'll say no more about that here, but if anyone wants to discuss it further then I'd be likely to respond to a new thread on that topic.
    Seems like a thread that you should name, unless you want a thread labelled "Ask John" … ;-)

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 17, 2012 at 7:43 pm

    On Tuesday, July 17, 2012 12:35:34 PM UTC-5, Jo wrote:
    On Jul 17, 2012, at 6:30 AM, jcbollinger wrote:

    Apparently so. I don't want to drag this thread off into a rehash of the
    constraints idea, but one of the central ideas is that it allows
    cooperative specification of resource properties. Constraints -- as I
    envision them -- are not a dynamic validation feature, but rather an
    indirect, deferred declaration feature. In many cases, explicit resource
    declarations could be replaced by one or more constraints on the same
    resource, which could appear anywhere in the manifest set. Everything gets
    resolved after all resources are compiled.


    Sounds like treating hiera data as virtualized to me (and sounds like a
    functional way to deal with the issues we are discussing). How would you
    implement this today?
    And now I'm breaking my promise, but it *is* your thread...

    The resource constraints idea has no direct relationship to hiera. There's
    no reason why the two shouldn't interoperate, but neither depends on the
    other.

    I am not at all sure there is a clean way to build it on the (intended)
    extension points available today, however. As I conceive it, it involves a
    catalog tweaking step between initial compilation and delivery to the
    agent, and as such it would require modifications to the Puppet core.
    Perhaps one could approximate it, however. The most likely way I can think
    of to do that involves custom functions.

    At least two functions would be needed: one to cache data somewhere in the
    master (invoked potentially many times) and one to read back and apply that
    data (probably called fewer times, and very late in the compilation -- such
    as in a final run stage). I apologize for how abstract and vague that
    description is, but there is a great deal more design effort needed than I
    am prepared to exert at the moment.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/TQBINg-GU4MJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 17, 2012 at 7:59 pm

    On Jul 17, 2012, at 12:42 PM, jcbollinger wrote:
    I apologize for how abstract and vague that description is, but there is a great deal more design effort needed than I am prepared to exert at the moment.

    No worries. It confirms what is necessary for me right now which is that it's not something I should be testing anytime soon ;-)

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 18, 2012 at 8:20 am

    On 07/17/2012 07:31 PM, Jo Rhett wrote:
    Is this not the epitome of diverse and redundant dependancies? I can't
    edit my hiera data without evaluating puppet manifests, I can't edit the
    puppet manifests without editing the hiera data…
    Hmm, by "managing your hiera data" I wasn't implying that puppet should
    manage them. I don't think that can work.

    Rather, if you're not intending to scribble YAML files by hand (which is
    entirely possible), you would have to write a web frontend or similar.
    On 07/17/2012 09:42 PM, jcbollinger wrote:
    At least two functions would be needed: one to cache data somewhere in
    the master (invoked potentially many times) and one to read back and
    apply that data (probably called fewer times, and very late in the
    compilation -- such as in a final run stage). I apologize for how
    abstract and vague that description is, but there is a great deal more
    design effort needed than I am prepared to exert at the moment.
    This seems reminiscent of the "futures" idea that puppetlabs bounced
    around a year back or so. Implementing such things is really a lot
    harder than it sounds, because you need to make sure that your compiler
    does not take x^n (x raised to the power of n) loops, where n is e.g.
    the number of resources in your manifest.

    I may be overthinking this, though. Also this branch of discussion
    probably belongs on the dev list. Kudos for thinking this up in the
    first place, anyhow.

    Cheers,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 18, 2012 at 6:59 pm

    On 07/17/2012 07:31 PM, Jo Rhett wrote:
    Is this not the epitome of diverse and redundant dependancies? I can't
    edit my hiera data without evaluating puppet manifests, I can't edit the
    puppet manifests without editing the hiera data…
    On Jul 18, 2012, at 1:19 AM, Felix Frank wrote:
    Rather, if you're not intending to scribble YAML files by hand (which is
    entirely possible), you would have to write a web frontend or similar.

    Understood. The problem is that every example I've seen to date you have to know the puppet module code well to edit the data. The value of a front-end would be to allow users other than ruby coders to manage the data. I haven't seen any examples with enough differentiation for that.

    Ultimately I guess you build an entire ecosystem where you tie the front-end data management code to the puppet manifests in git and update both at the same time--but that's one hell of an infrastructure creation that I don't have enough free time for. In short, I believe that hiera is only useful for companies who already have a large information schema from which to draw their data, and you are only editing the hiera adapters to get the data as you update the puppet manifests. Anyone else has a huge project to get to "useful".

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jo Rhett at Jul 13, 2012 at 6:02 pm

    On Jul 13, 2012, at 8:02 AM, jcbollinger wrote:
    Relying on a single source of information is exactly what what I have suggested you do, specifically by using an up-front group list both to filter users' declared secondary groups and to drive which groups get realized. I have described that three times now, and it's included in the example code I posted earlier. You can populate such a list by whatever means you want and from whatever source you want, and you can store it wherever you want, so long as you produce the entire list before any part of it is needed.
    I did not see that from what you showed. Your example didn't show how to aggregate or use the data at all. I saw six classes that were significantly more complex and appeared to require defining the data in multiple places. There was certainly no obvious way this would reduce my data sources.
    So no, I'm not suggesting you mirror information from your puppet manifests. Rather, I am suggesting that you move implicit information out of your manifests to someplace more accessible. Study my example if you still don't understand what I mean by that. The "someplace" where the information lands could be an explicit expression elsewhere in your manifests, or it could be external, as seems best to you. The information implicitly encoded in the structure of your manifests and/or developed during compilation is inherently difficult to use from within the manifests themselves, and if you insist on using it anyway then you're choosing to be stuck in an uncomfortable position.
    I hear what you are saying, but I really don't see how your example makes this idea clear. I saw multiple sets of classes relying on each other's data in an unreadable manner.

    I would argue that even if it does do what I meant, the very fact that I couldn't read it to understand this ensures nobody else here has a chance at maintaining it. "More complex" is not a desired trait here.

    In general I see ENCs as eventually providing a way to simplify the data input, but that's not what I've seen recommended or demonstrated. The case for ENCs would be made a lot stronger if some good examples of ways to simply via the use of ENCs were posted.
    More generally, people recommending various possible data sources to you -- hiera, ENC, etc. -- are not implying that you should "spread out" your data. That's a function of your own manifest designs and how you use the data. You do a disservice to those volunteering their help to you by criticizing them for deficiencies in your imagined applications of their suggestions.

    I could go back and make a line by line review of every single time people have told me that I should take data from the puppet manifests and reinforce it / control it via data from an ENC. There hasn't been a single situation where someone said what you are suggesting -- hey, pull this all out of puppet and incorporate it this way (x) so that you can get what you want. It has always been how to do half the job in puppet and half the job in something else, and manually manage the dependancies between the two.

    --
    Jo Rhett
    Net Consonance : net philanthropy to improve open source and internet projects.



    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJul 11, '12 at 8:03p
activeJul 18, '12 at 6:59p
posts28
users5
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase