FAQ
Hi all,

I've been improving our sysctl module and come across an interesting
design problem I'd like feedback on.

I approached the re-factor with Hiera in mind - I would put all our
sysctl data in Hiera hash and pull that into a hiera_hash, merging the
hierarchy of data and allowing higher priority sysctl settings to
override the baseline defaults. I then use create_resources to write
sysctl.conf. Works great to start with, but now I come across more and
more cases where the sysctl data is dependent on machine logic (virtual
vs physical, types of hardware, etc) that doesn't seem right to put into
Hiera as I'd have a complex hierarchy for a bunch of edge case Facts.

I seem to need to make decisions on two sources: business logic in Hiera
hierarchy (that's easy with merging hashes) as well as considering what
Facts or Classes applies to a node (machine logic). That's not trivial
to do, especially with a potentially large set of data like sysctl.conf
keys.

Does anyone have any thoughts or tips on how they might be managing a
similar situation?

Thanks,

-Luke

--
Luke Bigum

Information Systems
Ph: +44 (0) 20 3192 2520
luke.bigum@lmax.com | http://www.lmax.com
LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


FX and CFDs are leveraged products that can result in losses exceeding
your deposit. They are not suitable for everyone so please ensure you
fully understand the risks involved. The information in this email is not
directed at residents of the United States of America or any other
jurisdiction where trading in CFDs and/or FX is restricted or prohibited
by local laws or regulations.

The information in this email and any attachment is confidential and is
intended only for the named recipient(s). The email may not be disclosed
or used by any person other than the addressee, nor may it be copied in
any way. If you are not the intended recipient please notify the sender
immediately and delete any copies of this message. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.

LMAX operates a multilateral trading facility. Authorised and regulated
by the Financial Services Authority (firm registration number 509778) and
is registered in England and Wales (number 06505809).
Our registered address is Yellow Building, 1A Nicholas Road, London, W11
4AN.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Gary Larizza at May 11, 2012 at 3:54 pm
    I see this with people looking to move to the hierarchical system that
    Hiera brings. It essentially boils down to "How do I do this without
    having a ton of hierarchy levels?". Usually we tend to recommend using the
    hierarchy to hit the 80% mark for the data you need in your modules.
    Anything that's module-specific-data should then be broken out to a
    data.pp or params.pp file with conditional logic there. I tend to ask
    people: "Is this something others are going to hit when they try to use the
    module too?", as in - "Are there path differences between operating
    systems?", or "Are there important changes to the data between RHEL 5 and
    6?". If the answer to these is yes, then I tend to favor putting that data
    into a module's data class so that it's exposed for ANYONE who wants to use
    the module. Why would you want to hide these differences in the hierarchy
    - especially if others might run into them?

    Does this sound similar to the problems you're facing? Or is this a case
    where you have custom facts that are specific to your organization that
    determine how you manage sysctl?

    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:

    Hi all,

    I've been improving our sysctl module and come across an interesting
    design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our sysctl
    data in Hiera hash and pull that into a hiera_hash, merging the hierarchy
    of data and allowing higher priority sysctl settings to override the
    baseline defaults. I then use create_resources to write sysctl.conf. Works
    great to start with, but now I come across more and more cases where the
    sysctl data is dependent on machine logic (virtual vs physical, types of
    hardware, etc) that doesn't seem right to put into Hiera as I'd have a
    complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera
    hierarchy (that's easy with merging hashes) as well as considering what
    Facts or Classes applies to a node (machine logic). That's not trivial to
    do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a
    similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered
    address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@**
    googlegroups.com <puppet-users%2Bunsubscribe@googlegroups.com>.
    For more options, visit this group at http://groups.google.com/**
    group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en>
    .

    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Luke Bigum at May 11, 2012 at 4:48 pm
    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real
    problem. Hiera's ability to merge hashes together makes it perfect for
    arriving at one set of sysctl options for a server based on "business
    logic" (my hierarchy). For Hiera data on 'someserver' below which has
    'some_role', calling hiera_hash in a Puppet manifest will give me IP
    forwarding set and rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data
    to influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those
    is 'vm.swappiness'. Most of my nodes have a value of '10', but lets say
    hypothetically that I have a small set of nodes that require a different
    value because of the amount of RAM available in the machine (a decision
    needs to be made based on hardware, not business logic). This is purely
    a Fact. Introducing another level of hierarchy for Fact 'memorytotal' is
    a bit silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases
    that modify the data retrieved from Hiera based on Facts. Class
    inheritance won't work because create_resources() seems to insert into
    the catalog in an uninheritable way - bug report or fixable with Ruby
    DSL perhaps? Filling this class full of "if ($fact) modify hash" to
    munge the data pulled from Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke
    On 11/05/12 16:53, Gary Larizza wrote:
    I see this with people looking to move to the hierarchical system that
    Hiera brings. It essentially boils down to "How do I do this without
    having a ton of hierarchy levels?". Usually we tend to recommend
    using the hierarchy to hit the 80% mark for the data you need in your
    modules. Anything that's module-specific-data should then be broken
    out to a data.pp or params.pp file with conditional logic there. I
    tend to ask people: "Is this something others are going to hit when
    they try to use the module too?", as in - "Are there path differences
    between operating systems?", or "Are there important changes to the
    data between RHEL 5 and 6?". If the answer to these is yes, then I
    tend to favor putting that data into a module's data class so that
    it's exposed for ANYONE who wants to use the module. Why would you
    want to hide these differences in the hierarchy - especially if others
    might run into them?

    Does this sound similar to the problems you're facing? Or is this a
    case where you have custom facts that are specific to your
    organization that determine how you manage sysctl?


    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:

    Hi all,

    I've been improving our sysctl module and come across an
    interesting design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all
    our sysctl data in Hiera hash and pull that into a hiera_hash,
    merging the hierarchy of data and allowing higher priority sysctl
    settings to override the baseline defaults. I then use
    create_resources to write sysctl.conf. Works great to start with,
    but now I come across more and more cases where the sysctl data is
    dependent on machine logic (virtual vs physical, types of
    hardware, etc) that doesn't seem right to put into Hiera as I'd
    have a complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in
    Hiera hierarchy (that's easy with merging hashes) as well as
    considering what Facts or Classes applies to a node (machine
    logic). That's not trivial to do, especially with a potentially
    large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be
    managing a similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520 <tel:%2B44%20%280%29%2020%203192%202520>
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this
    email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or
    prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential
    and is
    intended only for the named recipient(s). The email may not be
    disclosed
    or used by any person other than the addressee, nor may it be
    copied in
    any way. If you are not the intended recipient please notify the
    sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and
    regulated by the Financial Services Authority (firm registration
    number 509778) and
    is registered in England and Wales (number 06505809). Our
    registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.




    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN



    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Aaron Grewell at May 11, 2012 at 4:56 pm
    I suppose you could create a separate class for the entries that will
    be fact-driven versus Hiera-driven. You wouldn't be able to use a
    single template, but either augeas or concat should work. I wouldn't
    call it elegant, but the code might be less ugly.
    On Fri, May 11, 2012 at 9:47 AM, Luke Bigum wrote:
    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real problem.
    Hiera's ability to merge hashes together makes it perfect for arriving at
    one set of sysctl options for a server based on "business logic" (my
    hierarchy).  For Hiera data on 'someserver' below which has 'some_role',
    calling hiera_hash in a Puppet manifest will give me IP forwarding set and
    rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data to
    influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those is
    'vm.swappiness'. Most of my nodes have a value of '10', but lets say
    hypothetically that I have a small set of nodes that require a different
    value because of the amount of RAM available in the machine (a decision
    needs to be made based on hardware, not business logic). This is purely a
    Fact. Introducing another level of hierarchy for Fact 'memorytotal' is a bit
    silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases that
    modify the data retrieved from Hiera based on Facts. Class inheritance won't
    work because create_resources() seems to insert into the catalog in an
    uninheritable way - bug report or fixable with Ruby DSL perhaps? Filling
    this class full of "if ($fact) modify hash" to munge the data pulled from
    Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke


    On 11/05/12 16:53, Gary Larizza wrote:

    I see this with people looking to move to the hierarchical system that Hiera
    brings.  It essentially boils down to "How do I do this without having a ton
    of hierarchy levels?".  Usually we tend to recommend using the hierarchy to
    hit the 80% mark for the data you need in your modules.  Anything that's
    module-specific-data should then be broken out to a data.pp or params.pp
    file with conditional logic there.  I tend to ask people: "Is this something
    others are going to hit when they try to use the module too?", as in - "Are
    there path differences between operating systems?", or "Are there important
    changes to the data between RHEL 5 and 6?".  If the answer to these is yes,
    then I tend to favor putting that data into a module's data class so that
    it's exposed for ANYONE who wants to use the module.  Why would you want to
    hide these differences in the hierarchy - especially if others might run
    into them?

    Does this sound similar to the problems you're facing?  Or is this a case
    where you have custom facts that are specific to your organization that
    determine how you manage sysctl?

    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:

    Hi all,

    I've been improving our sysctl module and come across an interesting
    design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our sysctl
    data in Hiera hash and pull that into a hiera_hash, merging the hierarchy of
    data and allowing higher priority sysctl settings to override the baseline
    defaults. I then use create_resources to write sysctl.conf. Works great to
    start with, but now I come across more and more cases where the sysctl data
    is dependent on machine logic (virtual vs physical, types of hardware, etc)
    that doesn't seem right to put into Hiera as I'd have a complex hierarchy
    for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera
    hierarchy (that's easy with merging hashes) as well as considering what
    Facts or Classes applies to a node (machine logic). That's not trivial to
    do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a
    similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit.  They are not suitable for everyone so please ensure you
    fully understand the risks involved.  The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility.  Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered
    address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.



    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Gary Larizza at May 11, 2012 at 5:19 pm
    I see (and I saw your post on create_resources - I tested it out and also
    noticed that inheritance didn't work as expected for me either). My
    initial thought would be to try and break down the individual sysctl
    entries as individual resources that could be declared with a defined type
    (as Aaron mentioned) - then you can use hiera to declare them with
    create_resources() AS WELL as individually declaring additional resources.


    On Fri, May 11, 2012 at 9:47 AM, Luke Bigum wrote:

    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real problem.
    Hiera's ability to merge hashes together makes it perfect for arriving at
    one set of sysctl options for a server based on "business logic" (my
    hierarchy). For Hiera data on 'someserver' below which has 'some_role',
    calling hiera_hash in a Puppet manifest will give me IP forwarding set and
    rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data to
    influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those is
    'vm.swappiness'. Most of my nodes have a value of '10', but lets say
    hypothetically that I have a small set of nodes that require a different
    value because of the amount of RAM available in the machine (a decision
    needs to be made based on hardware, not business logic). This is purely a
    Fact. Introducing another level of hierarchy for Fact 'memorytotal' is a
    bit silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases
    that modify the data retrieved from Hiera based on Facts. Class inheritance
    won't work because create_resources() seems to insert into the catalog in
    an uninheritable way - bug report or fixable with Ruby DSL perhaps? Filling
    this class full of "if ($fact) modify hash" to munge the data pulled from
    Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke


    On 11/05/12 16:53, Gary Larizza wrote:

    I see this with people looking to move to the hierarchical system that
    Hiera brings. It essentially boils down to "How do I do this without
    having a ton of hierarchy levels?". Usually we tend to recommend using the
    hierarchy to hit the 80% mark for the data you need in your modules.
    Anything that's module-specific-data should then be broken out to a
    data.pp or params.pp file with conditional logic there. I tend to ask
    people: "Is this something others are going to hit when they try to use the
    module too?", as in - "Are there path differences between operating
    systems?", or "Are there important changes to the data between RHEL 5 and
    6?". If the answer to these is yes, then I tend to favor putting that data
    into a module's data class so that it's exposed for ANYONE who wants to use
    the module. Why would you want to hide these differences in the hierarchy
    - especially if others might run into them?

    Does this sound similar to the problems you're facing? Or is this a
    case where you have custom facts that are specific to your organization
    that determine how you manage sysctl?

    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:

    Hi all,

    I've been improving our sysctl module and come across an interesting
    design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our
    sysctl data in Hiera hash and pull that into a hiera_hash, merging the
    hierarchy of data and allowing higher priority sysctl settings to override
    the baseline defaults. I then use create_resources to write sysctl.conf.
    Works great to start with, but now I come across more and more cases where
    the sysctl data is dependent on machine logic (virtual vs physical, types
    of hardware, etc) that doesn't seem right to put into Hiera as I'd have a
    complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera
    hierarchy (that's easy with merging hashes) as well as considering what
    Facts or Classes applies to a node (machine logic). That's not trivial to
    do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a
    similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered
    address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.



    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jeff at May 11, 2012 at 5:45 pm
    I have a similar issue with Hiera when decisions depend on more than facts
    alone... thus exluded from the hierarchy.

    In example, the hostname can tell a node is a database server (fact: role)
    but only a custom group will tell it should run Oracle 10g or 11g (product)
    and the proper sysctl to apply specifically. I can't build a fact about
    something that ain't installed yet and I don't want a local file to define
    group membership (giving up control). The node to group membership has to
    be in a CMDB somewhere.

    So far I think using an ENC with Hiera at the same time is the solution to
    cover all cases.... unless you guys know of a better way.

    jean
    On Friday, 11 May 2012 13:19:33 UTC-4, Gary Larizza wrote:

    I see (and I saw your post on create_resources - I tested it out and also
    noticed that inheritance didn't work as expected for me either). My
    initial thought would be to try and break down the individual sysctl
    entries as individual resources that could be declared with a defined type
    (as Aaron mentioned) - then you can use hiera to declare them with
    create_resources() AS WELL as individually declaring additional resources.


    On Fri, May 11, 2012 at 9:47 AM, Luke Bigum wrote:

    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real
    problem. Hiera's ability to merge hashes together makes it perfect for
    arriving at one set of sysctl options for a server based on "business
    logic" (my hierarchy). For Hiera data on 'someserver' below which has
    'some_role', calling hiera_hash in a Puppet manifest will give me IP
    forwarding set and rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data
    to influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those is
    'vm.swappiness'. Most of my nodes have a value of '10', but lets say
    hypothetically that I have a small set of nodes that require a different
    value because of the amount of RAM available in the machine (a decision
    needs to be made based on hardware, not business logic). This is purely a
    Fact. Introducing another level of hierarchy for Fact 'memorytotal' is a
    bit silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases
    that modify the data retrieved from Hiera based on Facts. Class inheritance
    won't work because create_resources() seems to insert into the catalog in
    an uninheritable way - bug report or fixable with Ruby DSL perhaps? Filling
    this class full of "if ($fact) modify hash" to munge the data pulled from
    Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke


    On 11/05/12 16:53, Gary Larizza wrote:

    I see this with people looking to move to the hierarchical system that
    Hiera brings. It essentially boils down to "How do I do this without
    having a ton of hierarchy levels?". Usually we tend to recommend using the
    hierarchy to hit the 80% mark for the data you need in your modules.
    Anything that's module-specific-data should then be broken out to a
    data.pp or params.pp file with conditional logic there. I tend to ask
    people: "Is this something others are going to hit when they try to use the
    module too?", as in - "Are there path differences between operating
    systems?", or "Are there important changes to the data between RHEL 5 and
    6?". If the answer to these is yes, then I tend to favor putting that data
    into a module's data class so that it's exposed for ANYONE who wants to use
    the module. Why would you want to hide these differences in the hierarchy
    - especially if others might run into them?

    Does this sound similar to the problems you're facing? Or is this a
    case where you have custom facts that are specific to your organization
    that determine how you manage sysctl?

    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:

    Hi all,

    I've been improving our sysctl module and come across an interesting
    design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our
    sysctl data in Hiera hash and pull that into a hiera_hash, merging the
    hierarchy of data and allowing higher priority sysctl settings to override
    the baseline defaults. I then use create_resources to write sysctl.conf.
    Works great to start with, but now I come across more and more cases where
    the sysctl data is dependent on machine logic (virtual vs physical, types
    of hardware, etc) that doesn't seem right to put into Hiera as I'd have a
    complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera
    hierarchy (that's easy with merging hashes) as well as considering what
    Facts or Classes applies to a node (machine logic). That's not trivial to
    do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a
    similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is
    not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered
    address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.



    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/0NSu9kWGZggJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Wolf Noble at May 14, 2012 at 6:44 pm
    Hi Luke,

    Would adding fqdn to the bottom of your hierarchy give you the ability to override the values on the outliers that you can't automagically set while retaining elegance in your class? Or am I missing something?



    Wolf Noble
    Datapipe Managed IT Services



    On May 11, 2012, at 12:45 PM, Jeff wrote:

    I have a similar issue with Hiera when decisions depend on more than facts alone... thus exluded from the hierarchy.

    In example, the hostname can tell a node is a database server (fact: role) but only a custom group will tell it should run Oracle 10g or 11g (product) and the proper sysctl to apply specifically. I can't build a fact about something that ain't installed yet and I don't want a local file to define group membership (giving up control). The node to group membership has to be in a CMDB somewhere.

    So far I think using an ENC with Hiera at the same time is the solution to cover all cases.... unless you guys know of a better way.

    jean

    On Friday, 11 May 2012 13:19:33 UTC-4, Gary Larizza wrote:
    I see (and I saw your post on create_resources - I tested it out and also noticed that inheritance didn't work as expected for me either). My initial thought would be to try and break down the individual sysctl entries as individual resources that could be declared with a defined type (as Aaron mentioned) - then you can use hiera to declare them with create_resources() AS WELL as individually declaring additional resources.



    On Fri, May 11, 2012 at 9:47 AM, Luke Bigum wrote:
    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real problem. Hiera's ability to merge hashes together makes it perfect for arriving at one set of sysctl options for a server based on "business logic" (my hierarchy). For Hiera data on 'someserver' below which has 'some_role', calling hiera_hash in a Puppet manifest will give me IP forwarding set and rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data to influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those is 'vm.swappiness'. Most of my nodes have a value of '10', but lets say hypothetically that I have a small set of nodes that require a different value because of the amount of RAM available in the machine (a decision needs to be made based on hardware, not business logic). This is purely a Fact. Introducing another level of hierarchy for Fact 'memorytotal' is a bit silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases that modify the data retrieved from Hiera based on Facts. Class inheritance won't work because create_resources() seems to insert into the catalog in an uninheritable way - bug report or fixable with Ruby DSL perhaps? Filling this class full of "if ($fact) modify hash" to munge the data pulled from Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke


    On 11/05/12 16:53, Gary Larizza wrote:
    I see this with people looking to move to the hierarchical system that Hiera brings. It essentially boils down to "How do I do this without having a ton of hierarchy levels?". Usually we tend to recommend using the hierarchy to hit the 80% mark for the data you need in your modules. Anything that's module-specific-data should then be broken out to a data.pp or params.pp file with conditional logic there. I tend to ask people: "Is this something others are going to hit when they try to use the module too?", as in - "Are there path differences between operating systems?", or "Are there important changes to the data between RHEL 5 and 6?". If the answer to these is yes, then I tend to favor putting that data into a module's data class so that it's exposed for ANYONE who wants to use the module. Why would you want to hide these differences in the hierarchy - especially if others might run into them?

    Does this sound similar to the problems you're facing? Or is this a case where you have custom facts that are specific to your organization that determine how you manage sysctl?


    On Fri, May 11, 2012 at 8:42 AM, Luke Bigum wrote:
    Hi all,

    I've been improving our sysctl module and come across an interesting design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our sysctl data in Hiera hash and pull that into a hiera_hash, merging the hierarchy of data and allowing higher priority sysctl settings to override the baseline defaults. I then use create_resources to write sysctl.conf. Works great to start with, but now I come across more and more cases where the sysctl data is dependent on machine logic (virtual vs physical, types of hardware, etc) that doesn't seem right to put into Hiera as I'd have a complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera hierarchy (that's easy with merging hashes) as well as considering what Facts or Classes applies to a node (machine logic). That's not trivial to do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com<http://www.lmax.com/>
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com .
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com .
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.




    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com .
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com .
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.



    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com<http://www.lmax.com/>
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN

    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com .
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com .
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.



    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/0NSu9kWGZggJ.
    To post to this group, send email to puppet-users@googlegroups.com .
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com .
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


    ________________________________

    This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Luke Bigum at May 15, 2012 at 10:57 am
    Hi Wolf,

    At the top of the hierarchy, yes (the hash merge is done top to bottom,
    where the top is the highest of the hierarchy, see mine below). I have
    this in place already and you're right it will work elegantly, however
    it moves the 'decision' of what to do into Hiera and you have to "know"
    that certain nodes require certain vm.swappiness
    values depending on what hardware they are on:

    ---
    :backends: - json
    :hierarchy: - %{fqdn}
    - %{lmax_role}_role
    - %{lmax_env}_server
    - %{pop}.tradefair
    - common
    :json:
    :datadir: - /etc/puppet/private/
    - /etc/puppet/environments/%{environment}/hiera_data_store/
    - /etc/puppet/environments/%{environment}/rebirth_data_store/
    -
    /etc/puppet/environments/%{environment}/satellite_system_groups/


    Thank you for everyone else who replied too. Re Gary, I already use a
    slightly modified version of puppetlabs-sysctl's custom type and
    provider, so I can define individual sysctl lines as well as with
    create_resources, it's the combination of the two I've got problems
    with. My next crazy idea is to try wrap them all in a class and use
    inline_template to dynamically generate class names - I hope I can
    override classes that are created with create_resources if resources
    cannot be - but that could get very messy very quickly, I'll let you
    know how I go.

    Thanks,

    -Luke
    On 14/05/12 19:44, Wolf Noble wrote:
    Hi Luke,

    Would adding fqdn to the bottom of your hierarchy give you the ability to override the values on the outliers that you can't automagically set while retaining elegance in your class? Or am I missing something?



    Wolf Noble
    Datapipe Managed IT Services



    On May 11, 2012, at 12:45 PM, Jeff wrote:

    I have a similar issue with Hiera when decisions depend on more than facts alone... thus exluded from the hierarchy.

    In example, the hostname can tell a node is a database server (fact: role) but only a custom group will tell it should run Oracle 10g or 11g (product) and the proper sysctl to apply specifically. I can't build a fact about something that ain't installed yet and I don't want a local file to define group membership (giving up control). The node to group membership has to be in a CMDB somewhere.

    So far I think using an ENC with Hiera at the same time is the solution to cover all cases.... unless you guys know of a better way.

    jean

    On Friday, 11 May 2012 13:19:33 UTC-4, Gary Larizza wrote:
    I see (and I saw your post on create_resources - I tested it out and also noticed that inheritance didn't work as expected for me either). My initial thought would be to try and break down the individual sysctl entries as individual resources that could be declared with a defined type (as Aaron mentioned) - then you can use hiera to declare them with create_resources() AS WELL as individually declaring additional resources.



    On Fri, May 11, 2012 at 9:47 AM, Luke Bigumwrote:
    Hi Gary,

    Not quite... Let me go into more detail.

    I'm trying to handle sysctl "perfectly" which is probably my real problem. Hiera's ability to merge hashes together makes it perfect for arriving at one set of sysctl options for a server based on "business logic" (my hierarchy). For Hiera data on 'someserver' below which has 'some_role', calling hiera_hash in a Puppet manifest will give me IP forwarding set and rp_filter set, which is what I want:

    --------- some_role.json ------------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "1"
    }
    }
    --------------------------------------------
    --------- common.json ----------
    {
    "sysctl" : {
    "net.ipv4.ip_forward" : {
    "comment" : "Controls IP packet forwarding",
    "value" : "0"
    },
    "net.ipv4.conf.default.rp_filter" : {
    "comment" : "Controls source route verification",
    "value" : "1"
    }
    ...
    ...
    }
    -------------------------------------

    Where it becomes difficult is trying to then incorporate pure Fact data to influence or modify these decisions.

    Lets say that I actually get back 20 keys of sysctl data, one of those is 'vm.swappiness'. Most of my nodes have a value of '10', but lets say hypothetically that I have a small set of nodes that require a different value because of the amount of RAM available in the machine (a decision needs to be made based on hardware, not business logic). This is purely a Fact. Introducing another level of hierarchy for Fact 'memorytotal' is a bit silly in this case.

    The sysctl class looks roughly like this:

    ------- sysctl.pp ----------
    class sysctl {
    $sysctl_hash = hiera_hash('sysctl')
    create_resources('sysctl', $sysctl_hash)
    }
    ------------------------------

    I love that simplicity, however it's difficult to introduce edge cases that modify the data retrieved from Hiera based on Facts. Class inheritance won't work because create_resources() seems to insert into the catalog in an uninheritable way - bug report or fixable with Ruby DSL perhaps? Filling this class full of "if ($fact) modify hash" to munge the data pulled from Hiera seems dirty too.

    There may be no elegant solution and as you say, 80-90% may have to do.

    -Luke


    On 11/05/12 16:53, Gary Larizza wrote:
    I see this with people looking to move to the hierarchical system that Hiera brings. It essentially boils down to "How do I do this without having a ton of hierarchy levels?". Usually we tend to recommend using the hierarchy to hit the 80% mark for the data you need in your modules. Anything that's module-specific-data should then be broken out to a data.pp or params.pp file with conditional logic there. I tend to ask people: "Is this something others are going to hit when they try to use the module too?", as in - "Are there path differences between operating systems?", or "Are there important changes to the data between RHEL 5 and 6?". If the answer to these is yes, then I tend to favor putting that data into a module's data class so that it's exposed for ANYONE who wants to use the module. Why would you want to hide these differences in the hierarchy - especially if others might run into them?

    Does this sound similar to the problems you're facing? Or is this a case where you have custom facts that are specific to your organization that determine how you manage sysctl?


    On Fri, May 11, 2012 at 8:42 AM, Luke Bigumwrote:
    Hi all,

    I've been improving our sysctl module and come across an interesting design problem I'd like feedback on.

    I approached the re-factor with Hiera in mind - I would put all our sysctl data in Hiera hash and pull that into a hiera_hash, merging the hierarchy of data and allowing higher priority sysctl settings to override the baseline defaults. I then use create_resources to write sysctl.conf. Works great to start with, but now I come across more and more cases where the sysctl data is dependent on machine logic (virtual vs physical, types of hardware, etc) that doesn't seem right to put into Hiera as I'd have a complex hierarchy for a bunch of edge case Facts.

    I seem to need to make decisions on two sources: business logic in Hiera hierarchy (that's easy with merging hashes) as well as considering what Facts or Classes applies to a node (machine logic). That's not trivial to do, especially with a potentially large set of data like sysctl.conf keys.

    Does anyone have any thoughts or tips on how they might be managing a similar situation?

    Thanks,

    -Luke

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com<http://www.lmax.com/>
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809). Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.




    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.



    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com<http://www.lmax.com/>
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN

    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.



    --

    Gary Larizza
    Professional Services Engineer
    Puppet Labs


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/0NSu9kWGZggJ.
    To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


    ________________________________

    This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

    --
    Luke Bigum

    Information Systems
    Ph: +44 (0) 20 3192 2520
    luke.bigum@lmax.com | http://www.lmax.com
    LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN



    FX and CFDs are leveraged products that can result in losses exceeding
    your deposit. They are not suitable for everyone so please ensure you
    fully understand the risks involved. The information in this email is not
    directed at residents of the United States of America or any other
    jurisdiction where trading in CFDs and/or FX is restricted or prohibited
    by local laws or regulations.

    The information in this email and any attachment is confidential and is
    intended only for the named recipient(s). The email may not be disclosed
    or used by any person other than the addressee, nor may it be copied in
    any way. If you are not the intended recipient please notify the sender
    immediately and delete any copies of this message. Any unauthorised
    copying, disclosure or distribution of the material in this e-mail is
    strictly forbidden.

    LMAX operates a multilateral trading facility. Authorised and regulated
    by the Financial Services Authority (firm registration number 509778) and
    is registered in England and Wales (number 06505809).
    Our registered address is Yellow Building, 1A Nicholas Road, London, W11
    4AN.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 11, '12 at 3:42p
activeMay 15, '12 at 10:57a
posts8
users5
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase