FAQ
How do I avoid a situation where all of my Linux servers execute a service
restart at the same time upon receiving a new configuration change via
Puppet? I am trying to avoid any possibility that the service would be
unavailable for any length of time. The servers are behind a load
balancer. At least one node needs to remain available. Any idea how I
might configure Puppet to work in this HA environment?

Thank you for your feedback!


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Matt Shields at Aug 30, 2013 at 12:22 am
    How about turning off the puppet agent on all the machines, make the
    change, then turning them back on a few at a time.

    Matt

    On Thu, Aug 29, 2013 at 4:50 PM, wrote:

    How do I avoid a situation where all of my Linux servers execute a service
    restart at the same time upon receiving a new configuration change via
    Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?****

    Thank you for your feedback!


    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Stuart Cracraft at Aug 30, 2013 at 2:21 am
    I think there should be a randomizer built-in to prevent this without these kinds of manual requirements.
    On Aug 29, 2013, at 5:16 PM, Matt Shields wrote:

    How about turning off the puppet agent on all the machines, make the change, then turning them back on a few at a time.

    Matt


    On Thu, Aug 29, 2013 at 4:50 PM, wrote:
    How do I avoid a situation where all of my Linux servers execute a service restart at the same time upon receiving a new configuration change via Puppet? I am trying to avoid any possibility that the service would be unavailable for any length of time. The servers are behind a load balancer. At least one node needs to remain available. Any idea how I might configure Puppet to work in this HA environment?

    Thank you for your feedback!




    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Peter Bukowinski at Aug 30, 2013 at 2:51 am
    There *is* a (semi-)randomizer. Look at the splay option in the puppet.conf

    http://docs.puppetlabs.com/references/latest/configuration.html#splay

    -- Peter (from phone)
    On Aug 29, 2013, at 10:21 PM, Stuart Cracraft wrote:

    I think there should be a randomizer built-in to prevent this without these kinds of manual requirements.
    On Aug 29, 2013, at 5:16 PM, Matt Shields wrote:

    How about turning off the puppet agent on all the machines, make the change, then turning them back on a few at a time.

    Matt

    On Thu, Aug 29, 2013 at 4:50 PM, wrote:
    How do I avoid a situation where all of my Linux servers execute a service restart at the same time upon receiving a new configuration change via Puppet? I am trying to avoid any possibility that the service would be unavailable for any length of time. The servers are behind a load balancer. At least one node needs to remain available. Any idea how I might configure Puppet to work in this HA environment?

    Thank you for your feedback!




    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matthew Burgess at Aug 30, 2013 at 9:42 am
    In your situation, I'd be tempted to not run the puppet agent in daemon
    mode at all so that you can retain full control of when the agents will
    check in. I can't see how the splay option will help avoid concurrent
    checkins:

    Imagine HA node 1 is rebooted for whatever reason and comes back up at
    12:00 at which point the puppet agent checks in due to its boot script
    running. Its splay parameter is set to true, its runinterval is the
    default 30m and its psuedo-random splay time is 5 mins. It will therefore
    next check in at 12:35.

    Now, HA node 2 just so happens to have checked in last at 11:55 and its
    pseudo-random splay time is 10 mins, so it's going to next check in at
    12:35. As you, by definition, cannot control the pseudo-random delay time
    you cannot guarantee the availability of services managed by Puppet.

    So, instead of running puppet in daemon mode, I'd look to use something
    like mcollective to control when the agents check in with the master.

    Kind Regards,

    Matt.

    On 29 August 2013 21:50, wrote:

    How do I avoid a situation where all of my Linux servers execute a service
    restart at the same time upon receiving a new configuration change via
    Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?****

    Thank you for your feedback!


    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matthew Burgess at Aug 30, 2013 at 10:26 am

    On 30 August 2013 10:42, Matthew Burgess wrote:
    So, instead of running puppet in daemon mode, I'd look to use something
    like mcollective to control when the agents check in with the master.
    You could of course just set up a cron job on each host, ensuring they
    check in at different times.

    Kind Regards,

    Matt.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rahul Khengare at Sep 2, 2013 at 5:27 am
    You can use different *runinterval *for each client.
    This can be done using editing of "/etc/puppet/puppet.conf" on each client
    machine.
    Set the following (add a new line if it's not already present)
    in the [agent] section of the file:

    runinterval=XXX

    where, XXX is the time in seconds(default is 180),

    Thanks and Regards,
    Rahul Khengare,
    NTT DATA OSS Center, Pune, India.

    On Friday, August 30, 2013 2:20:36 AM UTC+5:30, rjbu...@gmail.com wrote:

    How do I avoid a situation where all of my Linux servers execute a service
    restart at the same time upon receiving a new configuration change via
    Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?

    Thank you for your feedback!

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Stuart Cracraft at Sep 2, 2013 at 4:02 pm
    How can this be randomized within a range?

    I believe someone mentioned "splay" ?

    My fear is that all the boxes will request at a similar some day, by chance
    and send a tidal wave over to the master.
    On Sep 1, 2013, at 10:27 PM, Rahul Khengare wrote:

    You can use different runinterval for each client.
    This can be done using editing of "/etc/puppet/puppet.conf" on each client machine.
    Set the following (add a new line if it's not already present)
    in the [agent] section of the file:

    runinterval=XXX

    where, XXX is the time in seconds(default is 180),

    Thanks and Regards,
    Rahul Khengare,
    NTT DATA OSS Center, Pune, India.


    On Friday, August 30, 2013 2:20:36 AM UTC+5:30, rjbu...@gmail.com wrote:
    How do I avoid a situation where all of my Linux servers execute a service restart at the same time upon receiving a new configuration change via Puppet? I am trying to avoid any possibility that the service would be unavailable for any length of time. The servers are behind a load balancer. At least one node needs to remain available. Any idea how I might configure Puppet to work in this HA environment?

    Thank you for your feedback!




    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Matthew Burgess at Sep 3, 2013 at 10:45 am

    On 2 September 2013 17:01, Stuart Cracraft wrote:

    How can this be randomized within a range?
    I don't think it can; it suffers from the same issue as splay does, which
    I explained in some detail last week.

    My fear is that all the boxes will request at a similar some day, by chance
    and send a tidal wave over to the master.
    I think the only way to allay that fear is to maintain strict control over
    when each client (or group of clients) checks in, either using cron or
    mcollective. Or you could set up multiple masters in a load balanced
    configuration so that your puppet masters can cope with the maximum
    possible load that can be thrown at them.

    Regards,

    Matt.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Brad at Sep 3, 2013 at 5:13 pm
    Stuart,

    If I'm understanding your needs correctly, this may be what you're looking
    for:

    http://www.devco.net/archives/2010/03/17/scheduling_puppet_with_mcollective.php
    On Monday, September 2, 2013 11:01:46 AM UTC-5, Stuart Cracraft wrote:

    How can this be randomized within a range?

    I believe someone mentioned "splay" ?

    My fear is that all the boxes will request at a similar some day, by chance
    and send a tidal wave over to the master.

    On Sep 1, 2013, at 10:27 PM, Rahul Khengare wrote:

    You can use different *runinterval *for each client.
    This can be done using editing of "/etc/puppet/puppet.conf" on each
    client machine.
    Set the following (add a new line if it's not already present)
    in the [agent] section of the file:

    runinterval=XXX

    where, XXX is the time in seconds(default is 180),

    Thanks and Regards,
    Rahul Khengare,
    NTT DATA OSS Center, Pune, India.

    On Friday, August 30, 2013 2:20:36 AM UTC+5:30, rjbu...@gmail.com wrote:

    How do I avoid a situation where all of my Linux servers execute a
    service restart at the same time upon receiving a new configuration change
    via Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?

    Thank you for your feedback!

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users...@googlegroups.com <javascript:>.
    To post to this group, send email to puppet...@googlegroups.com<javascript:>
    .
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rich Burroughs at Sep 5, 2013 at 8:51 pm
    One of my former co-workers did a talk about Mcollective at PuppetConf.
    People involved in this thread might find it useful :) He gives some
    examples about doing rolling restarts, you could do something very similar
    with orchestrating agent runs.

    http://puppetlabs.com/presentations/intro-systems-orchestration-mcollective

    On Tue, Sep 3, 2013 at 10:13 AM, Brad wrote:

    Stuart,

    If I'm understanding your needs correctly, this may be what you're looking
    for:


    http://www.devco.net/archives/2010/03/17/scheduling_puppet_with_mcollective.php

    On Monday, September 2, 2013 11:01:46 AM UTC-5, Stuart Cracraft wrote:

    How can this be randomized within a range?

    I believe someone mentioned "splay" ?

    My fear is that all the boxes will request at a similar some day, by
    chance
    and send a tidal wave over to the master.

    On Sep 1, 2013, at 10:27 PM, Rahul Khengare wrote:

    You can use different *runinterval *for each client.
    This can be done using editing of "/etc/puppet/puppet.conf" on each
    client machine.
    Set the following (add a new line if it's not already present)
    in the [agent] section of the file:

    runinterval=XXX

    where, XXX is the time in seconds(default is 180),

    Thanks and Regards,
    Rahul Khengare,
    NTT DATA OSS Center, Pune, India.

    On Friday, August 30, 2013 2:20:36 AM UTC+5:30, rjbu...@gmail.com wrote:

    How do I avoid a situation where all of my Linux servers execute a
    service restart at the same time upon receiving a new configuration change
    via Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?

    Thank you for your feedback!

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users...@**googlegroups.com.
    To post to this group, send email to puppet...@googlegroups.com.

    Visit this group at http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users>
    .
    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .


    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Garrett Honeycutt at Sep 5, 2013 at 1:08 am

    On Monday, September 2, 2013 12:01:46 PM UTC-4, Stuart Cracraft wrote:
    How can this be randomized within a range?

    I believe someone mentioned "splay" ?

    My fear is that all the boxes will request at a similar some day, by chance
    and send a tidal wave over to the master.

    On Sep 1, 2013, at 10:27 PM, Rahul Khengare wrote:

    You can use different *runinterval *for each client.
    This can be done using editing of "/etc/puppet/puppet.conf" on each
    client machine.
    Set the following (add a new line if it's not already present)
    in the [agent] section of the file:

    runinterval=XXX

    where, XXX is the time in seconds(default is 180),

    Thanks and Regards,
    Rahul Khengare,
    NTT DATA OSS Center, Pune, India.

    On Friday, August 30, 2013 2:20:36 AM UTC+5:30, rjbu...@gmail.com wrote:

    How do I avoid a situation where all of my Linux servers execute a
    service restart at the same time upon receiving a new configuration change
    via Puppet? I am trying to avoid any possibility that the service would be
    unavailable for any length of time. The servers are behind a load
    balancer. At least one node needs to remain available. Any idea how I
    might configure Puppet to work in this HA environment?

    Thank you for your feedback!
    It is pretty common to run the puppet agent from cron as opposed to daemon
    mode. From there you can randomize when the agent runs using the fqdn as a
    seed with the fqdn_rand[1] function. Here is an example[2] of this in a
    manifest.

    [1] - http://docs.puppetlabs.com/references/latest/function.html#fqdnrand
    [2] -
    https://github.com/ghoneycutt/puppet-module-puppet/blob/master/manifests/agent.pp#L57

    Best regards,
    -g

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedAug 29, '13 at 9:58p
activeSep 5, '13 at 8:51p
posts12
users9
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase