FAQ
Hi

I have a resource that is slow to come into service and I have other resources that are dependent on the service in order to be configured. I've been searching for the "puppet way" to do this, but all the options I've found don't seem optimal, so I'm assuming there is a better way that I'm missing.

I have several instances, so I want puppet to kick them off in parallel. And I want the reliant services to be configured as soon as possible when the slow services are ready

So I don't really want to use long timeout/multiple retry because I believe that won't be executed in parallel - and anyway, it's not a retry I want, it's a wait

I don't want to have Mcollective/AMQP just for this one problem, it feels like a sledgehammer to crack a nut

I don't want some external process to repeatedly kick puppet repeatedly until the dependency is resolved and the reliant service can be configured

I'm imagining something like a semaphore, puppet sleeps and wakes up when the semaphore is flipped (the slow resource is available)

Sorry if this is a dumb question, first post etc ...

Thanks

Dave
--
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zZfbu37DrsQJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Christopher Wood at Jul 10, 2012 at 6:52 pm
    (inline)
    On Tue, Jul 10, 2012 at 11:39:58AM -0700, Dave Anderson wrote:
    Hi

    I have a resource that is slow to come into service and I have other resources that are dependent on the service in order to be configured. I've been searching for the "puppet way" to do this, but all the options I've found don't seem optimal, so I'm assuming there is a better way that I'm missing.

    I have several instances, so I want puppet to kick them off in parallel. And I want the reliant services to be configured as soon as possible when the slow services are ready

    So I don't really want to use long timeout/multiple retry because I believe that won't be executed in parallel - and anyway, it's not a retry I want, it's a wait

    I don't want to have Mcollective/AMQP just for this one problem, it feels like a sledgehammer to crack a nut

    I don't understand your issue with mcollective. You have an orchestration issue but don't want to use the puppet-ecosystem orchestration tool? This is the part that confuses me, especially since this won't be your last orchestration requirement.

    Speaking from ignorance, maybe look into exported resources and have the slow services export something.

    http://docs.puppetlabs.com/guides/exported_resources.html

    I don't want some external process to repeatedly kick puppet repeatedly until the dependency is resolved and the reliant service can be configured

    I'm imagining something like a semaphore, puppet sleeps and wakes up when the semaphore is flipped (the slow resource is available)

    Sorry if this is a dumb question, first post etc ...

    Thanks

    Dave
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zZfbu37DrsQJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Felix Frank at Jul 11, 2012 at 7:49 am

    On 07/10/2012 08:52 PM, Christopher Wood wrote:
    I don't understand your issue with mcollective. You have an orchestration issue but don't want to use the puppet-ecosystem orchestration tool? This is the part that confuses me, especially since this won't be your last orchestration requirement.
    I agree with the OP that there is quite some overhead involved and there
    should be a good reason for implementing the requried infrastructure.

    What's not clear to me is: Should all this happen during the same agent run?

    I disbelieve puppet can handle this. Puppet really only cares about
    this: Is service A running? If not, start it. If starting is implemented
    synchronously, i.e. the start operation does not finish before the
    service is ready, puppet will wait for it to get ready. Nothing works in
    parallel in this scenario.

    On the other hand, if the start operation finishes immediately, puppet
    will consider the start to be complete and apply all depending resources.

    So if you wish your start procedures to work asynchronously, you will
    probably have to design your depending services accordingly, i.e. when
    you start services B and C, their initscritps (or whatever) need to
    linger in the background and wait for service A to be completely
    initialized before actually performing their true startup operations.

    HTH,
    Felix

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Trevor Vaughan at Jul 11, 2012 at 2:12 pm
    Hi Dave,

    This is definitely not a dumb question. I wrote a post on what I
    thought a puppet semaphore system should do quite some time ago.
    Unfortunately, you don't really want to hang an instance until the
    semaphore flips so you end up having to wait through puppet runs.

    If you've got a solid SSH infrastructure working, then I would suggest
    looking at something like Capistrano or Func to give you that one-off,
    lightweight punch that you need.

    The big thing to watch out for is that you need to make sure that your
    manifests work properly even if they don't run in order since your
    nodes will be calling in over time.

    One way of doing this is to use a custom fact and drop a file onto the
    system when you're ready for that extra bit to run or have it based on
    hitting the remote system to see if your target service is actually
    functional.

    Ideally though, the startup scripts for the different services would
    be smart enough to figure out if the environment is ready for them to
    run. The puppet can do what it does best and the services can merrily
    go on their way trying to come up until they succeed (or ultimately
    fail).

    Trevor
    On Tue, Jul 10, 2012 at 2:39 PM, Dave Anderson wrote:
    Hi

    I have a resource that is slow to come into service and I have other resources that are dependent on the service in order to be configured. I've been searching for the "puppet way" to do this, but all the options I've found don't seem optimal, so I'm assuming there is a better way that I'm missing.

    I have several instances, so I want puppet to kick them off in parallel. And I want the reliant services to be configured as soon as possible when the slow services are ready

    So I don't really want to use long timeout/multiple retry because I believe that won't be executed in parallel - and anyway, it's not a retry I want, it's a wait

    I don't want to have Mcollective/AMQP just for this one problem, it feels like a sledgehammer to crack a nut

    I don't want some external process to repeatedly kick puppet repeatedly until the dependency is resolved and the reliant service can be configured

    I'm imagining something like a semaphore, puppet sleeps and wakes up when the semaphore is flipped (the slow resource is available)

    Sorry if this is a dumb question, first post etc ...

    Thanks

    Dave
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zZfbu37DrsQJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


    --
    Trevor Vaughan
    Vice President, Onyx Point, Inc
    (410) 541-6699
    tvaughan@onyxpoint.com

    -- This account not approved for unencrypted proprietary information --

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Brian Gallew at Jul 11, 2012 at 2:43 pm
    If I understand the OPs question correctly, he has an ordering problem:
    some services aren't waiting for other services to come online before
    starting themselves?

    If that's the case, you handle this with dependencies and chaining, e.g.
    service { fast_service: require => Service['slow_service']...}

    If the real issue is that the initial startup of a service involves a long
    initialization process, then you need to insert some kind of test in
    between the two services that verifies the slow service is truly up and
    running, e.g.
    exec { slow_service_check: require => Service['slow_service'], command =>
    "process that will wait for the slow service"...}
    service { fast_service: require => Exec['slow_service_check']...}

    If the real issue is that the slow service takes longer than puppet_timeout
    to complete its startup, then you have an insurmountable problem without
    either MCollective or multiple puppet runs.

    One way to address this with multiple puppet runs involves having a
    progression of environments (assuming this happens once on provisioning and
    never again). Essentially, you have systems boot up into a "provisioning"
    environment, where puppet runs, say, every 5 minutes. Then you have your
    ENC set up such that when a provisioning run completes with no changes, it
    switches to a "longterm" environment, which is generally much more
    lightweight and runs puppet on a slower schedule (e.g. once a day outside
    of business hours). This methodology can make the majority of your puppet
    runs very lightweight and fast (which can matter if you are paying for CPU
    cycles), but requires that everyone be good citizens who never change a
    system without using Puppet.
    On Wed, Jul 11, 2012 at 7:12 AM, Trevor Vaughan wrote:

    Hi Dave,

    This is definitely not a dumb question. I wrote a post on what I
    thought a puppet semaphore system should do quite some time ago.
    Unfortunately, you don't really want to hang an instance until the
    semaphore flips so you end up having to wait through puppet runs.

    If you've got a solid SSH infrastructure working, then I would suggest
    looking at something like Capistrano or Func to give you that one-off,
    lightweight punch that you need.

    The big thing to watch out for is that you need to make sure that your
    manifests work properly even if they don't run in order since your
    nodes will be calling in over time.

    One way of doing this is to use a custom fact and drop a file onto the
    system when you're ready for that extra bit to run or have it based on
    hitting the remote system to see if your target service is actually
    functional.

    Ideally though, the startup scripts for the different services would
    be smart enough to figure out if the environment is ready for them to
    run. The puppet can do what it does best and the services can merrily
    go on their way trying to come up until they succeed (or ultimately
    fail).

    Trevor
    On Tue, Jul 10, 2012 at 2:39 PM, Dave Anderson wrote:
    Hi

    I have a resource that is slow to come into service and I have other
    resources that are dependent on the service in order to be configured. I've
    been searching for the "puppet way" to do this, but all the options I've
    found don't seem optimal, so I'm assuming there is a better way that I'm
    missing.
    I have several instances, so I want puppet to kick them off in parallel.
    And I want the reliant services to be configured as soon as possible when
    the slow services are ready
    So I don't really want to use long timeout/multiple retry because I
    believe that won't be executed in parallel - and anyway, it's not a retry I
    want, it's a wait
    I don't want to have Mcollective/AMQP just for this one problem, it
    feels like a sledgehammer to crack a nut
    I don't want some external process to repeatedly kick puppet repeatedly
    until the dependency is resolved and the reliant service can be configured
    I'm imagining something like a semaphore, puppet sleeps and wakes up
    when the semaphore is flipped (the slow resource is available)
    Sorry if this is a dumb question, first post etc ...

    Thanks

    Dave
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are
    addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.

    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/zZfbu37DrsQJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --
    Trevor Vaughan
    Vice President, Onyx Point, Inc
    (410) 541-6699
    tvaughan@onyxpoint.com

    -- This account not approved for unencrypted proprietary information --

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jcbollinger at Jul 11, 2012 at 4:03 pm

    On Tuesday, July 10, 2012 1:39:58 PM UTC-5, Dave Anderson wrote:
    Hi

    I have a resource that is slow to come into service and I have other
    resources that are dependent on the service in order to be configured. I've
    been searching for the "puppet way" to do this, but all the options I've
    found don't seem optimal, so I'm assuming there is a better way that I'm
    missing.

    I have several instances, so I want puppet to kick them off in parallel.
    And I want the reliant services to be configured as soon as possible when
    the slow services are ready
    The Puppet agent is single-threaded, and it actively prevents multiple
    (Puppet) instances from running simultaneously. The only way you can start
    multiple resources in parallel is if Puppet is used only to trigger an
    asynchronous startup. Although that's doable, it's not very consistent
    with the Puppet resource model, because Puppet relies on knowing whether
    its attempts to synchronize a resource succeed. To put it another way,
    Puppet resource synchronization is (assumed to be) atomic when viewed at
    the DSL level.

    If you can model your group of slow resources so that together they look
    like a single resource to Puppet, then perhaps it would work out. Such a
    construction cannot be done in Puppet DSL, but you can either write a
    script that you will wrap an an Exec resource, or you can have a go at
    writing a custom resource type. Possibly you could use Puppet's Ruby DSL,
    but my intuition suggests that might be dangerous for this kind of task.

    So I don't really want to use long timeout/multiple retry because I
    believe that won't be executed in parallel - and anyway, it's not a retry I
    want, it's a wait
    Good, because Puppet assumes that waiting is what happens. Or bad, because
    Puppet isn't going to sync multiple (Puppet) resources on the same node in
    parallel no matter what you do.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/q5xaacPWRIUJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Dave Anderson at Jul 12, 2012 at 10:16 am
    Hi Everyone,

    First, thanks for all the great feedback!

    Chris, I'll take a look at exported resources. I don't have a problem with
    MCollective per se, I just don't want to add a bunch of other software if
    there's a "native puppet" way to solve the problem. From what I've seen,
    Puppet itself isn't supposed to solve this problem, MCollective is.

    Felix, yes my aim was to have everything complete in one Puppet run. My
    next-best option would be that Puppet would run again as soon as the
    slow-starting-service is ready - as opposed to waiting until the next
    scheduled run.

    Trevor, I'll take a look at Capistrano (that was mentioned at the recent
    Dublin Puppet Camp) but I have the same reservations as I would have with
    MCollective. Func isn't possible for us, due to arcane/licensing issues.

    Brian, thanks for your comments too. I will definitely model the service
    dependencies.

    My "plan A" right now is that when the slow-running service is up and
    running it will tell Puppet to run. I haven't really thought about how this
    would work for multiple instances of the slow-service, I'm pretty sure
    that's not a hard problem to solve though.

    Cheers everyone!

    Dave

    On Tuesday, 10 July 2012 19:39:58 UTC+1, Dave Anderson wrote:

    Hi

    I have a resource that is slow to come into service and I have other
    resources that are dependent on the service in order to be configured. I've
    been searching for the "puppet way" to do this, but all the options I've
    found don't seem optimal, so I'm assuming there is a better way that I'm
    missing.

    I have several instances, so I want puppet to kick them off in parallel.
    And I want the reliant services to be configured as soon as possible when
    the slow services are ready

    So I don't really want to use long timeout/multiple retry because I
    believe that won't be executed in parallel - and anyway, it's not a retry I
    want, it's a wait

    I don't want to have Mcollective/AMQP just for this one problem, it feels
    like a sledgehammer to crack a nut

    I don't want some external process to repeatedly kick puppet repeatedly
    until the dependency is resolved and the reliant service can be configured

    I'm imagining something like a semaphore, puppet sleeps and wakes up when
    the semaphore is flipped (the slow resource is available)

    Sorry if this is a dumb question, first post etc ...

    Thanks

    Dave
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for
    the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/pSQNRuLUHgAJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Tim Mooney at Jul 12, 2012 at 4:37 pm

    In regard to: [Puppet Users] Re: The Puppet Way to handle slow resources?...:

    Chris, I'll take a look at exported resources. I don't have a problem with
    MCollective per se, I just don't want to add a bunch of other software if
    there's a "native puppet" way to solve the problem. From what I've seen,
    Puppet itself isn't supposed to solve this problem, MCollective is. Agreed.
    My "plan A" right now is that when the slow-running service is up and
    running it will tell Puppet to run. I haven't really thought about how this
    would work for multiple instances of the slow-service, I'm pretty sure
    that's not a hard problem to solve though.
    I've only partially followed this thread so I don't know if someone else
    has already suggested this, but if the real issue is that the interaction
    between software, init script, and puppet isn't working correctly, then
    why not have puppet manage and use a "wrapper" init script? You keep the
    init script that came with the software, but instead of having puppet use
    that for start/stop/status, you write your own "local-service" or
    "mycompany-service" init script and have that script call the original
    script and augment the logic in start/stop/status/whatever to do whatever
    is needed to work correctly with puppet.

    Tim
    --
    Tim Mooney tim.mooney@ndsu.edu
    Enterprise Computing & Infrastructure 701-231-1076 (Voice)
    Room 242-J6, IACC Building 701-231-8541 (Fax)
    North Dakota State University, Fargo, ND 58105-5164

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Dave Anderson at Jul 16, 2012 at 8:11 am
    That sounds like a very elegant solution!
    I'm thinking that this pattern could be useful for others, not just me. I'll start a separate thread asking for a location to publish and critique recipes.
    /d
    --
    This email and any files transmitted with it are confidential and intended
    solely for the use of the individual or entity to whom they are addressed.
    If you have received this email in error please notify the system manager.
    This message contains confidential information and is intended only for the
    individual named. If you are not the named addressee you should not
    disseminate, distribute or copy this e-mail.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/10e0e6HN-xsJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJul 10, '12 at 6:48p
activeJul 16, '12 at 8:11a
posts9
users7
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase