FAQ
Hi all

I have a scenario in mind for MCollective and I was looking for
some feedback

The basic idea is this:
1) A code push to Git triggers a Jenkins build
2) A successful build triggers automatic deployment to an automated
functional test environment
3) Functional tests run automatically and the results are reported etc

This is just basic CI practice and 1) and 3) are sorted, the only issues
are with 2) and the trigger for 3). The deployment job in Jenkins just
copies files up to the Puppet master ready for deployment and service
restarts etc etc, however with a basic puppet setup one has to wait for the
clients to run to pickup the new changes and restart services etc and only
then can Jenkins run the automated tests.

So the plan is for Jenkins to trigger a run on all the required Puppet
clients via MCollective (filtered by environment), synchronously wait for
the clients to finish and then run the tests. This has another advantage in
that we can back off the Puppet run interval (as important changes are
triggered by MCollective) which gives the Puppet master a bit of breathing
room, and the clients can just check-in every hour or so to check
everything is in order.

This also extends to deployments to other environments that have manual
deployment triggers such as a QA environment. Even though the deployment is
triggered by the QA team, the upload of the artifacts, running of puppet
clients and notification of a successful deployment should all be automated.

1): Does this make sense as a pattern, if not what is the best practice for
implementing this part of the CI and CD pipeline with puppet / jenkins /
mcollective etc?

2): How would this work in practice. I can run "mco puppet runonce" which
is fine, but if I filter by environment "mco puppet runonce --environment
aft" I get the "Cannot specify any custom puppet options when the daemon is
running" error. I can workaround this by stopping the daemons on every box,
however this means that the clients no-longer check in periodically. Is
there a fix for this or would you recommend turning them off anyway when
using a tool such as MCollective and doing ALL configuration updates via
MCollective triggers?

3) I need to do some more testing now, but just in case anyone knows off
the top of their heads, will an MCollective runonce call be synchronous,
i.e. will the mco call only finish once each client has reported that they
have finished their runs? If it's not, does anyone know of a good way to
check for a finished deployment (some command line wizardry with mco puppet
status / mco puppet summary, etc)?

I have done my research with Google and read a fair amount about
MCollective from various sources during the investigation and
implementation, however if this is already documented or I'm missing
something stupid I apologise in advance

Thanks

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • R.I.Pienaar at May 13, 2013 at 8:46 pm

    ----- Original Message -----
    From: "Tom Poulton" <poulton.tm@gmail.com>
    To: puppet-users@googlegroups.com
    Sent: Monday, May 13, 2013 3:38:34 PM
    Subject: [Puppet Users] MCollective deployment pattern

    Hi all

    I have a scenario in mind for MCollective and I was looking for
    some feedback

    The basic idea is this:
    1) A code push to Git triggers a Jenkins build
    2) A successful build triggers automatic deployment to an automated
    functional test environment
    3) Functional tests run automatically and the results are reported etc

    This is just basic CI practice and 1) and 3) are sorted, the only issues
    are with 2) and the trigger for 3). The deployment job in Jenkins just
    copies files up to the Puppet master ready for deployment and service
    restarts etc etc, however with a basic puppet setup one has to wait for the
    clients to run to pickup the new changes and restart services etc and only
    then can Jenkins run the automated tests.

    So the plan is for Jenkins to trigger a run on all the required Puppet
    clients via MCollective (filtered by environment), synchronously wait for
    the clients to finish and then run the tests. This has another advantage in
    that we can back off the Puppet run interval (as important changes are
    triggered by MCollective) which gives the Puppet master a bit of breathing
    room, and the clients can just check-in every hour or so to check
    everything is in order.

    This also extends to deployments to other environments that have manual
    deployment triggers such as a QA environment. Even though the deployment is
    triggered by the QA team, the upload of the artifacts, running of puppet
    clients and notification of a successful deployment should all be automated.

    1): Does this make sense as a pattern, if not what is the best practice for
    implementing this part of the CI and CD pipeline with puppet / jenkins /
    mcollective etc?

    2): How would this work in practice. I can run "mco puppet runonce" which
    is fine, but if I filter by environment "mco puppet runonce --environment
    aft" I get the "Cannot specify any custom puppet options when the daemon is
    running" error. I can workaround this by stopping the daemons on every box,
    however this means that the clients no-longer check in periodically. Is
    there a fix for this or would you recommend turning them off anyway when
    using a tool such as MCollective and doing ALL configuration updates via
    MCollective triggers?
    --environment here is not a filter, you're asking it to do a one-off run against
    a different puppet environment. If you want to filter use -W environment=foo

    3) I need to do some more testing now, but just in case anyone knows off
    the top of their heads, will an MCollective runonce call be synchronous,
    i.e. will the mco call only finish once each client has reported that they
    have finished their runs? If it's not, does anyone know of a good way to
    check for a finished deployment (some command line wizardry with mco puppet
    status / mco puppet summary, etc)?
    it is not, it triggers a run and returns - puppet runs can take any number
    of times and do any number of things including restart mcollective, so this
    makes doing a synch run a rather nasty prospect.

    We def need some better approach and have plans for that but right now you need
    to involve triggering runs and either inspecting them or querying your reporting
    infrastructure for status.

    I'd have loved to be able to influence the config version - or similar - so that
    you can tag a specific puppet run with some identifiable piece of information that
    you can later correlate against reporting for instance to be able to say all
    the nodes run configuration tagged foo, and periodically query the reporting
    infra for nodes successfully having run a configuration tagged foo but this is
    not posible today

    I have done my research with Google and read a fair amount about
    MCollective from various sources during the investigation and
    implementation, however if this is already documented or I'm missing
    something stupid I apologise in advance

    Thanks

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Poulton at May 13, 2013 at 9:06 pm
    Thanks for the quick reply

    The -W environment=foo tip is very useful and definitely solves one
    problem. I appreciate that the synchronous part could get pretty nasty,
    rather you than me :) In the meantime have you got any tips on the best
    (most reliable) way to check up on triggered runs, you mentioned inspecting
    them or querying my reporting infrastructure: to be honest I'm
    pretty familiar with puppet in terms of writing modules etc, but I haven't
    really looked into the reporting or querying side of things at all, and
    I'm definitely a newcomer to MCollective, feel free to just dump a load of
    keywords and I can google around if that's easier?

    We're running Puppet Open Source but we do have Puppet Dashboard running if
    that offers any querying options against the DB or something?

    Thanks

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • R.I.Pienaar at May 13, 2013 at 9:16 pm

    ----- Original Message -----
    From: "Tom Poulton" <poulton.tm@gmail.com>
    To: puppet-users@googlegroups.com
    Sent: Monday, May 13, 2013 4:06:56 PM
    Subject: Re: [Puppet Users] MCollective deployment pattern

    Thanks for the quick reply

    The -W environment=foo tip is very useful and definitely solves one
    see http://docs.puppetlabs.com/mcollective/reference/basic/basic_cli_usage.html
    for the details on the cli, discovery etc, thats a little primer to get you going
    problem. I appreciate that the synchronous part could get pretty nasty,
    rather you than me :) In the meantime have you got any tips on the best
    (most reliable) way to check up on triggered runs, you mentioned inspecting
    them or querying my reporting infrastructure: to be honest I'm
    pretty familiar with puppet in terms of writing modules etc, but I haven't
    really looked into the reporting or querying side of things at all, and
    I'm definitely a newcomer to MCollective, feel free to just dump a load of
    keywords and I can google around if that's easier?

    We're running Puppet Open Source but we do have Puppet Dashboard running if
    that offers any querying options against the DB or something?
    well lets say your deployment updates some package, or you have a define like

        myapp{"acme": ensure => 1.2.3}

    and your upgrade is gong to do that, then this is something you can interrogate
    via mco to figure out the state of things. but that sux.

    if you know you dont manage mcollective and these deploy runs tend to have a
    reasonable short run time that doesnt fluctuate hugely, I'd just write a little
    agent that runs puppet in the foreground for you to do it synchronously

    there's nothing inherently wrong with running puppet synchronously like this
    as long as you know you wont be restarting mco or doing stuff that fluctuate
    in times heavily.

    alternative you can use the config_version feature in puppet and set the current
    catalog version to lets say a git tag or svn revision and interrogate the number
    via the last_run_summary action on the puppet agent which will show you the
    catalog version of the most recent run

    or puppet reports etc, there's an annoying amount of solutions to this problem
    and the right one depends on your environment unfortunately
    Thanks

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Poulton at May 14, 2013 at 3:01 pm
    I'll have a play around in the next few days when I get a chance and report
    back

    I'm thinking a little foreground agent is probably the way to go for now as
    a "quick fix", and then I'll work on something cleaner once we start phase
    2 in a couple of months

    Thanks for the pointers, much appreciated


    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users?hl=en.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tom Poulton at Aug 21, 2013 at 4:09 pm
    Well I've been using an initial solution for the past month or two which
    seems to be working ok.

    I have a custom puppet function that reads the version of the artifact
    being deployed and I add that to the mcollective facts.yaml file at the
    same time that the artifact is deployed. I've also got a groovy script
    which I've put up on GitHub (
    https://github.com/TomPoulton/mcollective-synchronous-puppet/blob/master/deployment_trigger.groovy)
    that runs as part of the Jenkins deployment job that finds the correct
    instances, triggers a puppet run, then waits until all the instances show
    the new version in their facts. It handles the situation where puppet is
    already running, and also has timeouts so the script doesn't run forever.

    The only problem is that sometimes (although it's started to get more
    frequent) the script reaches it's timeout waiting for the new version to
    show in the facts, the timeout is pretty generous and is certainly longer
    than a puppet run and an mcollective restart would take, and the facts are
    being updated correctly. Does mcollective have a cache or something that
    would prevent the new values being returned to the mco client on the Jenins
    box?

    Thanks

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 13, '13 at 8:38p
activeAug 21, '13 at 4:09p
posts6
users2
websitepuppetlabs.com

2 users in discussion

Tom Poulton: 4 posts R.I.Pienaar: 2 posts

People

Translate

site design / logo © 2022 Grokbase