FAQ
I've heard that after (some # of machines) x (some size of manifests)
puppet does not scale well, due to bottlenecking on the puppetmaster.

Anyone doing large scale use by some other methods?
For example, running puppet on each machine individually using cron or
something? With the manifest distributed....
(via NFS? http? something else?)

I'd be interested to hear alternatives.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ucp6vuDFbjYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Aaron Grewell at Apr 27, 2012 at 8:07 pm
    Note that using multiple masters is one way to solve this.
    On Apr 27, 2012 10:46 AM, "Philip Brown" wrote:

    I've heard that after (some # of machines) x (some size of manifests)
    puppet does not scale well, due to bottlenecking on the puppetmaster.

    Anyone doing large scale use by some other methods?
    For example, running puppet on each machine individually using cron or
    something? With the manifest distributed....
    (via NFS? http? something else?)

    I'd be interested to hear alternatives.

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/ucp6vuDFbjYJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Ryan Bowlby at Apr 27, 2012 at 11:02 pm
    Currently we have two puppetmaster servers with ALL requests being
    load balanced. I use unison to keep the ssl directory in sync between
    hosts. Each server runs keepalived and requests go to a VIP that
    exists on one of the servers. The server with the VIP load balances
    the requests (mod_proxy) between both servers.

    It's working relatively fine, though it would be ideal to have the
    agents connect at semi-random intervals in order to reduce "thundering
    herd" issues. We are over 500 without any real issues. Also, the
    decentralized approach works fine but there are caveats related to the
    use of custom functions that rely on a central server, virtual
    resources(?), etc. I would try to scale your masters as it's not that
    hard.

    -Ryan

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Luke Bigum at Apr 28, 2012 at 9:12 am
    I use Puppet Commander, an MCollective tool:
    http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/ToolPuppetcommander

    Rather than have my Puppet Agents check in, no Puppet service runs on
    any server, instead MCollective is running. Puppet Commander uses the
    MCollective framework to schedule Puppet runs. I decide how many
    should run at once and what their run interval should be based on the
    load of the Puppet Master.

    When I hit the upper limit of the VM running the Puppet Master but
    want to increase the run speed, I'll put in load balanced Puppet like
    others have mentioned.

    Masterless Puppet is an option, however files copied from the Puppet
    Master using Puppet's in built file transfer (source =>
    "puppet:///...") are not included in the catalog, so each host needs
    to be able to request files from the Puppet Master which may re-
    introduce your bottleneck depending on how many there are. You can put
    file _contents_ into a catalog, but this blows out the size and you'll
    be re-transferring every file with each catalog (plus any security
    concerns of having this file content stored on disk in many places).

    From your other post if we manage 15 config files and services on 200
    servers with Puppet - that should be nothing in terms of load for one
    Apache/Passenger driven Puppet Master. A few CPUs and 4GiB RAM, tops.
    On Apr 27, 6:46 pm, Philip Brown wrote:
    I've heard that after (some # of machines) x (some size of manifests)
    puppet does not scale well, due to bottlenecking on the puppetmaster.

    Anyone doing large scale use by some other methods?
    For example, running puppet on each machine individually using cron or
    something? With the manifest distributed....
    (via NFS? http? something else?)

    I'd be interested to hear alternatives.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Philip Brown at Apr 28, 2012 at 2:58 pm

    On Sat, Apr 28, 2012 at 2:12 AM, Luke Bigum wrote:
    I use Puppet Commander, an MCollective tool:
    http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/ToolPuppetcommander

    Rather than have my Puppet Agents check in, no Puppet service runs on
    any server, instead MCollective is running. Puppet Commander uses the
    MCollective framework to schedule Puppet runs. I decide how many
    should run at once and what their run interval should be based on the
    load of the Puppet Master.
    Huh. Sounds like central scheduled rdist, in a way. The main
    difference being, your scheduler tells clients, "okay get files from
    me now", whereas rdist tells clients, "HERE are your files, take em!"
    :)

    Thanks, I'll look into mcollective.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedApr 27, '12 at 5:46p
activeApr 28, '12 at 2:58p
posts5
users4
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase