[Puppet Users] scaling puppet, skipping puppetmaster?

Note that using multiple masters is one way to solve this.

Currently we have two puppetmaster servers with ALL requests being
load balanced. I use unison to keep the ssl directory in sync between
hosts. Each server runs keepalived and requests go to a VIP that
exists on one of the servers. The server with the VIP load balances
the requests (mod_proxy) between both servers.

It's working relatively fine, though it would be ideal to have the
agents connect at semi-random intervals in order to reduce "thundering
herd" issues. We are over 500 without any real issues. Also, the
decentralized approach works fine but there are caveats related to the
use of custom functions that rely on a central server, virtual
resources(?), etc. I would try to scale your masters as it's not that
hard.

-Ryan

I use Puppet Commander, an MCollective tool:
http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/ToolPuppetcommander

Rather than have my Puppet Agents check in, no Puppet service runs on
any server, instead MCollective is running. Puppet Commander uses the
MCollective framework to schedule Puppet runs. I decide how many
should run at once and what their run interval should be based on the
load of the Puppet Master.

When I hit the upper limit of the VM running the Puppet Master but
want to increase the run speed, I'll put in load balanced Puppet like
others have mentioned.

Masterless Puppet is an option, however files copied from the Puppet
Master using Puppet's in built file transfer (source =>
"puppet:///...") are not included in the catalog, so each host needs
to be able to request files from the Puppet Master which may re-
introduce your bottleneck depending on how many there are. You can put
file _contents_ into a catalog, but this blows out the size and you'll
be re-transferring every file with each catalog (plus any security
concerns of having this file content stored on disk in many places).

From your other post if we manage 15 config files and services on 200
servers with Puppet - that should be nothing in terms of load for one
Apache/Passenger driven Puppet Master. A few CPUs and 4GiB RAM, tops.

Huh. Sounds like central scheduled rdist, in a way. The main
difference being, your scheduler tells clients, "okay get files from
me now", whereas rdist tells clients, "HERE are your files, take em!"
:)

Thanks, I'll look into mcollective.