[Puppet Users] High CPU load / queued passenger requests

After restarting our puppetmaster host, we are seeing high CPU load and a
ton of queued passenger requests. I would suspect that our puppetmaster is
falling victim to the thundering herd of puppet agents, but we have
deployed most of our agents using a randomized cron job. The others are
using 'splay = true' if they are running puppet agent.

Some info:
Puppetmaster is RHEL6 x86_64 (VM)
4 vCPU 8GB memory
Apache 2.2
Passenger 3.0.7

The vhost for our puppetmaster includes these passenger options:
PassengerHighPerformance on
PassengerUseGlobalQueue on
PassengerPoolIdleTime 1500
PassengerMaxPoolSize 24
PassengerMaxRequests 4000
PassengerStatThrottleRate 120
PassengerSpawnMethod smart
RackAutoDetect Off
RailsAutoDetect Off


Look at all of these queued requests!

Passenger-status:
----------- General information -----------
max = 24
count = 24
active = 22
inactive = 2
Waiting on global queue: 234

----------- Application groups -----------
/etc/puppet/rack:
App root: /etc/puppet/rack
* PID: 24840 Sessions: 1 Processed: 528 Uptime: 9h 41m 19s
* PID: 24703 Sessions: 1 Processed: 513 Uptime: 9h 42m 17s
* PID: 24642 Sessions: 1 Processed: 567 Uptime: 9h 42m 29s
* PID: 24827 Sessions: 1 Processed: 572 Uptime: 9h 41m 28s
* PID: 24869 Sessions: 1 Processed: 530 Uptime: 9h 41m 4s
* PID: 24586 Sessions: 1 Processed: 572 Uptime: 9h 42m 32s
* PID: 24715 Sessions: 1 Processed: 539 Uptime: 9h 42m 17s
* PID: 29046 Sessions: 1 Processed: 487 Uptime: 8h 52m 35s
* PID: 24609 Sessions: 1 Processed: 575 Uptime: 9h 42m 30s
* PID: 24605 Sessions: 1 Processed: 488 Uptime: 9h 42m 30s
* PID: 24831 Sessions: 1 Processed: 555 Uptime: 9h 41m 27s
* PID: 29036 Sessions: 1 Processed: 549 Uptime: 8h 52m 35s
* PID: 24583 Sessions: 1 Processed: 565 Uptime: 9h 42m 32s
* PID: 24768 Sessions: 1 Processed: 521 Uptime: 9h 41m 56s
* PID: 24850 Sessions: 1 Processed: 579 Uptime: 9h 41m 16s
* PID: 29050 Sessions: 1 Processed: 497 Uptime: 8h 52m 35s
* PID: 24567 Sessions: 1 Processed: 536 Uptime: 9h 42m 37s
* PID: 24761 Sessions: 1 Processed: 570 Uptime: 9h 42m 5s
* PID: 24709 Sessions: 1 Processed: 575 Uptime: 9h 42m 17s
* PID: 29042 Sessions: 1 Processed: 488 Uptime: 8h 52m 35s
* PID: 24865 Sessions: 1 Processed: 559 Uptime: 9h 41m 5s
* PID: 29039 Sessions: 1 Processed: 506 Uptime: 8h 52m 35s

/usr/share/foreman:
App root: /usr/share/foreman
* PID: 24724 Sessions: 0 Processed: 5743 Uptime: 9h 42m 16s
* PID: 24717 Sessions: 0 Processed: 5099 Uptime: 9h 42m 17s