FAQ
Hi list,

I am looking at configuration management tool options.

I have a large fleet (> 3,000 hosts) and highly heterogeneous
(RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
new versions of Solaris & AIX but ability to compile on older versions
is an advantage. Probably, the Windows will be managed by SCCM.

I have read that Puppet could have scalability problems to a site as
large as ours. To keep this simple, I'd like feedback on whether that
is likely to be true for us.

A rep from CFengine has told me that ours would be the largest Puppet
site in the world (I think that's not true). Could someone confirm?

General feedback also welcome.

Kind regards,
Alex Harvey

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Christopher Johnston at May 14, 2012 at 4:33 pm
    3k? I know of larger but can't comment on who they are, but they are close
    to 8-10k nodes AFAIK if not more by now.
    On Mon, May 14, 2012 at 2:53 AM, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?

    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Nigel Kersten at May 14, 2012 at 4:35 pm

    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?
    There are at least a couple of deployments that are more like 30k hosts
    than 3k....

    It's absolutely not true.


    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    --
    Nigel Kersten | http://puppetlabs.com | @nigelkersten
    Schedule Meetings at: http://tungle.me/nigelkersten

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Eric Shamow at May 14, 2012 at 4:38 pm
    Nigel,

    Don't we have one or two an order of magnitude larger than that?

    http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning: large PDF)

    I think they're looking at 300K or so. And I believe we have a few more at that size as well.

    -Eric

    --

    Eric Shamow
    Professional Services
    http://puppetlabs.com/
    (c)631.871.6441

    On Monday, May 14, 2012 at 12:35 PM, Nigel Kersten wrote:


    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey (mailto:alexharv074@gmail.com)> wrote:
    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?
    There are at least a couple of deployments that are more like 30k hosts than 3k....

    It's absolutely not true.

    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com (mailto:puppet-users@googlegroups.com).
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com (mailto:puppet-users%2Bunsubscribe@googlegroups.com).
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


    --
    Nigel Kersten | http://puppetlabs.com | @nigelkersten
    Schedule Meetings at: http://tungle.me/nigelkersten

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com (mailto:puppet-users@googlegroups.com).
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com (mailto:puppet-users+unsubscribe@googlegroups.com).
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Steve Traylen at May 15, 2012 at 12:10 pm

    On Monday, 14 May 2012 18:38:29 UTC+2, Eric Shamow wrote:
    Nigel,

    Don't we have one or two an order of magnitude larger than that?


    http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning:
    large PDF)

    I think they're looking at 300K or so. And I believe we have a few more
    at that size as well.

    We are currently along way from that with a single master and around 200
    nodes
    while we learn puppet and migrate away from quattor.
    The 300k figures are a couple of years away, we will have a respectable
    figure
    sometime this year we would hope.

    Steve (CERN IT)



    -Eric

    --

    Eric Shamow
    Professional Services
    http://puppetlabs.com/
    (c)631.871.6441

    On Monday, May 14, 2012 at 12:35 PM, Nigel Kersten wrote:



    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?


    There are at least a couple of deployments that are more like 30k hosts
    than 3k....

    It's absolutely not true.




    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.




    --
    Nigel Kersten | http://puppetlabs.com | @nigelkersten
    Schedule Meetings at: http://tungle.me/nigelkersten

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/P6IzZ2PT1DEJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Alex Harvey at May 19, 2012 at 3:53 am

    On 15 May 2012 22:10, Steve Traylen wrote:
    We are currently along way from that with a single master and around 200
    nodes
    while we learn puppet and migrate away from quattor.
    The 300k figures are a couple of years away, we will have a respectable
    figure
    sometime this year we would hope.

    Steve (CERN IT)
    I have had a careful look at
    http://projects.puppetlabs.com/projects/1/wiki/Whos_Using_Puppet

    This list is, in fairness, consistent with what the CFEngine rep told me.
    If there are sites out there with >3K or even >10K nodes, but not listed at
    Puppet's website, it is reasonable to suppose that the CFEngine rep doesn't
    know about them. :)

    Is it possible that this list needs updating? Maybe large sites aren't as
    keen to share their stories with the world?

    From the list, I also get the feeling that sites using Puppet to manage
    their infrastructure tend to be Linux sites. Google appears to be using
    Puppet to manage "all recent Mac OS X and Linux desktops, laptops, servers
    in the corporate infrastructure". They don't use it "in production",
    whatever that means. For those taking it beyond Linux and Mac OS X, I see
    Solaris mentioned a bit. Are Sun/Oracle still using it for their Solaris
    kit? I don't see any mention of AIX at all. I believe I have heard the
    same rumour of the large company at 10K nodes that uses Solaris, AIX and
    Linux.

    So, I am no longer doubting that Puppet can scale to a site as large as
    ours - especially given what James Turnbull wrote above - but I'd like to
    ensure that I get my facts right.

    Is it possible that the large ~ 10K site is one of the first of its kind in
    the world - especially to use it for AIX? What about the 30K site? A
    CFEngine document I downloaded claims, "Puppet has recently retracted
    claims of running 35000 servers at a major bank." Is this also true?

    Best wishes.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Erik Dalén at May 15, 2012 at 2:03 pm

    On 14 May 2012 18:35, Nigel Kersten wrote:
    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows).  We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage.  Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours.  To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true).  Could someone confirm?

    There are at least a couple of deployments that are more like 30k hosts than
    3k....

    It's absolutely not true.
    However, storeconfigs does have some scalability issues, so I guess
    that they aren't using that feature with 30k hosts?

    (soon there is of course the puppetdb that should replace it and be
    more scalable afaik)

    --
    Erik Dalén

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Alex Harvey at May 16, 2012 at 2:20 pm
    Thanks kindly to all for the thoughtful responses. I'll be looking closely
    at the idea of using load balanced Puppet servers.

    I wonder if anyone has any thoughts on the other problem I'm told I'll
    probably encounter, namely the difficulty in compiling Ruby and other
    packages in AIX, HPUX & earlier versions of Solaris?

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Bill Proud at May 16, 2012 at 4:49 pm

    On Wednesday, May 16, 2012 4:20:23 PM UTC+2, Alex Harvey wrote:
    Thanks kindly to all for the thoughtful responses. I'll be looking
    closely at the idea of using load balanced Puppet servers.

    I wonder if anyone has any thoughts on the other problem I'm told I'll
    probably encounter, namely the difficulty in compiling Ruby and other
    packages in AIX, HPUX & earlier versions of Solaris?
    I'm running puppet with Solaris and I haven't had to compile anything.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/LQUmnKs43DsJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Nigel Kersten at May 16, 2012 at 6:26 pm

    On Tue, May 15, 2012 at 7:03 AM, Erik Dalén wrote:
    On 14 May 2012 18:35, Nigel Kersten wrote:

    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?

    There are at least a couple of deployments that are more like 30k hosts than
    3k....

    It's absolutely not true.
    However, storeconfigs does have some scalability issues, so I guess
    that they aren't using that feature with 30k hosts?

    (soon there is of course the puppetdb that should replace it and be
    more scalable afaik)
    Let me firm that up a bit :)

    PuppetDB *does* replace the existing storeconfigs, and *is* much more
    scalable :)


    --
    Nigel Kersten | http://puppetlabs.com | @nigelkersten
    Schedule Meetings at: http://tungle.me/nigelkersten

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Jeff McCune at May 14, 2012 at 6:24 pm

    On Sun, May 13, 2012 at 11:53 PM, Alex Harvey wrote:
    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?
    Here's a recording of a short talk I gave about some performance testing I
    did last summer. These are the numbers I gathered from a real world
    production deployment of tens of thousands of nodes running against
    multiple Puppet Masters:

    http://www.youtube.com/watch?v=wjXrJjKymM4

    The CFengine rep is certainly not speaking the truth on this matter. I
    hope they're simply ignorant and not willingly lying.

    Cheers,
    --
    Jeff McCune

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • James Turnbull at May 14, 2012 at 7:44 pm
    Hi Alex
    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.
    On the actual scaling question... Puppet (2.6.x+) is easy to scale as it
    is just HTTP(S) traffic. You can front-end Puppet with a load balancer
    and then just cluster master nodes to scale to your needs in the same
    way you can grow any HTTP service.

    Regards

    James Turnbull

    --
    James Turnbull
    Puppet Labs
    1-503-734-8571
    To schedule a meeting with me: http://tungle.me/jamtur01

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Mister IT Guru at May 14, 2012 at 10:20 pm
    Hi Alex,

    Just a quick point,

    Glad to see Team Puppet out in force to show you the light! That CF rep, needs a swift kick in the …. well, I don't want to advocate violence here, I think he can do with being added to your junk mail list.

    3000 hosts? No problem! You have to serve one session per client every 30 mins, and your puppet master doesn't process much unless you've changed something. That's about a hundred hits a minute, so use apache instead of webrick, and your good to go!

    CERN as I understand it, are the guys trying to "recreate the big bang". Not simulate it, like NASA, or do some complex maths research about it… actually do it. Now, if anyone on the planet needs to be 100% sure that their gazillion or so hosts are all playing nice, it's them. They use Puppet.

    Next question!?
    On 14 May 2012, at 07:53, Alex Harvey wrote:

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?

    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Shawn at May 15, 2012 at 12:58 am
    Wanted to point out I manage over 3000 x86 tablets with puppet. This doesn't include production servers.
    Sent via BlackBerry

    -----Original Message-----
    From: Alex Harvey <alexharv074@gmail.com>
    Sender: puppet-users@googlegroups.com
    Date: Sun, 13 May 2012 23:53:52
    To: Puppet Users<puppet-users@googlegroups.com>
    Reply-To: puppet-users@googlegroups.com
    Subject: [Puppet Users] Puppet / scalability

    Hi list,

    I am looking at configuration management tool options.

    I have a large fleet (> 3,000 hosts) and highly heterogeneous
    (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
    6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
    new versions of Solaris & AIX but ability to compile on older versions
    is an advantage. Probably, the Windows will be managed by SCCM.

    I have read that Puppet could have scalability problems to a site as
    large as ours. To keep this simple, I'd like feedback on whether that
    is likely to be true for us.

    A rep from CFengine has told me that ours would be the largest Puppet
    site in the world (I think that's not true). Could someone confirm?

    General feedback also welcome.

    Kind regards,
    Alex Harvey

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedMay 14, '12 at 3:06p
activeMay 19, '12 at 3:53a
posts14
users11
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase