Grokbase › Groups › Puppet › puppet-users › October 2012
FAQ
Hello everyone,

I'm currently looking into using puppet to manage my homedir .dot files,
including .vim/ folder with a bunch of plugins totaling a few tens of files
(almost 100), in order to have a pleasant and productive working experience
on our organization's 500+ machines.

While this looks like a cool thing to do, I can't help but wonder how will
this affect puppet's performance both on the masters' and agents' sides and
how good puppet is at distributing lots of infrequently updated files.

My common sense tells me that from a performance point of view, I could use
the following methods, first in the list being the most recommended:

file {
content => file('file'),
}

file {
source => "puppet:///modules/...",
}

file {
content => template('template.erb'),
}


Personally, I'd go with using file() function for distributing static files
and source => "puppet:///" to manage directories recursively, which would
simplify the setup part a lot.


Does anyone have a more in-depth insight into the performance of all these
methods ? Or, for that matter, why I shouldn't be doing the above as it
blows puppet performance in ways I can't comprehend ?


Thanks for your feedback,
Mihai

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/psSkZ5Ww2H8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Jcbollinger at Oct 11, 2012 at 1:39 pm ⇧

    On Tuesday, October 9, 2012 9:45:42 AM UTC-5, Mihai Stan wrote:
    Hello everyone,

    I'm currently looking into using puppet to manage my homedir .dot files,
    including .vim/ folder with a bunch of plugins totaling a few tens of files
    (almost 100), in order to have a pleasant and productive working experience
    on our organization's 500+ machines.

    While this looks like a cool thing to do, I can't help but wonder how will
    this affect puppet's performance both on the masters' and agents' sides and
    how good puppet is at distributing lots of infrequently updated files.

    My common sense tells me that from a performance point of view, I could
    use the following methods, first in the list being the most recommended:

    file {
    content => file('file'),
    }

    file {
    source => "puppet:///modules/...",
    }

    file {
    content => template('template.erb'),
    }


    Personally, I'd go with using file() function for distributing static
    files and source => "puppet:///" to manage directories recursively, which
    would simplify the setup part a lot.


    Does anyone have a more in-depth insight into the performance of all these
    methods ? Or, for that matter, why I shouldn't be doing the above as it
    blows puppet performance in ways I can't comprehend ?
    I would expect a bigger difference between using 'content' and using
    'source' than between specifying the content via file() and specifying it
    via template(). When you use 'content', the body of the target file is
    included in the catalog, whereas when you use 'source', the agent will
    request file content from the Puppet file server if it needs it.

    For ~100 small files (<= 1 KB on average, say), my gut tells me that
    'content' will win at least when there are more than a couple of changes,
    and that it will not lose too badly when there are few or none.

    Certainly you should not use template() for completely static files, but
    when you do need it, you need it.

    Managing directories recursively is a potential problem, but it really
    comes down more to the total number and combined size of the files you are
    managing. Each file and directory adds memory and processing cost on both
    ends, and, separately, Puppet expends computing effort proportional to the
    total content size to compute hashes of file contents.

    My expectation would be that for dotfiles, you will see acceptable
    performance any way around. Nevertheless, as with any performance
    question, the only way to get a reliable answer is to test it.


    John

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wmKO8HfxPvMJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Eric Sorenson at Oct 12, 2012 at 12:17 am ⇧
    I think all that is right John--one subtlety that led me to use
    content=>template everywhere instead of mixing it with content=>file for
    things which didn't need erb interpolation is that the argument to
    template() uses module-like syntax and file() needs a full filesystem path,
    so it doesn't work very well with environments that have different root
    paths.

    (this is https://projects.puppetlabs.com/issues/5158)

    -=Eric
    On Thursday, October 11, 2012 6:39:30 AM UTC-7, jcbollinger wrote:


    On Tuesday, October 9, 2012 9:45:42 AM UTC-5, Mihai Stan wrote:

    Hello everyone,

    I'm currently looking into using puppet to manage my homedir .dot files,
    including .vim/ folder with a bunch of plugins totaling a few tens of files
    (almost 100), in order to have a pleasant and productive working experience
    on our organization's 500+ machines.

    While this looks like a cool thing to do, I can't help but wonder how
    will this affect puppet's performance both on the masters' and agents'
    sides and how good puppet is at distributing lots of infrequently updated
    files.

    My common sense tells me that from a performance point of view, I could
    use the following methods, first in the list being the most recommended:

    file {
    content => file('file'),
    }

    file {
    source => "puppet:///modules/...",
    }

    file {
    content => template('template.erb'),
    }


    Personally, I'd go with using file() function for distributing static
    files and source => "puppet:///" to manage directories recursively, which
    would simplify the setup part a lot.


    Does anyone have a more in-depth insight into the performance of all
    these methods ? Or, for that matter, why I shouldn't be doing the above as
    it blows puppet performance in ways I can't comprehend ?
    I would expect a bigger difference between using 'content' and using
    'source' than between specifying the content via file() and specifying it
    via template(). When you use 'content', the body of the target file is
    included in the catalog, whereas when you use 'source', the agent will
    request file content from the Puppet file server if it needs it.

    For ~100 small files (<= 1 KB on average, say), my gut tells me that
    'content' will win at least when there are more than a couple of changes,
    and that it will not lose too badly when there are few or none.

    Certainly you should not use template() for completely static files, but
    when you do need it, you need it.

    Managing directories recursively is a potential problem, but it really
    comes down more to the total number and combined size of the files you are
    managing. Each file and directory adds memory and processing cost on both
    ends, and, separately, Puppet expends computing effort proportional to the
    total content size to compute hashes of file contents.

    My expectation would be that for dotfiles, you will see acceptable
    performance any way around. Nevertheless, as with any performance
    question, the only way to get a reliable answer is to test it.


    John
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/cL9ngDH2YEgJ.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedOct 9, '12 at 5:09p
activeOct 12, '12 at 12:17a
posts3
users3
websitepuppetlabs.com

3 users in discussion

Mihai Stan: 1 post Jcbollinger: 1 post Eric Sorenson: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase