FAQ
I'm getting more and more "execution expired" as systems checking and
hit puppetdb for the first time (switching from a mysql instance). The
command queue isn't long (1-5, if anything, all the time), and ym
master itself seems to be dealing well enough. I have seen the
collection time growing higher and higher though. This is a ~2K node
deployment, and one of the few things storedconfigs is used for is
ssh-key collection/exports (basically collect from 90% of there hosts
and export to the same 90%).

mysql seemed to deal with this fine...

I'm hesitant to tune puppetdb in terms of threads and heap size, as
both of those SEEM okay right now...would either affect collection
query time?

--
Matthew Nicholson

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Walter Heck at Jul 9, 2012 at 6:03 pm
    I tried out puppetdb a few weeks ago, and I remember reading somewhere that
    for larger deployments it is recommended to use postgres as the backend for
    puppetdb for any installations of size.

    At http://docs.puppetlabs.com/puppetdb/0.9/requirements.html it says:
    "Deployments with more than 100 nodes should configure a PostgreSQL
    database for PuppetDB. Smaller deployments may also wish to use the
    PostgreSQL backend.There are two available backends for PuppetDB
    storage:PuppetDB’s embedded databasePostgreSQLThe embedded database works
    well for small deployments (up to approximately 100 hosts). It requires no
    additional daemons or setup, and as such is very simple to get started
    with. It supports all PuppetDB features.However, there is a cost: the
    embedded database requires a fair amount of RAM to operate correctly. We’d
    recommend allocating 1GB to PuppetDB as a starting point. Additionally, the
    embedded database is somewhat opaque; unlike more off-the-shelf database
    daemons, there isn’t much companion tooling for things like interactive SQL
    consoles, performance analysis, or backups.That said, if you have a small
    installation and enough RAM, then the embedded database will work just
    fine.For most “real” use, we recommend running an instance of PostgreSQL.
    Simply install PostgreSQL using a module from the Puppet Forge or your
    local package manager, create a new (empty) database for PuppetDB, and
    verify that you can login via psql to this DB you just created. Then just
    supply PuppetDB with the DB host, port, name, and credentials you’ve just
    configured, and we’ll take care of the rest!"

    Hope this helps!
    On Jul 10, 2012 1:41 AM, "Matthew Nicholson" wrote:

    I'm getting more and more "execution expired" as systems checking and
    hit puppetdb for the first time (switching from a mysql instance). The
    command queue isn't long (1-5, if anything, all the time), and ym
    master itself seems to be dealing well enough. I have seen the
    collection time growing higher and higher though. This is a ~2K node
    deployment, and one of the few things storedconfigs is used for is
    ssh-key collection/exports (basically collect from 90% of there hosts
    and export to the same 90%).

    mysql seemed to deal with this fine...

    I'm hesitant to tune puppetdb in terms of threads and heap size, as
    both of those SEEM okay right now...would either affect collection
    query time?

    --
    Matthew Nicholson

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Matthew Nicholson at Jul 9, 2012 at 6:08 pm
    Sorry, I wasn't clear, this is with Ppostgres as the DB backend. Just
    upped the db servers kernel.shmmax value and postgres's shared_buffers
    values, seems to be helping thus far...
    On Mon, Jul 9, 2012 at 2:03 PM, Walter Heck wrote:
    I tried out puppetdb a few weeks ago, and I remember reading somewhere that
    for larger deployments it is recommended to use postgres as the backend for
    puppetdb for any installations of size.

    At http://docs.puppetlabs.com/puppetdb/0.9/requirements.html it says:
    "Deployments with more than 100 nodes should configure a PostgreSQL database
    for PuppetDB. Smaller deployments may also wish to use the PostgreSQL
    backend.There are two available backends for PuppetDB storage:PuppetDB’s
    embedded databasePostgreSQLThe embedded database works well for small
    deployments (up to approximately 100 hosts). It requires no additional
    daemons or setup, and as such is very simple to get started with. It
    supports all PuppetDB features.However, there is a cost: the embedded
    database requires a fair amount of RAM to operate correctly. We’d recommend
    allocating 1GB to PuppetDB as a starting point. Additionally, the embedded
    database is somewhat opaque; unlike more off-the-shelf database daemons,
    there isn’t much companion tooling for things like interactive SQL consoles,
    performance analysis, or backups.That said, if you have a small installation
    and enough RAM, then the embedded database will work just fine.For most
    “real” use, we recommend running an instance of PostgreSQL. Simply install
    PostgreSQL using a module from the Puppet Forge or your local package
    manager, create a new (empty) database for PuppetDB, and verify that you can
    login via psql to this DB you just created. Then just supply PuppetDB with
    the DB host, port, name, and credentials you’ve just configured, and we’ll
    take care of the rest!"

    Hope this helps!
    On Jul 10, 2012 1:41 AM, "Matthew Nicholson" wrote:

    I'm getting more and more "execution expired" as systems checking and
    hit puppetdb for the first time (switching from a mysql instance). The
    command queue isn't long (1-5, if anything, all the time), and ym
    master itself seems to be dealing well enough. I have seen the
    collection time growing higher and higher though. This is a ~2K node
    deployment, and one of the few things storedconfigs is used for is
    ssh-key collection/exports (basically collect from 90% of there hosts
    and export to the same 90%).

    mysql seemed to deal with this fine...

    I'm hesitant to tune puppetdb in terms of threads and heap size, as
    both of those SEEM okay right now...would either affect collection
    query time?

    --
    Matthew Nicholson

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --
    Matthew Nicholson

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Deepak Giridharagopal at Jul 9, 2012 at 8:59 pm
    FYI, we've been debugging this on IRC. We believe the issues were that
    there were insufficient system resources allocated to the database,
    and that frequent queries from the puppetdb dashboard itself were
    causing load spikes. Allocating more shared_buffers to the database
    and reducing the dashboard's polling interval (via the
    "pollingInterval" URL argument) appears sufficient.

    deepak
    On Mon, Jul 09, 2012 at 02:08:13PM -0400, Matthew Nicholson wrote:
    Sorry, I wasn't clear, this is with Ppostgres as the DB backend. Just
    upped the db servers kernel.shmmax value and postgres's shared_buffers
    values, seems to be helping thus far...
    On Mon, Jul 9, 2012 at 2:03 PM, Walter Heck wrote:
    I tried out puppetdb a few weeks ago, and I remember reading somewhere that
    for larger deployments it is recommended to use postgres as the backend for
    puppetdb for any installations of size.

    At http://docs.puppetlabs.com/puppetdb/0.9/requirements.html it says:
    "Deployments with more than 100 nodes should configure a PostgreSQL database
    for PuppetDB. Smaller deployments may also wish to use the PostgreSQL
    backend.There are two available backends for PuppetDB storage:PuppetDB’s
    embedded databasePostgreSQLThe embedded database works well for small
    deployments (up to approximately 100 hosts). It requires no additional
    daemons or setup, and as such is very simple to get started with. It
    supports all PuppetDB features.However, there is a cost: the embedded
    database requires a fair amount of RAM to operate correctly. We’d recommend
    allocating 1GB to PuppetDB as a starting point. Additionally, the embedded
    database is somewhat opaque; unlike more off-the-shelf database daemons,
    there isn’t much companion tooling for things like interactive SQL consoles,
    performance analysis, or backups.That said, if you have a small installation
    and enough RAM, then the embedded database will work just fine.For most
    “real” use, we recommend running an instance of PostgreSQL. Simply install
    PostgreSQL using a module from the Puppet Forge or your local package
    manager, create a new (empty) database for PuppetDB, and verify that you can
    login via psql to this DB you just created. Then just supply PuppetDB with
    the DB host, port, name, and credentials you’ve just configured, and we’ll
    take care of the rest!"

    Hope this helps!

    On Jul 10, 2012 1:41 AM, "Matthew Nicholson" <matthew.a.nicholson@gmail.com>
    wrote:
    I'm getting more and more "execution expired" as systems checking and
    hit puppetdb for the first time (switching from a mysql instance). The
    command queue isn't long (1-5, if anything, all the time), and ym
    master itself seems to be dealing well enough. I have seen the
    collection time growing higher and higher though. This is a ~2K node
    deployment, and one of the few things storedconfigs is used for is
    ssh-key collection/exports (basically collect from 90% of there hosts
    and export to the same 90%).

    mysql seemed to deal with this fine...

    I'm hesitant to tune puppetdb in terms of threads and heap size, as
    both of those SEEM okay right now...would either affect collection
    query time?

    --
    Matthew Nicholson

    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to
    puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.


    --
    Matthew Nicholson

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
    --
    Deepak Giridharagopal / Puppet Labs / grim_radical

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to puppet-users@googlegroups.com.
    To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJul 9, '12 at 5:41p
activeJul 9, '12 at 8:59p
posts4
users3
websitepuppetlabs.com

People

Translate

site design / logo © 2022 Grokbase