FAQ
Hi guys,

The command below was working when my puppet dashboard mysql database was
running in the same machine as the puppet master.

rake RAILS_ENV=production -f /usr/share/puppet-dashboard/Rakefile node:del
name=my-app-server

However now I moved the mysql database to RDS and it doesn't work anymore,
my database.yml reflects the change and the Puppet Dashboard web interface
works fine accessing the RDS database, but the rake task to delete is
broken. Interesting is that node:list works and list all my nodes.

production:
   database: dashboard_production
   username: username
   password: password
   host: something.at.rds.amazonaws.com
   encoding: utf8
   adapter: mysql

Did I miss something during the local mysql to RDS migration ?

Thanks,
Felipe

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Felipe Salum at Jul 24, 2013 at 8:49 pm
    I had a friend helping me to debug and looks like it is taking forever to
    delete the entries from resource status table (a lot of entries).

    Is that something we can improve ?
    On Wednesday, July 24, 2013 11:42:26 AM UTC-7, Felipe Salum wrote:

    Hi guys,

    The command below was working when my puppet dashboard mysql database was
    running in the same machine as the puppet master.

    rake RAILS_ENV=production -f /usr/share/puppet-dashboard/Rakefile node:del
    name=my-app-server

    However now I moved the mysql database to RDS and it doesn't work anymore,
    my database.yml reflects the change and the Puppet Dashboard web interface
    works fine accessing the RDS database, but the rake task to delete is
    broken. Interesting is that node:list works and list all my nodes.

    production:
    database: dashboard_production
    username: username
    password: password
    host: something.at.rds.amazonaws.com
    encoding: utf8
    adapter: mysql

    Did I miss something during the local mysql to RDS migration ?

    Thanks,
    Felipe
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ramin K at Jul 24, 2013 at 9:04 pm

    On 7/24/2013 1:49 PM, Felipe Salum wrote:
    I had a friend helping me to debug and looks like it is taking forever
    to delete the entries from resource status table (a lot of entries).

    Is that something we can improve ?
    env RAILS_ENV=production rake reports:prune upto=5 unit=day
    env RAILS_ENV=production rake reports:prune:orphaned

    You can change the RAILS_ENV and number of day, weeks, months, etc to
    match your system and its needs.

    Sounds like you've never pruned so it's going to be slow going. Try
    walking it forward like so.

    env RAILS_ENV=production rake reports:prune upto=3 unit=mon
    env RAILS_ENV=production rake reports:prune upto=1 unit=mon
    env RAILS_ENV=production rake reports:prune upto=2 unit=wk
    env RAILS_ENV=production rake reports:prune upto=5 unit=day

    Don't forget to add this to your crontab with the retentions settings
    that best fit your system.

    Ramin

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Felipe Salum at Jul 24, 2013 at 9:08 pm
    I actually do for 2 weeks.

    rake RAILS_ENV=production reports:prune upto=2 unit=wk


    On Wed, Jul 24, 2013 at 2:03 PM, Ramin K wrote:
    On 7/24/2013 1:49 PM, Felipe Salum wrote:

    I had a friend helping me to debug and looks like it is taking forever
    to delete the entries from resource status table (a lot of entries).

    Is that something we can improve ?
    env RAILS_ENV=production rake reports:prune upto=5 unit=day
    env RAILS_ENV=production rake reports:prune:orphaned

    You can change the RAILS_ENV and number of day, weeks, months, etc to
    match your system and its needs.

    Sounds like you've never pruned so it's going to be slow going. Try
    walking it forward like so.

    env RAILS_ENV=production rake reports:prune upto=3 unit=mon
    env RAILS_ENV=production rake reports:prune upto=1 unit=mon
    env RAILS_ENV=production rake reports:prune upto=2 unit=wk
    env RAILS_ENV=production rake reports:prune upto=5 unit=day

    Don't forget to add this to your crontab with the retentions settings that
    best fit your system.

    Ramin

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit https://groups.google.com/d/**
    topic/puppet-users/7-**sjp1Ef1p0/unsubscribe<https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe>
    .
    To unsubscribe from this group and all its topics, send an email to
    puppet-users+unsubscribe@**googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
    .
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users>
    .
    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ramin K at Jul 24, 2013 at 10:21 pm
    Doesn't matter, it's the size of the data and the amount you're deleting.

    Let's say that your reports:prune upto=2 unit=wk is ultimately going to
    delete 100k records because you've never pruned the database before.

    reports:prune upto=2 unit=wk, 100k, time: 3600s

    By breaking it up, it'll run faster because large deletes can cause poor
    performance in Mysql.

    reports:prune upto=9 unit=wk, deletes: 9k, time: 280s
    reports:prune upto=8 unit=wk, deletes: 11k, time: 250s
    reports:prune upto=7 unit=wk, deletes: 10k, time: 251s
    reports:prune upto=6 unit=wk, deletes: 12k, time: 239s
    reports:prune upto=5 unit=wk, deletes: 11k, time: 205s
    reports:prune upto=4 unit=wk, deletes: 11k, time: 180s
    reports:prune upto=2 unit=wk, deletes: 22k, time: 200s

    Running it nightly with a fully prune data set.
    reports:prune upto=2 unit=wk, deletes: 1.5k, time: 25s

    ** I made up all these numbers up, but the performance ratio should be
    roughly correct based on past experience. ymmv. **

    Also the Rake task will load the id of the records it plans to delete
    into memory. This can be quite large and is another reason to break it
    into smaller chunks.

    Ramin
    On 7/24/2013 2:08 PM, Felipe Salum wrote:
    I actually do for 2 weeks.

    rake RAILS_ENV=production reports:prune upto=2 unit=wk



    On Wed, Jul 24, 2013 at 2:03 PM, Ramin K wrote:

    On 7/24/2013 1:49 PM, Felipe Salum wrote:

    I had a friend helping me to debug and looks like it is taking
    forever
    to delete the entries from resource status table (a lot of entries).

    Is that something we can improve ?


    env RAILS_ENV=production rake reports:prune upto=5 unit=day
    env RAILS_ENV=production rake reports:prune:orphaned

    You can change the RAILS_ENV and number of day, weeks, months, etc
    to match your system and its needs.

    Sounds like you've never pruned so it's going to be slow going. Try
    walking it forward like so.

    env RAILS_ENV=production rake reports:prune upto=3 unit=mon
    env RAILS_ENV=production rake reports:prune upto=1 unit=mon
    env RAILS_ENV=production rake reports:prune upto=2 unit=wk
    env RAILS_ENV=production rake reports:prune upto=5 unit=day

    Don't forget to add this to your crontab with the retentions
    settings that best fit your system.

    Ramin

    --
    You received this message because you are subscribed to a topic in
    the Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/__topic/puppet-users/7-__sjp1Ef1p0/unsubscribe
    <https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe>.
    To unsubscribe from this group and all its topics, send an email to
    puppet-users+unsubscribe@__googlegroups.com
    To post to this group, send email to puppet-users@googlegroups.com
    Visit this group at http://groups.google.com/__group/puppet-users
    <http://groups.google.com/group/puppet-users>.
    For more options, visit https://groups.google.com/__groups/opt_out
    <https://groups.google.com/groups/opt_out>.



    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Felipe Salum at Jul 24, 2013 at 10:28 pm
    I understood, what I wanted to say is that I run a cron job every day for
    the 2 weeks prune.. and I have less than 2 weeks data in the database right
    now since I moved it recently to a new server.

    I'm running a optimize task, the resource_statuses is really big.

    On Wed, Jul 24, 2013 at 3:21 PM, Ramin K wrote:

    Doesn't matter, it's the size of the data and the amount you're
    deleting.

    Let's say that your reports:prune upto=2 unit=wk is ultimately going to
    delete 100k records because you've never pruned the database before.

    reports:prune upto=2 unit=wk, 100k, time: 3600s

    By breaking it up, it'll run faster because large deletes can cause poor
    performance in Mysql.

    reports:prune upto=9 unit=wk, deletes: 9k, time: 280s
    reports:prune upto=8 unit=wk, deletes: 11k, time: 250s
    reports:prune upto=7 unit=wk, deletes: 10k, time: 251s
    reports:prune upto=6 unit=wk, deletes: 12k, time: 239s
    reports:prune upto=5 unit=wk, deletes: 11k, time: 205s
    reports:prune upto=4 unit=wk, deletes: 11k, time: 180s
    reports:prune upto=2 unit=wk, deletes: 22k, time: 200s

    Running it nightly with a fully prune data set.
    reports:prune upto=2 unit=wk, deletes: 1.5k, time: 25s

    ** I made up all these numbers up, but the performance ratio should be
    roughly correct based on past experience. ymmv. **

    Also the Rake task will load the id of the records it plans to delete into
    memory. This can be quite large and is another reason to break it into
    smaller chunks.

    Ramin

    On 7/24/2013 2:08 PM, Felipe Salum wrote:

    I actually do for 2 weeks.

    rake RAILS_ENV=production reports:prune upto=2 unit=wk



    On Wed, Jul 24, 2013 at 2:03 PM, Ramin K <ramin-list@badapple.net
    <mailto:ramin-list@badapple.**net wrote:

    On 7/24/2013 1:49 PM, Felipe Salum wrote:

    I had a friend helping me to debug and looks like it is taking
    forever
    to delete the entries from resource status table (a lot of
    entries).

    Is that something we can improve ?


    env RAILS_ENV=production rake reports:prune upto=5 unit=day
    env RAILS_ENV=production rake reports:prune:orphaned

    You can change the RAILS_ENV and number of day, weeks, months, etc
    to match your system and its needs.

    Sounds like you've never pruned so it's going to be slow going. Try
    walking it forward like so.

    env RAILS_ENV=production rake reports:prune upto=3 unit=mon
    env RAILS_ENV=production rake reports:prune upto=1 unit=mon
    env RAILS_ENV=production rake reports:prune upto=2 unit=wk
    env RAILS_ENV=production rake reports:prune upto=5 unit=day

    Don't forget to add this to your crontab with the retentions
    settings that best fit your system.

    Ramin

    --
    You received this message because you are subscribed to a topic in
    the Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/__**topic/puppet-users/7-__**
    sjp1Ef1p0/unsubscribe<https://groups.google.com/d/__topic/puppet-users/7-__sjp1Ef1p0/unsubscribe>
    <https://groups.google.com/d/**topic/puppet-users/7-**
    sjp1Ef1p0/unsubscribe<https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe>
    .
    To unsubscribe from this group and all its topics, send an email to
    puppet-users+unsubscribe@__goo**glegroups.com<http://googlegroups.com>
    <mailto:puppet-users%**2Bunsubscribe@googlegroups.com<puppet-users%252Bunsubscribe@googlegroups.com>
    **>.

    To post to this group, send email to puppet-users@googlegroups.com
    <mailto:puppet-users@**googlegroups.com<puppet-users@googlegroups.com>
    .
    Visit this group at http://groups.google.com/__**group/puppet-users<http://groups.google.com/__group/puppet-users>
    <http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users>
    .
    For more options, visit https://groups.google.com/__**groups/opt_out<https://groups.google.com/__groups/opt_out>
    <https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .


    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send

    an email to puppet-users+unsubscribe@**googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
    .
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users>
    .
    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "Puppet Users" group.
    To unsubscribe from this topic, visit https://groups.google.com/d/**
    topic/puppet-users/7-**sjp1Ef1p0/unsubscribe<https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe>
    .
    To unsubscribe from this group and all its topics, send an email to
    puppet-users+unsubscribe@**googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
    .
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users>
    .
    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .

    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
    To post to this group, send email to puppet-users@googlegroups.com.
    Visit this group at http://groups.google.com/group/puppet-users.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedJul 24, '13 at 6:42p
activeJul 24, '13 at 10:28p
posts6
users2
websitepuppetlabs.com

2 users in discussion

Felipe Salum: 4 posts Ramin K: 2 posts

People

Translate

site design / logo © 2022 Grokbase