FAQ
Hi everyone,


We are using collectd collecting system information, I am trying to setup
the postgres plugin for puppetdb (
http://collectd.org/wiki/index.php/Plugin:PostgreSQL), what could be the
relevant value to get? I am trying to see what is going on because lately
sometime I get some "Sun Aug 05 10:39:12 +0200 2012 Puppet (err): Could not
retrieve catalog from remote server: Error 400 on SERVER: Failed to submit
'replace catalog' command for XXXXXXXXXX to PuppetDB at
puppetmaster.fqdn:8081: [500 Server Error] "


I am using puppet:
ii puppet 2.7.18-1puppetlabs1
Centralized configuration management - agent startup and compatibility
scripts
ii puppet-common 2.7.18-1puppetlabs1 Centralized
configuration management
ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard for
Puppet
ii puppetdb 0.9.2-1puppetlabs2 PuppetDB
Centralized Storage.
ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect Puppet
to PuppetDB by setting up a terminus for PuppetDB.
ii puppetmaster 2.7.18-1puppetlabs1 Centralized
configuration management - master startup and compatibility scripts
ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
common scripts


On debian squeeze for handling 120 server and the puppetmaster is a 1950
with 4go of memory and it doesn't seems to be under heavy load.

Regards,
JM

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to [email protected].
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Search Discussions

  • Chris Price at Aug 6, 2012 at 4:48 pm
    Hello A_SAAS,

    - Do you see any messages in the file "/var/log/puppetdb/puppetdb.log"?
    - If not, can you try editing the file "/etc/puppetdb/log4j.properties"
    and replace "INFO" with "DEBUG", and then see if you get any messages in
    the aforementioned log file?
    - What version of java are you using? We've encountered some
    SSL-related problems between very early versions of the Oracle JDK 7 and
    puppetdb.

    Thanks!
    Chris

    On Monday, August 6, 2012 1:51:07 AM UTC-7, A_SAAS wrote:

    Hi everyone,


    We are using collectd collecting system information, I am trying to setup
    the postgres plugin for puppetdb (
    http://collectd.org/wiki/index.php/Plugin:PostgreSQL), what could be the
    relevant value to get? I am trying to see what is going on because lately
    sometime I get some "Sun Aug 05 10:39:12 +0200 2012 Puppet (err): Could not
    retrieve catalog from remote server: Error 400 on SERVER: Failed to submit
    'replace catalog' command for XXXXXXXXXX to PuppetDB at
    puppetmaster.fqdn:8081: [500 Server Error] "


    I am using puppet:
    ii puppet 2.7.18-1puppetlabs1
    Centralized configuration management - agent startup and compatibility
    scripts
    ii puppet-common 2.7.18-1puppetlabs1 Centralized
    configuration management
    ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard for
    Puppet
    ii puppetdb 0.9.2-1puppetlabs2 PuppetDB
    Centralized Storage.
    ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect
    Puppet to PuppetDB by setting up a terminus for PuppetDB.
    ii puppetmaster 2.7.18-1puppetlabs1 Centralized
    configuration management - master startup and compatibility scripts
    ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
    common scripts


    On debian squeeze for handling 120 server and the puppetmaster is a 1950
    with 4go of memory and it doesn't seems to be under heavy load.

    Regards,
    JM
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/JHdLQ5zvobAJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Antidot SAS at Aug 7, 2012 at 7:58 am
    Here is the puppetdb.log:
    2012-08-05 10:39:12,431 WARN [qtp2101818118-104743]
    [server.AbstractHttpConnection] /commands
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2894)
    at
    java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117)
    at
    java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589)
    at java.lang.StringBuffer.append(StringBuffer.java:337)
    at java.net.URLDecoder.decode(URLDecoder.java:199)
    at ring.util.codec$form_decode_str.doInvoke(codec.clj:97)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.util.codec$form_decode$fn__5979.invoke(codec.clj:110)
    at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:60)
    at clojure.core.protocols$fn__5565.invoke(protocols.clj:30)
    at
    clojure.core.protocols$fn__5543$G__5538__5552.invoke(protocols.clj:11)
    at clojure.core$reduce.invoke(core.clj:5995)
    at ring.util.codec$form_decode.doInvoke(codec.clj:113)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.middleware.params$parse_params.invoke(params.clj:6)
    at ring.middleware.params$assoc_form_params.invoke(params.clj:29)
    at ring.middleware.params$wrap_params$fn__5994.invoke(params.clj:51)
    at
    com.puppetlabs.middleware$wrap_with_authorization$fn__7935.invoke(middleware.clj:20)
    at
    com.puppetlabs.middleware$wrap_with_certificate_cn$fn__7939.invoke(middleware.clj:35)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945$fn__7946.invoke(middleware.clj:60)
    at
    com.puppetlabs.middleware.proxy$java.lang.Object$Callable$f8c5758f.call(Unknown
    Source)
    at com.yammer.metrics.core.Timer.time(Timer.java:91)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945.invoke(middleware.clj:58)
    at
    com.puppetlabs.middleware$wrap_with_globals$fn__7942.invoke(middleware.clj:43)
    at ring.adapter.jetty$proxy_handler$fn__2028.invoke(jetty.clj:18)
    at
    ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown
    Source)
    at
    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:349)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894)
    at
    org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)


    Regarding the java version:
    ii openjdk-6-jre-headless
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime, using Hotspot JIT
    (headless)
    ii openjdk-6-jre-lib
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime (architecture
    independent libraries)

    Does it help?

    Regards,
    JM

    On Mon, Aug 6, 2012 at 6:40 PM, Chris Price wrote:

    Hello A_SAAS,

    - Do you see any messages in the file "/var/log/puppetdb/puppetdb.log"?
    - If not, can you try editing the file
    "/etc/puppetdb/log4j.properties" and replace "INFO" with "DEBUG", and then
    see if you get any messages in the aforementioned log file?
    - What version of java are you using? We've encountered some
    SSL-related problems between very early versions of the Oracle JDK 7 and
    puppetdb.

    Thanks!
    Chris

    On Monday, August 6, 2012 1:51:07 AM UTC-7, A_SAAS wrote:

    Hi everyone,


    We are using collectd collecting system information, I am trying to setup
    the postgres plugin for puppetdb (http://collectd.org/wiki/**
    index.php/Plugin:PostgreSQL<http://collectd.org/wiki/index.php/Plugin:PostgreSQL>),
    what could be the relevant value to get? I am trying to see what is going
    on because lately sometime I get some "Sun Aug 05 10:39:12 +0200 2012
    Puppet (err): Could not retrieve catalog from remote server: Error 400 on
    SERVER: Failed to submit 'replace catalog' command for XXXXXXXXXX to
    PuppetDB at puppetmaster.fqdn:8081: [500 Server Error] "


    I am using puppet:
    ii puppet 2.7.18-1puppetlabs1
    Centralized configuration management - agent startup and compatibility
    scripts
    ii puppet-common 2.7.18-1puppetlabs1 Centralized
    configuration management
    ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard
    for Puppet
    ii puppetdb 0.9.2-1puppetlabs2
    PuppetDB Centralized Storage.
    ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect
    Puppet to PuppetDB by setting up a terminus for PuppetDB.
    ii puppetmaster 2.7.18-1puppetlabs1
    Centralized configuration management - master startup and compatibility
    scripts
    ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
    common scripts


    On debian squeeze for handling 120 server and the puppetmaster is a 1950
    with 4go of memory and it doesn't seems to be under heavy load.

    Regards,
    JM
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/JHdLQ5zvobAJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to
    [email protected].
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Antidot SAS at Aug 7, 2012 at 3:03 pm
    In fact I am starting to have more and more errors regarding java exception:
    2012-08-07 16:39:23,864 ERROR [command-proc-77] [puppetdb.command] Fatal
    error parsing command java.lang.OutOfMemoryError: Java heap space
    2012-08-07 16:39:26,628 WARN [command-proc-77]
    [listener.DefaultMessageListenerContainer] Execution of JMS message
    listener failed, and no ErrorHandler has been
    set. java.lang.OutOfMemoryError: Java heap space

    Regards,
    JM


    On Tue, Aug 7, 2012 at 9:58 AM, Antidot SAS wrote:

    Here is the puppetdb.log:
    2012-08-05 10:39:12,431 WARN [qtp2101818118-104743]
    [server.AbstractHttpConnection] /commands
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2894)
    at
    java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117)
    at
    java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589)
    at java.lang.StringBuffer.append(StringBuffer.java:337)
    at java.net.URLDecoder.decode(URLDecoder.java:199)
    at ring.util.codec$form_decode_str.doInvoke(codec.clj:97)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.util.codec$form_decode$fn__5979.invoke(codec.clj:110)
    at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:60)
    at clojure.core.protocols$fn__5565.invoke(protocols.clj:30)
    at
    clojure.core.protocols$fn__5543$G__5538__5552.invoke(protocols.clj:11)
    at clojure.core$reduce.invoke(core.clj:5995)
    at ring.util.codec$form_decode.doInvoke(codec.clj:113)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.middleware.params$parse_params.invoke(params.clj:6)
    at ring.middleware.params$assoc_form_params.invoke(params.clj:29)
    at
    ring.middleware.params$wrap_params$fn__5994.invoke(params.clj:51)
    at
    com.puppetlabs.middleware$wrap_with_authorization$fn__7935.invoke(middleware.clj:20)
    at
    com.puppetlabs.middleware$wrap_with_certificate_cn$fn__7939.invoke(middleware.clj:35)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945$fn__7946.invoke(middleware.clj:60)
    at
    com.puppetlabs.middleware.proxy$java.lang.Object$Callable$f8c5758f.call(Unknown
    Source)
    at com.yammer.metrics.core.Timer.time(Timer.java:91)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945.invoke(middleware.clj:58)
    at
    com.puppetlabs.middleware$wrap_with_globals$fn__7942.invoke(middleware.clj:43)
    at ring.adapter.jetty$proxy_handler$fn__2028.invoke(jetty.clj:18)
    at
    ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown
    Source)
    at
    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:349)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894)
    at
    org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)


    Regarding the java version:
    ii openjdk-6-jre-headless
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime, using Hotspot JIT
    (headless)
    ii openjdk-6-jre-lib
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime (architecture
    independent libraries)

    Does it help?

    Regards,
    JM

    On Mon, Aug 6, 2012 at 6:40 PM, Chris Price wrote:

    Hello A_SAAS,

    - Do you see any messages in the file
    "/var/log/puppetdb/puppetdb.log"?
    - If not, can you try editing the file
    "/etc/puppetdb/log4j.properties" and replace "INFO" with "DEBUG", and then
    see if you get any messages in the aforementioned log file?
    - What version of java are you using? We've encountered some
    SSL-related problems between very early versions of the Oracle JDK 7 and
    puppetdb.

    Thanks!
    Chris

    On Monday, August 6, 2012 1:51:07 AM UTC-7, A_SAAS wrote:

    Hi everyone,


    We are using collectd collecting system information, I am trying to
    setup the postgres plugin for puppetdb (http://collectd.org/wiki/**
    index.php/Plugin:PostgreSQL<http://collectd.org/wiki/index.php/Plugin:PostgreSQL>),
    what could be the relevant value to get? I am trying to see what is going
    on because lately sometime I get some "Sun Aug 05 10:39:12 +0200 2012
    Puppet (err): Could not retrieve catalog from remote server: Error 400 on
    SERVER: Failed to submit 'replace catalog' command for XXXXXXXXXX to
    PuppetDB at puppetmaster.fqdn:8081: [500 Server Error] "


    I am using puppet:
    ii puppet 2.7.18-1puppetlabs1
    Centralized configuration management - agent startup and compatibility
    scripts
    ii puppet-common 2.7.18-1puppetlabs1
    Centralized configuration management
    ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard
    for Puppet
    ii puppetdb 0.9.2-1puppetlabs2
    PuppetDB Centralized Storage.
    ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect
    Puppet to PuppetDB by setting up a terminus for PuppetDB.
    ii puppetmaster 2.7.18-1puppetlabs1
    Centralized configuration management - master startup and compatibility
    scripts
    ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
    common scripts


    On debian squeeze for handling 120 server and the puppetmaster is a 1950
    with 4go of memory and it doesn't seems to be under heavy load.

    Regards,
    JM
    --
    You received this message because you are subscribed to the Google Groups
    "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/JHdLQ5zvobAJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to
    [email protected].
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Antidot SAS at Aug 7, 2012 at 3:13 pm
    If you wanna see the memory of the puppet server here it is.


    Regards,
    JM
    On Tue, Aug 7, 2012 at 5:03 PM, Antidot SAS wrote:

    In fact I am starting to have more and more errors regarding java
    exception:
    2012-08-07 16:39:23,864 ERROR [command-proc-77] [puppetdb.command] Fatal
    error parsing command java.lang.OutOfMemoryError: Java heap space
    2012-08-07 16:39:26,628 WARN [command-proc-77]
    [listener.DefaultMessageListenerContainer] Execution of JMS message
    listener failed, and no ErrorHandler has been
    set. java.lang.OutOfMemoryError: Java heap space

    Regards,
    JM


    On Tue, Aug 7, 2012 at 9:58 AM, Antidot SAS wrote:

    Here is the puppetdb.log:
    2012-08-05 10:39:12,431 WARN [qtp2101818118-104743]
    [server.AbstractHttpConnection] /commands
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2894)
    at
    java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117)
    at
    java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589)
    at java.lang.StringBuffer.append(StringBuffer.java:337)
    at java.net.URLDecoder.decode(URLDecoder.java:199)
    at ring.util.codec$form_decode_str.doInvoke(codec.clj:97)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.util.codec$form_decode$fn__5979.invoke(codec.clj:110)
    at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:60)
    at clojure.core.protocols$fn__5565.invoke(protocols.clj:30)
    at
    clojure.core.protocols$fn__5543$G__5538__5552.invoke(protocols.clj:11)
    at clojure.core$reduce.invoke(core.clj:5995)
    at ring.util.codec$form_decode.doInvoke(codec.clj:113)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.middleware.params$parse_params.invoke(params.clj:6)
    at ring.middleware.params$assoc_form_params.invoke(params.clj:29)
    at
    ring.middleware.params$wrap_params$fn__5994.invoke(params.clj:51)
    at
    com.puppetlabs.middleware$wrap_with_authorization$fn__7935.invoke(middleware.clj:20)
    at
    com.puppetlabs.middleware$wrap_with_certificate_cn$fn__7939.invoke(middleware.clj:35)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945$fn__7946.invoke(middleware.clj:60)
    at
    com.puppetlabs.middleware.proxy$java.lang.Object$Callable$f8c5758f.call(Unknown
    Source)
    at com.yammer.metrics.core.Timer.time(Timer.java:91)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945.invoke(middleware.clj:58)
    at
    com.puppetlabs.middleware$wrap_with_globals$fn__7942.invoke(middleware.clj:43)
    at ring.adapter.jetty$proxy_handler$fn__2028.invoke(jetty.clj:18)
    at
    ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown
    Source)
    at
    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:349)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894)
    at
    org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948)
    at
    org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)


    Regarding the java version:
    ii openjdk-6-jre-headless
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime, using Hotspot JIT
    (headless)
    ii openjdk-6-jre-lib
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime (architecture
    independent libraries)

    Does it help?

    Regards,
    JM

    On Mon, Aug 6, 2012 at 6:40 PM, Chris Price wrote:

    Hello A_SAAS,

    - Do you see any messages in the file
    "/var/log/puppetdb/puppetdb.log"?
    - If not, can you try editing the file
    "/etc/puppetdb/log4j.properties" and replace "INFO" with "DEBUG", and then
    see if you get any messages in the aforementioned log file?
    - What version of java are you using? We've encountered some
    SSL-related problems between very early versions of the Oracle JDK 7 and
    puppetdb.

    Thanks!
    Chris

    On Monday, August 6, 2012 1:51:07 AM UTC-7, A_SAAS wrote:

    Hi everyone,


    We are using collectd collecting system information, I am trying to
    setup the postgres plugin for puppetdb (http://collectd.org/wiki/**
    index.php/Plugin:PostgreSQL<http://collectd.org/wiki/index.php/Plugin:PostgreSQL>),
    what could be the relevant value to get? I am trying to see what is going
    on because lately sometime I get some "Sun Aug 05 10:39:12 +0200 2012
    Puppet (err): Could not retrieve catalog from remote server: Error 400 on
    SERVER: Failed to submit 'replace catalog' command for XXXXXXXXXX to
    PuppetDB at puppetmaster.fqdn:8081: [500 Server Error] "


    I am using puppet:
    ii puppet 2.7.18-1puppetlabs1
    Centralized configuration management - agent startup and compatibility
    scripts
    ii puppet-common 2.7.18-1puppetlabs1
    Centralized configuration management
    ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard
    for Puppet
    ii puppetdb 0.9.2-1puppetlabs2
    PuppetDB Centralized Storage.
    ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect
    Puppet to PuppetDB by setting up a terminus for PuppetDB.
    ii puppetmaster 2.7.18-1puppetlabs1
    Centralized configuration management - master startup and compatibility
    scripts
    ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
    common scripts


    On debian squeeze for handling 120 server and the puppetmaster is a
    1950 with 4go of memory and it doesn't seems to be under heavy load.

    Regards,
    JM
    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/JHdLQ5zvobAJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to
    [email protected].
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Chris Price at Aug 7, 2012 at 4:59 pm
    JM,

    Hmm... so, those versions of Java seem fine--those are probably the ones
    we've done the most testing with.

    So, the error message that you sent indicates that the JVM is running out
    of RAM. This could possibly indicate that you are still using the embedded
    database instead of postgres--the embedded database uses a lot of RAM. Can
    we see what your database.ini file looks like? In fact, perhaps we can
    take a look at all of the files in your /etc/puppetdb/conf.d directory?

    You could also connect directly to your postgres database using psql or
    pgadmin, and check to see if the puppetdb tables exist and contain data.

    The only other possibility that I can think of would be if one or more of
    your nodes has an extremely large catalog, and that the puppetdb JVM
    instance doesn't have enough memory to process the catalog. If that's the
    case, we can try increasing your JVM max heap space in
    /etc/default/puppetdb... but we would be really interested in collecting
    some data from you about what that catalog looks like (for our own testing
    and debugging purposes) if it turns out that that is the culprit.

    Thanks!
    Chris
    On Tuesday, August 7, 2012 8:13:15 AM UTC-7, A_SAAS wrote:

    If you wanna see the memory of the puppet server here it is.


    Regards,
    JM
    On Tue, Aug 7, 2012 at 5:03 PM, Antidot SAS wrote:

    In fact I am starting to have more and more errors regarding java
    exception:
    2012-08-07 16:39:23,864 ERROR [command-proc-77] [puppetdb.command] Fatal
    error parsing command java.lang.OutOfMemoryError: Java heap space
    2012-08-07 16:39:26,628 WARN [command-proc-77]
    [listener.DefaultMessageListenerContainer] Execution of JMS message
    listener failed, and no ErrorHandler has been
    set. java.lang.OutOfMemoryError: Java heap space

    Regards,
    JM


    On Tue, Aug 7, 2012 at 9:58 AM, Antidot SAS wrote:

    Here is the puppetdb.log:
    2012-08-05 10:39:12,431 WARN [qtp2101818118-104743]
    [server.AbstractHttpConnection] /commands
    java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2894)
    at
    java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117)
    at
    java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589)
    at java.lang.StringBuffer.append(StringBuffer.java:337)
    at java.net.URLDecoder.decode(URLDecoder.java:199)
    at ring.util.codec$form_decode_str.doInvoke(codec.clj:97)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.util.codec$form_decode$fn__5979.invoke(codec.clj:110)
    at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:60)
    at clojure.core.protocols$fn__5565.invoke(protocols.clj:30)
    at
    clojure.core.protocols$fn__5543$G__5538__5552.invoke(protocols.clj:11)
    at clojure.core$reduce.invoke(core.clj:5995)
    at ring.util.codec$form_decode.doInvoke(codec.clj:113)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at ring.middleware.params$parse_params.invoke(params.clj:6)
    at ring.middleware.params$assoc_form_params.invoke(params.clj:29)
    at
    ring.middleware.params$wrap_params$fn__5994.invoke(params.clj:51)
    at
    com.puppetlabs.middleware$wrap_with_authorization$fn__7935.invoke(middleware.clj:20)
    at
    com.puppetlabs.middleware$wrap_with_certificate_cn$fn__7939.invoke(middleware.clj:35)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945$fn__7946.invoke(middleware.clj:60)
    at
    com.puppetlabs.middleware.proxy$java.lang.Object$Callable$f8c5758f.call(Unknown
    Source)
    at com.yammer.metrics.core.Timer.time(Timer.java:91)
    at
    com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__7945.invoke(middleware.clj:58)
    at
    com.puppetlabs.middleware$wrap_with_globals$fn__7942.invoke(middleware.clj:43)
    at ring.adapter.jetty$proxy_handler$fn__2028.invoke(jetty.clj:18)
    at
    ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown
    Source)
    at
    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:349)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
    at
    org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894)
    at
    org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948)
    at
    org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)


    Regarding the java version:
    ii openjdk-6-jre-headless
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime, using Hotspot JIT
    (headless)
    ii openjdk-6-jre-lib
    6b18-1.8.13-0+squeeze2 OpenJDK Java runtime (architecture
    independent libraries)

    Does it help?

    Regards,
    JM

    On Mon, Aug 6, 2012 at 6:40 PM, Chris Price wrote:

    Hello A_SAAS,

    - Do you see any messages in the file
    "/var/log/puppetdb/puppetdb.log"?
    - If not, can you try editing the file
    "/etc/puppetdb/log4j.properties" and replace "INFO" with "DEBUG", and then
    see if you get any messages in the aforementioned log file?
    - What version of java are you using? We've encountered some
    SSL-related problems between very early versions of the Oracle JDK 7 and
    puppetdb.

    Thanks!
    Chris

    On Monday, August 6, 2012 1:51:07 AM UTC-7, A_SAAS wrote:

    Hi everyone,


    We are using collectd collecting system information, I am trying to
    setup the postgres plugin for puppetdb (http://collectd.org/wiki/**
    index.php/Plugin:PostgreSQL<http://collectd.org/wiki/index.php/Plugin:PostgreSQL>),
    what could be the relevant value to get? I am trying to see what is going
    on because lately sometime I get some "Sun Aug 05 10:39:12 +0200 2012
    Puppet (err): Could not retrieve catalog from remote server: Error 400 on
    SERVER: Failed to submit 'replace catalog' command for XXXXXXXXXX to
    PuppetDB at puppetmaster.fqdn:8081: [500 Server Error] "


    I am using puppet:
    ii puppet 2.7.18-1puppetlabs1
    Centralized configuration management - agent startup and compatibility
    scripts
    ii puppet-common 2.7.18-1puppetlabs1
    Centralized configuration management
    ii puppet-dashboard 1.2.10-1puppetlabs2 Dashboard
    for Puppet
    ii puppetdb 0.9.2-1puppetlabs2
    PuppetDB Centralized Storage.
    ii puppetdb-terminus 0.9.2-1puppetlabs2 Connect
    Puppet to PuppetDB by setting up a terminus for PuppetDB.
    ii puppetmaster 2.7.18-1puppetlabs1
    Centralized configuration management - master startup and compatibility
    scripts
    ii puppetmaster-common 2.7.18-1puppetlabs1 Puppet master
    common scripts


    On debian squeeze for handling 120 server and the puppetmaster is a
    1950 with 4go of memory and it doesn't seems to be under heavy load.

    Regards,
    JM
    --
    You received this message because you are subscribed to the Google
    Groups "Puppet Users" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/puppet-users/-/JHdLQ5zvobAJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to
    [email protected].
    For more options, visit this group at
    http://groups.google.com/group/puppet-users?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/eoIR0s_RTDQJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Antidot SAS at Aug 8, 2012 at 8:08 am
    Hi,

    The configuration files are attached.

    Here is the postgres login:
    [root@puppetmaster]:/data/local/postgresql/dumps # psql -h localhost
    puppetdb puppetdb
    psql (8.4.12)
    SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
    Type "help" for help.

    puppetdb=> \l
    List of databases
    Name | Owner | Encoding | Collation | Ctype | Access privileges
    -----------+----------+-----------+-----------+-------+-----------------------
    postgres | postgres | SQL_ASCII | C | C |
    puppetdb | puppetdb | SQL_ASCII | C | C |
    template0 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    template1 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    (4 rows)
    puppetdb=> \d
    List of relations
    Schema | Name | Type | Owner
    --------+-------------------------+-------+----------
    public | catalog_resources | table | puppetdb
    public | catalogs | table | puppetdb
    public | certname_catalogs | table | puppetdb
    public | certname_facts | table | puppetdb
    public | certname_facts_metadata | table | puppetdb
    public | certnames | table | puppetdb
    public | classes | table | puppetdb
    public | edges | table | puppetdb
    public | resource_params | table | puppetdb
    public | schema_migrations | table | puppetdb
    public | tags | table | puppetdb


    And by the way I changed the JVM max heap at 256M and since then no issue.
    Regarding the postgres dump file, I'll be willing to give it to you so
    contact me in private to get the URL or let me know where I can put it as
    long as nobody could access it publicly.


    Regards,
    JM



    On Tue, Aug 7, 2012 at 6:59 PM, Chris Price wrote:

    JM,

    Hmm... so, those versions of Java seem fine--those are probably the ones
    we've done the most testing with.

    So, the error message that you sent indicates that the JVM is running out
    of RAM. This could possibly indicate that you are still using the embedded
    database instead of postgres--the embedded database uses a lot of RAM. Can
    we see what your database.ini file looks like? In fact, perhaps we can
    take a look at all of the files in your /etc/puppetdb/conf.d directory?

    You could also connect directly to your postgres database using psql or
    pgadmin, and check to see if the puppetdb tables exist and contain data.

    The only other possibility that I can think of would be if one or more of
    your nodes has an extremely large catalog, and that the puppetdb JVM
    instance doesn't have enough memory to process the catalog. If that's the
    case, we can try increasing your JVM max heap space in
    /etc/default/puppetdb... but we would be really interested in collecting
    some data from you about what that catalog looks like (for our own testing
    and debugging purposes) if it turns out that that is the culprit.

    Thanks!
    Chris
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
  • Chris Price at Aug 8, 2012 at 4:49 pm
    Thanks JM. Your config files look good and the existence of all of those
    tables in your puppetdb database certainly makes it look as though puppetdb
    is communicating with postgres properly. Since increasing the heap size
    seems to have gotten you past the issues for now, my next guess is that it
    is some combination of catalog size and number of concurrent agent
    requests, and perhaps number of cores on your machine. We would love to
    investigate further, so if you are willing to share a postgres dump with us
    that would be fantastic!

    I'll e-mail you off-list to follow up on that. I'd also be interested in
    seeing the output of 'cat /proc/cpuinfo'.

    Thanks!
    Chris
    On Wednesday, August 8, 2012 1:08:09 AM UTC-7, A_SAAS wrote:

    Hi,

    The configuration files are attached.

    Here is the postgres login:
    [root@puppetmaster]:/data/local/postgresql/dumps # psql -h localhost
    puppetdb puppetdb
    psql (8.4.12)
    SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
    Type "help" for help.

    puppetdb=> \l
    List of databases
    Name | Owner | Encoding | Collation | Ctype | Access privileges

    -----------+----------+-----------+-----------+-------+-----------------------
    postgres | postgres | SQL_ASCII | C | C |
    puppetdb | puppetdb | SQL_ASCII | C | C |
    template0 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    template1 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    (4 rows)
    puppetdb=> \d
    List of relations
    Schema | Name | Type | Owner
    --------+-------------------------+-------+----------
    public | catalog_resources | table | puppetdb
    public | catalogs | table | puppetdb
    public | certname_catalogs | table | puppetdb
    public | certname_facts | table | puppetdb
    public | certname_facts_metadata | table | puppetdb
    public | certnames | table | puppetdb
    public | classes | table | puppetdb
    public | edges | table | puppetdb
    public | resource_params | table | puppetdb
    public | schema_migrations | table | puppetdb
    public | tags | table | puppetdb


    And by the way I changed the JVM max heap at 256M and since then no issue.
    Regarding the postgres dump file, I'll be willing to give it to you so
    contact me in private to get the URL or let me know where I can put it as
    long as nobody could access it publicly.


    Regards,
    JM




    On Tue, Aug 7, 2012 at 6:59 PM, Chris Price <[email protected]<javascript:>
    wrote:
    JM,

    Hmm... so, those versions of Java seem fine--those are probably the ones
    we've done the most testing with.

    So, the error message that you sent indicates that the JVM is running out
    of RAM. This could possibly indicate that you are still using the embedded
    database instead of postgres--the embedded database uses a lot of RAM. Can
    we see what your database.ini file looks like? In fact, perhaps we can
    take a look at all of the files in your /etc/puppetdb/conf.d directory?

    You could also connect directly to your postgres database using psql or
    pgadmin, and check to see if the puppetdb tables exist and contain data.

    The only other possibility that I can think of would be if one or more of
    your nodes has an extremely large catalog, and that the puppetdb JVM
    instance doesn't have enough memory to process the catalog. If that's the
    case, we can try increasing your JVM max heap space in
    /etc/default/puppetdb... but we would be really interested in collecting
    some data from you about what that catalog looks like (for our own testing
    and debugging purposes) if it turns out that that is the culprit.

    Thanks!
    Chris
    On Wednesday, August 8, 2012 1:08:09 AM UTC-7, A_SAAS wrote:

    Hi,

    The configuration files are attached.

    Here is the postgres login:
    [root@puppetmaster]:/data/local/postgresql/dumps # psql -h localhost
    puppetdb puppetdb
    psql (8.4.12)
    SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
    Type "help" for help.

    puppetdb=> \l
    List of databases
    Name | Owner | Encoding | Collation | Ctype | Access privileges

    -----------+----------+-----------+-----------+-------+-----------------------
    postgres | postgres | SQL_ASCII | C | C |
    puppetdb | puppetdb | SQL_ASCII | C | C |
    template0 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    template1 | postgres | SQL_ASCII | C | C | =c/postgres
    :
    postgres=CTc/postgres
    (4 rows)
    puppetdb=> \d
    List of relations
    Schema | Name | Type | Owner
    --------+-------------------------+-------+----------
    public | catalog_resources | table | puppetdb
    public | catalogs | table | puppetdb
    public | certname_catalogs | table | puppetdb
    public | certname_facts | table | puppetdb
    public | certname_facts_metadata | table | puppetdb
    public | certnames | table | puppetdb
    public | classes | table | puppetdb
    public | edges | table | puppetdb
    public | resource_params | table | puppetdb
    public | schema_migrations | table | puppetdb
    public | tags | table | puppetdb


    And by the way I changed the JVM max heap at 256M and since then no issue.
    Regarding the postgres dump file, I'll be willing to give it to you so
    contact me in private to get the URL or let me know where I can put it as
    long as nobody could access it publicly.


    Regards,
    JM




    On Tue, Aug 7, 2012 at 6:59 PM, Chris Price <[email protected]<javascript:>
    wrote:
    JM,

    Hmm... so, those versions of Java seem fine--those are probably the ones
    we've done the most testing with.

    So, the error message that you sent indicates that the JVM is running out
    of RAM. This could possibly indicate that you are still using the embedded
    database instead of postgres--the embedded database uses a lot of RAM. Can
    we see what your database.ini file looks like? In fact, perhaps we can
    take a look at all of the files in your /etc/puppetdb/conf.d directory?

    You could also connect directly to your postgres database using psql or
    pgadmin, and check to see if the puppetdb tables exist and contain data.

    The only other possibility that I can think of would be if one or more of
    your nodes has an extremely large catalog, and that the puppetdb JVM
    instance doesn't have enough memory to process the catalog. If that's the
    case, we can try increasing your JVM max heap space in
    /etc/default/puppetdb... but we would be really interested in collecting
    some data from you about what that catalog looks like (for our own testing
    and debugging purposes) if it turns out that that is the culprit.

    Thanks!
    Chris
    --
    You received this message because you are subscribed to the Google Groups "Puppet Users" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zwXstXYZfFgJ.
    To post to this group, send email to [email protected].
    To unsubscribe from this group, send email to [email protected].
    For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppuppet-users @
categoriespuppet
postedAug 6, '12 at 8:51a
activeAug 8, '12 at 4:49p
posts8
users2
websitepuppetlabs.com

2 users in discussion

Antidot SAS: 5 posts Chris Price: 3 posts

People

Translate

site design / logo © 2023 Grokbase