FAQ
We have a CDH4 cluster that I want to monitor with Nagios. What should I
monitor with it?
I have basic checks for disk space and whether servers are up. But what
about Cloudera specific checks, like monitoring services on the Named node
etc?

Search Discussions

  • Hari Sekhon at Jun 27, 2013 at 12:36 pm
    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/nagios-plugins

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-replicated) / Available DataNodes / Available +
    Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should I
    monitor with it?
    I have basic checks for disk space and whether servers are up. But what
    about Cloudera specific checks, like monitoring services on the Named node
    etc?
  • Matthew Jacobs at Jun 27, 2013 at 4:31 pm
    Hi Jaykay,

    If you're using Cloudera Manager, you might want to check out the Nagios
    integration API sample on the CM API github:
    https://github.com/cloudera/cm_api/tree/master/nagios

    You can use the sample script to provide health summary information from
    Cloudera Manager to a Nagios external commands file. There's a README.md to
    help you get started. The script provided is just a sample, so you may want
    to extend it or use it as a reference in building a more specific solution.

    Thanks,
    mj

    On Thu, Jun 27, 2013 at 5:35 AM, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/nagios-plugins

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-replicated) / Available DataNodes / Available +
    Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should I
    monitor with it?
    I have basic checks for disk space and whether servers are up. But what
    about Cloudera specific checks, like monitoring services on the Named node
    etc?
  • Hari Sekhon at Jul 1, 2013 at 5:40 pm
    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the given
    --cluster --service / --hostid etc so if you don't know what you want to
    monitor you can see what's available for the given section (cluster service
    / role / namespace / host)

    check_hadoop_cloudera_manager_metrics.pl

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/nagios-plugins

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-replicated) / Available DataNodes / Available +
    Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should I
    monitor with it?
    I have basic checks for disk space and whether servers are up. But what
    about Cloudera specific checks, like monitoring services on the Named node
    etc?
  • Jaykay at Jul 2, 2013 at 9:25 am
    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just the
    standard install steps for any Nagios plugin?
    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the given
    --cluster --service / --hostid etc so if you don't know what you want to
    monitor you can see what's available for the given section (cluster service
    / role / namespace / host)

    check_hadoop_cloudera_manager_metrics.pl

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 27 June 2013 13:35, Hari Sekhon <ha...@cloudera.com <javascript:>>wrote:
    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/nagios-plugins

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-replicated) / Available DataNodes / Available +
    Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 27 June 2013 13:15, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should I
    monitor with it?
    I have basic checks for disk space and whether servers are up. But what
    about Cloudera specific checks, like monitoring services on the Named node
    etc?
  • Hari Sekhon at Jul 2, 2013 at 8:51 pm
    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/harisekhon/nagios-plugins.git
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_manager_metrics.pl --help

    You can also then move this nagios-plugins directory where ever you want on
    your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at the
    request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just the
    standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_**metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replicated) / Available DataNodes / Available
    + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should
    I monitor with it?
    I have basic checks for disk space and whether servers are up. But what
    about Cloudera specific checks, like monitoring services on the Named node
    etc?
  • Jaykay at Jul 4, 2013 at 9:38 am
    Much appreciated Hari. I am in the process of installing now, so I'll let
    you know any feedback.
    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/harisekhon/nagios-plugins.git
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_manager_metrics.pl --help

    You can also then move this nagios-plugins directory where ever you want
    on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at the
    request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 2 July 2013 10:25, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just the
    standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_**metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replicated) / Available DataNodes / Available
    + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should
    I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Jaykay at Jul 4, 2013 at 9:56 am
    When I try to run ./check_hadoop_cloudera_manager_metrics.pl --help, I get
    this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log for
    that be stored?
    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/harisekhon/nagios-plugins.git
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_manager_metrics.pl --help

    You can also then move this nagios-plugins directory where ever you want
    on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at the
    request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 2 July 2013 10:25, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just the
    standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_**metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replicated) / Available DataNodes / Available
    + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise node list
    matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What should
    I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Hari Sekhon at Jul 8, 2013 at 7:54 pm
    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you tried
    to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log for
    that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/**harisekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you want
    on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at the
    request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just the
    standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM monitoring
    integration which was on my todo list anyway so I took a few hours at the
    weekend and wrote a new Nagios plugin to do that using the Cloudera Manager
    API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_****metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in my
    Nagios Plugins collection:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica**ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Hari Sekhon at Jul 9, 2013 at 3:22 am
    Incidentally in case anyone else is interested, tonight I've just updated
    all the hadoop dfs based plugins for HDFS space, replication, % balance,
    datanodes available/dead etc as well as the
    check_hadoop_jobtracker.plplugin for tasktracker counts, expected
    listed, blacklisted tasktrackers,
    JT heap used to work on modern CDH since I originally wrote them on old
    vanilla Hadoop 0.20.x and they needed updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin which
    can pull metrics directly from hadoop JobTracker and TaskTracker daemons by
    parsing the metrics pages. Can take one metric with thresholds, or multiple
    metrics, or just get all metrics which is useful if you don't know what to
    monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you tried
    to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log for
    that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/**harisekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you want
    on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at the
    request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_****metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in
    my Nagios Plugins collection:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica**ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Jaykay at Jul 10, 2013 at 7:52 am
    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just updated
    all the hadoop dfs based plugins for HDFS space, replication, % balance,
    datanodes available/dead etc as well as the check_hadoop_jobtracker.plplugin for tasktracker counts, expected listed, blacklisted tasktrackers,
    JT heap used to work on modern CDH since I originally wrote them on old
    vanilla Hadoop 0.20.x and they needed updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin which
    can pull metrics directly from hadoop JobTracker and TaskTracker daemons by
    parsing the metrics pages. Can take one metric with thresholds, or multiple
    metrics, or just get all metrics which is useful if you don't know what to
    monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon


    On 8 July 2013 20:53, Hari Sekhon <ha...@cloudera.com <javascript:>>wrote:
    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you tried
    to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 4 July 2013 10:56, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    When I try to run ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a fairly
    basic setup requirement:

    git clone https://github.com/**harisekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_****metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in
    my Nagios Plugins collection:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica**ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Jaykay at Jul 10, 2013 at 10:29 am
    Hi Hari,

    So I have the plugins imported and I can see what switches to use with
    them. But how do I integrate it with Nagios, where do I call the script
    from? Do I need to modify nrpe.cfg and add commands in there to monitor
    services?
    Eg I could add:

    command[check_hadoop_cloudera_manager_metrics]=/root/nagios-plugins/check_hadoop_cloudera_manager_metrics.pl
    -C cdh4 -S mapreduce -w

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just updated
    all the hadoop dfs based plugins for HDFS space, replication, % balance,
    datanodes available/dead etc as well as the check_hadoop_jobtracker.plplugin for tasktracker counts, expected listed, blacklisted tasktrackers,
    JT heap used to work on modern CDH since I originally wrote them on old
    vanilla Hadoop 0.20.x and they needed updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**harisekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_****metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in
    my Nagios Plugins collection:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica**ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Jaykay at Jul 10, 2013 at 11:04 am
    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180': 503
    Service Unavailable. Message returned by CM: This feature is only available
    in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?
    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just updated
    all the hadoop dfs based plugins for HDFS space, replication, % balance,
    datanodes available/dead etc as well as the check_hadoop_jobtracker.plplugin for tasktracker counts, expected listed, blacklisted tasktrackers,
    JT heap used to work on modern CDH since I originally wrote them on old
    vanilla Hadoop 0.20.x and they needed updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/nagios-plugins

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**harisekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running this
    command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manager_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how many
    different metrics it collects, so this opens up a huge amount of monitoring
    information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve many
    metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for the
    given --cluster --service / --hostid etc so if you don't know what you want
    to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_****metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here in
    my Nagios Plugins collection:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica**ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up. But
    what about Cloudera specific checks, like monitoring services on the Named
    node etc?
  • Hari Sekhon at Jul 10, 2013 at 11:38 am
    That's the only plugin that uses the CM API. You can still use the other
    check_hadoop* plugins, you won't have as many vast things to monitor as CM
    does but it'll cover a lot of different basics that I used to monitor as a
    Hadoop administrator like hdfs space, replication, balance, datanodes,
    tasktrackers, and I recently wrote a generic metrics collection plugin
    which will work on JobTracker and TaskTrackers metrics pages to allow you
    to collect MapReduce and a few GC stats.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 10 July 2013 12:04, Jaykay wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris**ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_******metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica****ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?
  • Jaykay at Jul 10, 2013 at 3:28 pm
    OK, I'm trying check_hadoop_datanodes.pl. I want to use it to check for the
    number of available nodes. It is complaining that hadoop is not in $PATH. I
    can that is set in the script $ENV{"PATH"} .=
    ":/opt/hadoop/bin:/usr/local/hadoop/bin";
    My understanding is that I have the plugins on the nagios server (which is
    also the cloudera manager server). It then consults the named node. So the
    $PATH to hadoop should be set on the named node?
    On Wednesday, July 10, 2013 12:37:42 PM UTC+1, Hari Sekhon wrote:

    That's the only plugin that uses the CM API. You can still use the other
    check_hadoop* plugins, you won't have as many vast things to monitor as CM
    does but it'll cover a lot of different basics that I used to monitor as a
    Hadoop administrator like hdfs space, replication, balance, datanodes,
    tasktrackers, and I recently wrote a generic metrics collection plugin
    which will work on JobTracker and TaskTrackers metrics pages to allow you
    to collect MapReduce and a few GC stats.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 10 July 2013 12:04, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris**ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it
    just the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_******metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica****ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?
  • Hari Sekhon at Jul 10, 2013 at 5:49 pm
    The plugins linking to check_hadoop_dfs.pl need to be run on a correctly
    configured hadoop node with a user account having permissions to run a
    hadoop dfsadmin -report (hdfs user in CDH or any user with hdfs kerberos
    principal in a kerberos secured cluster). It obviously also needs either
    the "hadoop" or newer "hdfs" command to be in the in the $PATH on the
    system; PATH is set to /bin:/usr/bin for security reasons as well as having
    appended /opt/hadoop/bin and /usr/local/hadoop/bin which are 2 other
    commonly used paths in old manual Apache installs like I used to run. If
    you've deployed standard CDH then the hdfs command will be found in
    /usr/bin.

    In this case if you're running this on the CM node and the CM node doesn't
    have at least the HDFS Gateway service deployed it won't work since you
    haven't set up HDFS in the first place on that system. You can either
    deploy HDFS Gateway service to the CM node or copy the nagios-plugins
    directory to a hadoop cluster node and call via NRPE.

    The check_hadoop_namenode.pl and check_hadoop_jobtracker.pl plugins will
    work over the network via JSP, but where functionality overlaps between
    check_hadoop_namenode.pl --help and check_hadoop_dfs.pl --help I strongly
    prefer the check_hadoop_dfs.pl plugin since I used it much more in
    production and it'll handle variation across different hadoop version
    changes better.

    Since this has now diverged from CM specific to general Nagios + CDH/Hadoop
    we should probably move any further follow ups to the cdh-users and/or
    nagios-users mailing lists and I'll follow up on there.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 10 July 2013 16:28, Jaykay wrote:

    OK, I'm trying check_hadoop_datanodes.pl. I want to use it to check for
    the number of available nodes. It is complaining that hadoop is not in
    $PATH. I can that is set in the script $ENV{"PATH"} .=
    ":/opt/hadoop/bin:/usr/local/hadoop/bin";
    My understanding is that I have the plugins on the nagios server (which is
    also the cloudera manager server). It then consults the named node. So the
    $PATH to hadoop should be set on the named node?
    On Wednesday, July 10, 2013 12:37:42 PM UTC+1, Hari Sekhon wrote:

    That's the only plugin that uses the CM API. You can still use the other
    check_hadoop* plugins, you won't have as many vast things to monitor as CM
    does but it'll cover a lot of different basics that I used to monitor as a
    Hadoop administrator like hdfs space, replication, balance, datanodes,
    tasktrackers, and I recently wrote a generic metrics collection plugin
    which will work on JobTracker and TaskTrackers metrics pages to allow you
    to collect MapReduce and a few GC stats.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 10 July 2013 12:04, Jaykay wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working.
    The problem was I didn't have the development tools installed. I had git
    and make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that
    would help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage****r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the
    log for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris****ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage****r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening
    at the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it
    just the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_********metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/********nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har******isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/********nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica******ted) / Available
    DataNodes / Available + Blacklisted TaskTrackers, JobTracker & NameNode
    Heap, precise node list matches (which you can dynamically generate in
    Nagios), ZooKeeper mode/writeable/latency, as well as some HDFS, MR and
    ZooKeeper metrics for Nagios integrated graphing. A couple may need
    updating, let me know as I've been meaning to do that anyway. They all come
    with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har******isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?
  • Hari Sekhon at Jul 11, 2013 at 4:17 pm
    Hi Jaykay,

    I checked with the CM devs that Monitoring API should be available in CM
    Standard Edition 4.6 and have just confirmed by querying CM 4.6 using the
    CM metrics plugin. I suggest you upgrade your CM installation to 4.6 to get
    access to it, it's easy to do and will open up a lot of monitoring options
    for you.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 10 July 2013 12:04, Jaykay wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris**ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it just
    the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_******metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica****ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?
  • Jaykay at Jul 16, 2013 at 1:53 pm
    Thanks for the tip. 2 questions:
    Does this apply to the free version too?
    Does an upgrade of cloudera manager update the cloudera version on the
    whole cluster?
    On Thursday, July 11, 2013 5:16:40 PM UTC+1, Hari Sekhon wrote:

    Hi Jaykay,

    I checked with the CM devs that Monitoring API should be available in CM
    Standard Edition 4.6 and have just confirmed by querying CM 4.6 using the
    CM metrics plugin. I suggest you upgrade your CM installation to 4.6 to get
    access to it, it's easy to do and will open up a lot of monitoring options
    for you.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/harisekhon

    On 10 July 2013 12:04, Jaykay <johnk...@gmail.com <javascript:>> wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working. The
    problem was I didn't have the development tools installed. I had git and
    make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/**nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that would
    help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the log
    for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris**ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage**r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening at
    the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it
    just the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_******metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/******nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica****ted) / Available DataNodes /
    Available + Blacklisted TaskTrackers, JobTracker & NameNode Heap, precise
    node list matches (which you can dynamically generate in Nagios), ZooKeeper
    mode/writeable/latency, as well as some HDFS, MR and ZooKeeper metrics for
    Nagios integrated graphing. A couple may need updating, let me know as I've
    been meaning to do that anyway. They all come with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?
  • Darren Lo at Jul 16, 2013 at 5:35 pm
    Hi Jaykay,

    The Standard Edition in 4.6 is free and has these APIs.
    Upgrading Cloudera Manager does not upgrade your CDH versions. You can
    upgrade Cloudera Manager and CDH separately.

    Thanks,
    Darren

    On Tue, Jul 16, 2013 at 6:53 AM, Jaykay wrote:

    Thanks for the tip. 2 questions:
    Does this apply to the free version too?
    Does an upgrade of cloudera manager update the cloudera version on the
    whole cluster?

    On Thursday, July 11, 2013 5:16:40 PM UTC+1, Hari Sekhon wrote:

    Hi Jaykay,

    I checked with the CM devs that Monitoring API should be available in CM
    Standard Edition 4.6 and have just confirmed by querying CM 4.6 using the
    CM metrics plugin. I suggest you upgrade your CM installation to 4.6 to get
    access to it, it's easy to do and will open up a lot of monitoring options
    for you.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**harisekhon<http://www.linkedin.com/in/harisekhon>

    On 10 July 2013 12:04, Jaykay wrote:

    I am playing with the plugins, but I get the message:
    CRITICAL: failed to query Cloudera Manager at 'http://localhost:7180':
    503 Service Unavailable. Message returned by CM: This feature is only
    available in Cloudera Manager Enterprise.

    I only have the free version of Cloudera Manager. Will none of these
    plugins work with the free version?

    On Wednesday, July 10, 2013 8:52:28 AM UTC+1, Jaykay wrote:

    Hi Hari,

    Thanks again for your help. Managed to get the CPAN command working.
    The problem was I didn't have the development tools installed. I had git
    and make, but not the tools, so I had to run

    yum groupinstall 'Development Tools'
    I can use your plugins ok now.

    Thanks,
    John

    On Tuesday, July 9, 2013 4:21:27 AM UTC+1, Hari Sekhon wrote:

    Incidentally in case anyone else is interested, tonight I've just
    updated all the hadoop dfs based plugins for HDFS space, replication, %
    balance, datanodes available/dead etc as well as the
    check_hadoop_jobtracker.pl plugin for tasktracker counts, expected
    listed, blacklisted tasktrackers, JT heap used to work on modern CDH since
    I originally wrote them on old vanilla Hadoop 0.20.x and they needed
    updates.

    I've also just written a new generic check_hadoop_metrics.pl plugin
    which can pull metrics directly from hadoop JobTracker and TaskTracker
    daemons by parsing the metrics pages. Can take one metric with thresholds,
    or multiple metrics, or just get all metrics which is useful if you don't
    know what to monitor yet.

    The Cloudera Manager plugin is still the best thing to use if possible
    since it uses the CM API and is by far the most comprehensive in terms of
    metrics and monitoring, far beyond what core Hadoop itself can provide with
    metrics pages, but I know not everyone has CM yet.

    The new and updated plugins are available on github:

    https://github.com/harisekhon/****nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 8 July 2013 20:53, Hari Sekhon wrote:

    Hi Jaykay,

    The 2 most common errors I see are either you weren't root when you
    tried to install the CPAN module, or you were root and your umask was too
    restrictive so the module was installed with too restrictive permissions
    for user level accounts such as Nagios to read it.

    If you can paste the output of the cpan command run as ROOT that
    would help.

    cpan JSON::XS


    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har**isekhon<http://www.linkedin.com/in/harisekhon>

    On 4 July 2013 10:56, Jaykay wrote:

    When I try to run ./check_hadoop_cloudera_**manage****r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help, I get this error:

    Can't locate JSON/XS.pm in @INC (@INC contains: ./lib
    /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at (eval
    14) line 2.

    I did get a few errors running the cpan JSON step-where would the
    log for that be stored?

    On Tuesday, July 2, 2013 9:51:13 PM UTC+1, Hari Sekhon wrote:

    There are install instructions on the GitHub page itself, it's a
    fairly basic setup requirement:

    git clone https://github.com/**haris****ekhon/nagios-plugins.git<https://github.com/harisekhon/nagios-plugins.git>
    cd nagios-plugins
    git submodule init
    git submodule update

    You will need some CPAN modules which can be installed by running
    this command as root:

    cpan LWP::UserAgent JSON JSON:XS

    Then should be ready to rock:

    ./check_hadoop_cloudera_**manage****r_metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>--help

    You can also then move this nagios-plugins directory where ever you
    want on your Nagios server.

    Fyi I've just added TLS support for Cloudera Manager this evening
    at the request of a couple of other people.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har****isekhon<http://www.linkedin.com/in/harisekhon>

    On 2 July 2013 10:25, Jaykay wrote:

    Thanks Hari and Matthew-much appreciated.
    Hari-is there a guide on how to install these plugins? Or is it
    just the standard install steps for any Nagios plugin?

    On Monday, July 1, 2013 6:39:44 PM UTC+1, Hari Sekhon wrote:

    As a follow up to this, I was asked if I could write some CM
    monitoring integration which was on my todo list anyway so I took a few
    hours at the weekend and wrote a new Nagios plugin to do that using the
    Cloudera Manager API.

    If you've got Cloudera Manager you've probably noticed just how
    many different metrics it collects, so this opens up a huge amount of
    monitoring information to Nagios allowing you to leverage your Nagios SMS,
    escalations, event handlers etc.

    You can specify one metric with optional thresholds or retrieve
    many metrics at the same time.

    I've also added an --all-metrics switch to fetch all metrics for
    the given --cluster --service / --hostid etc so if you don't know what you
    want to monitor you can see what's available for the given section (cluster
    service / role / namespace / host)

    check_hadoop_cloudera_manager_********metrics.pl<http://check_hadoop_cloudera_manager_metrics.pl>

    is now part of my Nagios Plugins collection found at:

    https://github.com/harisekhon/********nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har******isekhon<http://www.linkedin.com/in/harisekhon>

    On 27 June 2013 13:35, Hari Sekhon wrote:

    Hi Jaykay,

    I wrote several before I joined Cloudera, you can find them here
    in my Nagios Plugins collection:

    https://github.com/harisekhon/********nagios-plugins<https://github.com/harisekhon/nagios-plugins>

    They include Hadoop HDFS Capacity / % Balance / Replication
    (Corrupt/Missing/Under-**replica******ted) / Available
    DataNodes / Available + Blacklisted TaskTrackers, JobTracker & NameNode
    Heap, precise node list matches (which you can dynamically generate in
    Nagios), ZooKeeper mode/writeable/latency, as well as some HDFS, MR and
    ZooKeeper metrics for Nagios integrated graphing. A couple may need
    updating, let me know as I've been meaning to do that anyway. They all come
    with --help.

    Regards,

    Hari Sekhon
    Cloudera
    http://www.linkedin.com/in/**har******isekhon<http://www.linkedin.com/in/harisekhon>


    On 27 June 2013 13:15, Jaykay wrote:

    We have a CDH4 cluster that I want to monitor with Nagios. What
    should I monitor with it?
    I have basic checks for disk space and whether servers are up.
    But what about Cloudera specific checks, like monitoring services on the
    Named node etc?

    --
    Thanks,
    Darren

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJun 27, '13 at 12:15p
activeJul 16, '13 at 5:35p
posts19
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase