Hyperic is a great monitoring suite as well, as it auto discovers all sorts
of daemons and lets you monitor them without much work at all. It's great.

A plugin to collect information from Hadoop itself also exists: <

On Fri, Jun 5, 2009 at 9:37 AM, Brian Bockelman wrote:

Hey Anthony,

I'd add you can look at our Ganglia stats:



The second one is the node-specific view, and you can see all the Hadoop
metrics integrated nicely with the system metrics.


On Jun 5, 2009, at 11:33 AM, Matt Massie wrote:

The ganglia web site is at http://ganglia.info/ with documentation in a
wiki at http://ganglia.wiki.sourceforge.net/. There is also a good wiki
page at IBM as well
http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia .
Ganglia packages are available for most distributions to help with
installation so make sure to grep for ganglia with your favorite package
manager (e.g. aptitude, yum, etc). Ganglia will give you more information
about your cluster than just Hadoop metrics. You'll get CPU, load, memory,
disk and network monitoring as well for free.

You can see live demos of ganglia at http://ganglia.info/?page_id=69.

Good luck.


On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote:

Hey Anthony,
Look into hooking your Hadoop system into Ganglia; this produces about 20
real-time statistics per node.

Hadoop also does JMX, which hooks into more "enterprise"-y monitoring


On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote:

Hey all,
I'm currently tasked to come up with a web/flex-based
visualization/monitoring system for a cloud system using hadoop as part
of a
university research project. I was wondering if I could elicit some
feedback from all of you with regards to:

- If you were an engineer of a cloud system running hadoop, what
information would you be interested in capturing, viewing, monitoring,
- Is there any sort of real-time stats or monitoring currently available
for hadoop? if so, is in a web-friendly format?

Thanks in advance,

- Anthony

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 9 | next ›
Discussion Overview
groupcommon-user @
postedJun 5, '09 at 2:07p
activeJun 12, '09 at 3:32a



site design / logo © 2022 Grokbase