FAQ
Hyperic is a great monitoring suite as well, as it auto discovers all sorts
of daemons and lets you monitor them without much work at all. It's great.

A plugin to collect information from Hadoop itself also exists: <
http://github.com/hyperic/hq-hadoop/tree/master>

Alex
On Fri, Jun 5, 2009 at 9:37 AM, Brian Bockelman wrote:

Hey Anthony,

I'd add you can look at our Ganglia stats:

http://rcf.unl.edu/ganglia/?c=red-workers


http://rcf.unl.edu/ganglia/?c=red-workers&h=node123&m=load_one&r=hour&s=descending&hc=4

The second one is the node-specific view, and you can see all the Hadoop
metrics integrated nicely with the system metrics.

Brian


On Jun 5, 2009, at 11:33 AM, Matt Massie wrote:

Anthony-
The ganglia web site is at http://ganglia.info/ with documentation in a
wiki at http://ganglia.wiki.sourceforge.net/. There is also a good wiki
page at IBM as well
http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia .
Ganglia packages are available for most distributions to help with
installation so make sure to grep for ganglia with your favorite package
manager (e.g. aptitude, yum, etc). Ganglia will give you more information
about your cluster than just Hadoop metrics. You'll get CPU, load, memory,
disk and network monitoring as well for free.

You can see live demos of ganglia at http://ganglia.info/?page_id=69.

Good luck.

-Matt

On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote:

Hey Anthony,
Look into hooking your Hadoop system into Ganglia; this produces about 20
real-time statistics per node.

Hadoop also does JMX, which hooks into more "enterprise"-y monitoring
systems.

Brian

On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote:

Hey all,
I'm currently tasked to come up with a web/flex-based
visualization/monitoring system for a cloud system using hadoop as part
of a
university research project. I was wondering if I could elicit some
feedback from all of you with regards to:


- If you were an engineer of a cloud system running hadoop, what
information would you be interested in capturing, viewing, monitoring,
etc?
- Is there any sort of real-time stats or monitoring currently available
for hadoop? if so, is in a web-friendly format?

Thanks in advance,

- Anthony

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 9 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 5, '09 at 2:07p
activeJun 12, '09 at 3:32a
posts9
users8
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase