of daemons and lets you monitor them without much work at all. It's great.
A plugin to collect information from Hadoop itself also exists: <
http://github.com/hyperic/hq-hadoop/tree/master>
Alex
On Fri, Jun 5, 2009 at 9:37 AM, Brian Bockelman wrote:
Hey Anthony,
I'd add you can look at our Ganglia stats:
http://rcf.unl.edu/ganglia/?c=red-workers
http://rcf.unl.edu/ganglia/?c=red-workers&h=node123&m=load_one&r=hour&s=descending&hc=4
The second one is the node-specific view, and you can see all the Hadoop
metrics integrated nicely with the system metrics.
Brian
On Jun 5, 2009, at 11:33 AM, Matt Massie wrote:
Anthony-
Hey Anthony,
I'd add you can look at our Ganglia stats:
http://rcf.unl.edu/ganglia/?c=red-workers
http://rcf.unl.edu/ganglia/?c=red-workers&h=node123&m=load_one&r=hour&s=descending&hc=4
The second one is the node-specific view, and you can see all the Hadoop
metrics integrated nicely with the system metrics.
Brian
On Jun 5, 2009, at 11:33 AM, Matt Massie wrote:
Anthony-
The ganglia web site is at http://ganglia.info/ with documentation in a
wiki at http://ganglia.wiki.sourceforge.net/. There is also a good wiki
page at IBM as well
http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia .
Ganglia packages are available for most distributions to help with
installation so make sure to grep for ganglia with your favorite package
manager (e.g. aptitude, yum, etc). Ganglia will give you more information
about your cluster than just Hadoop metrics. You'll get CPU, load, memory,
disk and network monitoring as well for free.
You can see live demos of ganglia at http://ganglia.info/?page_id=69.
Good luck.
-Matt
On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote:
Hey Anthony,
wiki at http://ganglia.wiki.sourceforge.net/. There is also a good wiki
page at IBM as well
http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia .
Ganglia packages are available for most distributions to help with
installation so make sure to grep for ganglia with your favorite package
manager (e.g. aptitude, yum, etc). Ganglia will give you more information
about your cluster than just Hadoop metrics. You'll get CPU, load, memory,
disk and network monitoring as well for free.
You can see live demos of ganglia at http://ganglia.info/?page_id=69.
Good luck.
-Matt
On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote:
Hey Anthony,
Look into hooking your Hadoop system into Ganglia; this produces about 20
real-time statistics per node.
Hadoop also does JMX, which hooks into more "enterprise"-y monitoring
systems.
Brian
On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote:
Hey all,
real-time statistics per node.
Hadoop also does JMX, which hooks into more "enterprise"-y monitoring
systems.
Brian
On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote:
Hey all,
I'm currently tasked to come up with a web/flex-based
visualization/monitoring system for a cloud system using hadoop as part
of a
university research project. I was wondering if I could elicit some
feedback from all of you with regards to:
- If you were an engineer of a cloud system running hadoop, what
information would you be interested in capturing, viewing, monitoring,
etc?
- Is there any sort of real-time stats or monitoring currently available
for hadoop? if so, is in a web-friendly format?
Thanks in advance,
- Anthony
visualization/monitoring system for a cloud system using hadoop as part
of a
university research project. I was wondering if I could elicit some
feedback from all of you with regards to:
- If you were an engineer of a cloud system running hadoop, what
information would you be interested in capturing, viewing, monitoring,
etc?
- Is there any sort of real-time stats or monitoring currently available
for hadoop? if so, is in a web-friendly format?
Thanks in advance,
- Anthony
