FAQ
Jamie,

Its my team that is working on
cityindex/logsearch-development-flow<https://github.com/cityindex/logsearch-development-flow>,
and I think it would make a great basis for a CF dashboard.

1. Logstash works *really* well as an agent fetching log events from
somewhere; parsing them; and feeding them into another system for indexing.
  Look at using logstash as the basis for your worker agent.
2. ElasticSearch is actually a JSON database + powerful searching with a
nice REST interface. So, it both stores and indexes whatever JSON you feed
it. And it scales fantastically - we're currently indexing 50,000 log
events / minute within 15sec of the logs getting written with almost zero
server load.
3. Kibana 3 is a collection of AngularJS widgets backed by data fetched
via REST calls (to ElasticSearch). It would be easy to integrate extra
panels; although if you got your data into ElasticSearch correctly I'd
imagine you wouldn't need to write much extra - see
http://goo.gl/uDlyFcfor a few examples of the kind of dashboards you
can create. (I have no
idea why it insists on starting on slide 4!)

D
On Wednesday, 7 August 2013, Jamie van Dyke wrote:

Good ideas. I'll definitely look into integrating these in the future.
Phase 1 is just displaying simple graphs, charts and lists. Phase 2 is to
integrate org and user management, I think.


On Wed, Aug 7, 2013 at 11:28 AM, ruben.koster@innovationfactory.eu <
ruben.koster@innovationfactory.eu> wrote:

Maybe I would also be nice to integrate http://kibana.org for viewing
logs.
It makes use of logstash which is compatible with syslog_aggregator. Also
have a look here https://github.com/cityindex/logsearch-development-flow.

With regards to health, I think we should be looking for anomalies with
something like https://github.com/etsy/skyline.

Showing a task log would definitely be useful.

Keep up the good work!


On Wednesday, August 7, 2013 11:29:22 AM UTC+2, Jamie van Dyke wrote:

Hey everyone!

I thought I'd start a discussion on what exactly we'd like from an admin
dashboard perspective, for both BOSH and Cloud Foundry. Having played with
a few technologies now, I'm starting to settle on a plan of execution.

I have a meteor app that can poll simple GET requests from both BOSH and
Cloud Foundry. However, re-implementing the API in javascript might not be
the best plan so I'm thinking of going with my original plan and having a
worker app that polls for information continuously from the API's and
inserts that information into the meteor mongodb, which automatically
updates the web site. This seems to work well although running it on
meteor.com can take up to 15 seconds to actually update the site. Not
ideal. I'm hoping that when I get mongo running on my CF setup that it'll
be quicker.

I've hooked up the d3js.org libraries onto my front end too, and I can
make pretty animated graphs/charts from the information with ease. It's SO
MUCH FUN. Now I've prototyped the idea a little I'm building out what I
actually want to see on the dashboard. So, here are the questions:

Initially I'm just doing read only, what would you like to see on the
dashboard? Maybe:

1 - Log files from each of your CF instances? I haven't looked into how to
do this, but it would be nice to see, huh?

2 - Health of instances that are running. CPU usage, memory, drive space?

3 - The latest BOSH tasks that have run?

4 - Something else?

I'd like your feedback as I go, and I'll probably open this up on my
github account soon to get new eyes on the idea. Because I started again
it's quite new and shiny so it could easily be improved by many of you.

Let me know!



--
*Jamie van Dyke*: Chief Science Officer at PharmMD Inc.<http://www.pharmmd.com>
phone: 615-713-2020
web: www.pharmmd.com
twitter: @fearoffish <http://twitter.com/fearoffish>


------------------------------

Have an innovative day
--
David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 5 | next ›
Discussion Overview
groupvcap-dev @
postedAug 7, '13 at 9:29a
activeAug 7, '13 at 12:59p
posts5
users4

People

Translate

site design / logo © 2021 Grokbase