FAQ
Hey everyone!

I thought I'd start a discussion on what exactly we'd like from an admin
dashboard perspective, for both BOSH and Cloud Foundry. Having played
with a few technologies now, I'm starting to settle on a plan of execution.

I have a meteor app that can poll simple GET requests from both BOSH and
Cloud Foundry. However, re-implementing the API in javascript might not
be the best plan so I'm thinking of going with my original plan and
having a worker app that polls for information continuously from the
API's and inserts that information into the meteor mongodb, which
automatically updates the web site. This seems to work well although
running it on meteor.com can take up to 15 seconds to actually update
the site. Not ideal. I'm hoping that when I get mongo running on my CF
setup that it'll be quicker.

I've hooked up the d3js.org libraries onto my front end too, and I can
make pretty animated graphs/charts from the information with ease. It's
SO MUCH FUN. Now I've prototyped the idea a little I'm building out what
I actually want to see on the dashboard. So, here are the questions:

Initially I'm just doing read only, what would you like to see on the
dashboard? Maybe:

1 - Log files from each of your CF instances? I haven't looked into how
to do this, but it would be nice to see, huh?

2 - Health of instances that are running. CPU usage, memory, drive space?

3 - The latest BOSH tasks that have run?

4 - Something else?

I'd like your feedback as I go, and I'll probably open this up on my
github account soon to get new eyes on the idea. Because I started again
it's quite new and shiny so it could easily be improved by many of you.

Let me know!



--
*Jamie van Dyke*: Chief Science Officer at PharmMD Inc.
<http://www.pharmmd.com>
phone: 615-713-2020 <callto:415-526-2339>
web: www.pharmmd.com <http://www.pharmmd.com>
twitter: @fearoffish <http://twitter.com/fearoffish>

Search Discussions

  • Ruben Koster at Aug 7, 2013 at 10:28 am
    Maybe I would also be nice to integrate http://kibana.org for viewing logs.
    It makes use of logstash which is compatible with syslog_aggregator. Also
    have a look here https://github.com/cityindex/logsearch-development-flow.

    With regards to health, I think we should be looking for anomalies with
    something like https://github.com/etsy/skyline.

    Showing a task log would definitely be useful.

    Keep up the good work!

    On Wednesday, August 7, 2013 11:29:22 AM UTC+2, Jamie van Dyke wrote:

    Hey everyone!

    I thought I'd start a discussion on what exactly we'd like from an admin
    dashboard perspective, for both BOSH and Cloud Foundry. Having played with
    a few technologies now, I'm starting to settle on a plan of execution.

    I have a meteor app that can poll simple GET requests from both BOSH and
    Cloud Foundry. However, re-implementing the API in javascript might not be
    the best plan so I'm thinking of going with my original plan and having a
    worker app that polls for information continuously from the API's and
    inserts that information into the meteor mongodb, which automatically
    updates the web site. This seems to work well although running it on
    meteor.com can take up to 15 seconds to actually update the site. Not
    ideal. I'm hoping that when I get mongo running on my CF setup that it'll
    be quicker.

    I've hooked up the d3js.org libraries onto my front end too, and I can
    make pretty animated graphs/charts from the information with ease. It's SO
    MUCH FUN. Now I've prototyped the idea a little I'm building out what I
    actually want to see on the dashboard. So, here are the questions:

    Initially I'm just doing read only, what would you like to see on the
    dashboard? Maybe:

    1 - Log files from each of your CF instances? I haven't looked into how to
    do this, but it would be nice to see, huh?

    2 - Health of instances that are running. CPU usage, memory, drive space?

    3 - The latest BOSH tasks that have run?

    4 - Something else?

    I'd like your feedback as I go, and I'll probably open this up on my
    github account soon to get new eyes on the idea. Because I started again
    it's quite new and shiny so it could easily be improved by many of you.

    Let me know!



    --
    *Jamie van Dyke*: Chief Science Officer at PharmMD Inc.<http://www.pharmmd.com>
    phone: 615-713-2020
    web: www.pharmmd.com
    twitter: @fearoffish <http://twitter.com/fearoffish>
    --

    ------------------------------

    Have an innovative day

    *Innovation Factory *De Lairessestraat 180* *1075 HM Amsterdam* *+31
    20 7787008 www.innovationfactory.eu

    *
    Disclaimer*
    *The information transmitted is intended only for the person or entity to
    which it is addressed and may contain confidential and/or privileged
    material. Any use of, or taking of any action in reliance upon, this
    information by persons or entities other than the intended recipient is
    prohibited. If you received this in error, please contact the sender and
    delete the material from any computer. Innovation Factory does not accept
    liability for any errors, viruses or omissions in the contents of this
    message, which may arise as a result of e-mail transmission. No employee or
    agent is authorized to conclude any binding agreement on behalf of
    Innovation Factory with another party by email.*
  • Jamie van Dyke at Aug 7, 2013 at 12:19 pm
    Good ideas. I'll definitely look into integrating these in the future. Phase 1 is just displaying simple graphs, charts and lists. Phase 2 is to integrate org and user management, I think.
    On Wed, Aug 7, 2013 at 11:28 AM, null wrote:

    Maybe I would also be nice to integrate http://kibana.org for viewing logs.
    It makes use of logstash which is compatible with syslog_aggregator. Also
    have a look here https://github.com/cityindex/logsearch-development-flow.
    With regards to health, I think we should be looking for anomalies with
    something like https://github.com/etsy/skyline.
    Showing a task log would definitely be useful.
    Keep up the good work!
    On Wednesday, August 7, 2013 11:29:22 AM UTC+2, Jamie van Dyke wrote:

    Hey everyone!

    I thought I'd start a discussion on what exactly we'd like from an admin
    dashboard perspective, for both BOSH and Cloud Foundry. Having played with
    a few technologies now, I'm starting to settle on a plan of execution.

    I have a meteor app that can poll simple GET requests from both BOSH and
    Cloud Foundry. However, re-implementing the API in javascript might not be
    the best plan so I'm thinking of going with my original plan and having a
    worker app that polls for information continuously from the API's and
    inserts that information into the meteor mongodb, which automatically
    updates the web site. This seems to work well although running it on
    meteor.com can take up to 15 seconds to actually update the site. Not
    ideal. I'm hoping that when I get mongo running on my CF setup that it'll
    be quicker.

    I've hooked up the d3js.org libraries onto my front end too, and I can
    make pretty animated graphs/charts from the information with ease. It's SO
    MUCH FUN. Now I've prototyped the idea a little I'm building out what I
    actually want to see on the dashboard. So, here are the questions:

    Initially I'm just doing read only, what would you like to see on the
    dashboard? Maybe:

    1 - Log files from each of your CF instances? I haven't looked into how to
    do this, but it would be nice to see, huh?

    2 - Health of instances that are running. CPU usage, memory, drive space?

    3 - The latest BOSH tasks that have run?

    4 - Something else?

    I'd like your feedback as I go, and I'll probably open this up on my
    github account soon to get new eyes on the idea. Because I started again
    it's quite new and shiny so it could easily be improved by many of you.

    Let me know!



    --
    *Jamie van Dyke*: Chief Science Officer at PharmMD Inc.<http://www.pharmmd.com>
    phone: 615-713-2020
    web: www.pharmmd.com
    twitter: @fearoffish <http://twitter.com/fearoffish>
    --

    ------------------------------
    Have an innovative day
    *Innovation Factory *De Lairessestraat 180* *1075 HM Amsterdam* *+31
    20 7787008 www.innovationfactory.eu
    *
    Disclaimer*
    *The information transmitted is intended only for the person or entity to
    which it is addressed and may contain confidential and/or privileged
    material. Any use of, or taking of any action in reliance upon, this
    information by persons or entities other than the intended recipient is
    prohibited. If you received this in error, please contact the sender and
    delete the material from any computer. Innovation Factory does not accept
    liability for any errors, viruses or omissions in the contents of this
    message, which may arise as a result of e-mail transmission. No employee or
    agent is authorized to conclude any binding agreement on behalf of
    Innovation Factory with another party by email.*
  • David Laing at Aug 7, 2013 at 12:43 pm
    Jamie,

    Its my team that is working on
    cityindex/logsearch-development-flow<https://github.com/cityindex/logsearch-development-flow>,
    and I think it would make a great basis for a CF dashboard.

    1. Logstash works *really* well as an agent fetching log events from
    somewhere; parsing them; and feeding them into another system for indexing.
      Look at using logstash as the basis for your worker agent.
    2. ElasticSearch is actually a JSON database + powerful searching with a
    nice REST interface. So, it both stores and indexes whatever JSON you feed
    it. And it scales fantastically - we're currently indexing 50,000 log
    events / minute within 15sec of the logs getting written with almost zero
    server load.
    3. Kibana 3 is a collection of AngularJS widgets backed by data fetched
    via REST calls (to ElasticSearch). It would be easy to integrate extra
    panels; although if you got your data into ElasticSearch correctly I'd
    imagine you wouldn't need to write much extra - see
    http://goo.gl/uDlyFcfor a few examples of the kind of dashboards you
    can create. (I have no
    idea why it insists on starting on slide 4!)

    D
    On Wednesday, 7 August 2013, Jamie van Dyke wrote:

    Good ideas. I'll definitely look into integrating these in the future.
    Phase 1 is just displaying simple graphs, charts and lists. Phase 2 is to
    integrate org and user management, I think.


    On Wed, Aug 7, 2013 at 11:28 AM, ruben.koster@innovationfactory.eu <
    ruben.koster@innovationfactory.eu> wrote:

    Maybe I would also be nice to integrate http://kibana.org for viewing
    logs.
    It makes use of logstash which is compatible with syslog_aggregator. Also
    have a look here https://github.com/cityindex/logsearch-development-flow.

    With regards to health, I think we should be looking for anomalies with
    something like https://github.com/etsy/skyline.

    Showing a task log would definitely be useful.

    Keep up the good work!


    On Wednesday, August 7, 2013 11:29:22 AM UTC+2, Jamie van Dyke wrote:

    Hey everyone!

    I thought I'd start a discussion on what exactly we'd like from an admin
    dashboard perspective, for both BOSH and Cloud Foundry. Having played with
    a few technologies now, I'm starting to settle on a plan of execution.

    I have a meteor app that can poll simple GET requests from both BOSH and
    Cloud Foundry. However, re-implementing the API in javascript might not be
    the best plan so I'm thinking of going with my original plan and having a
    worker app that polls for information continuously from the API's and
    inserts that information into the meteor mongodb, which automatically
    updates the web site. This seems to work well although running it on
    meteor.com can take up to 15 seconds to actually update the site. Not
    ideal. I'm hoping that when I get mongo running on my CF setup that it'll
    be quicker.

    I've hooked up the d3js.org libraries onto my front end too, and I can
    make pretty animated graphs/charts from the information with ease. It's SO
    MUCH FUN. Now I've prototyped the idea a little I'm building out what I
    actually want to see on the dashboard. So, here are the questions:

    Initially I'm just doing read only, what would you like to see on the
    dashboard? Maybe:

    1 - Log files from each of your CF instances? I haven't looked into how to
    do this, but it would be nice to see, huh?

    2 - Health of instances that are running. CPU usage, memory, drive space?

    3 - The latest BOSH tasks that have run?

    4 - Something else?

    I'd like your feedback as I go, and I'll probably open this up on my
    github account soon to get new eyes on the idea. Because I started again
    it's quite new and shiny so it could easily be improved by many of you.

    Let me know!



    --
    *Jamie van Dyke*: Chief Science Officer at PharmMD Inc.<http://www.pharmmd.com>
    phone: 615-713-2020
    web: www.pharmmd.com
    twitter: @fearoffish <http://twitter.com/fearoffish>


    ------------------------------

    Have an innovative day
    --
    David Laing
    Open source @ City Index - github.com/cityindex
    http://davidlaing.com
    Twitter: @davidlaing
  • Jamie Van Dyke at Aug 7, 2013 at 12:59 pm
    My initial plan was to go for an open-source and free version.
    Capabilities that cost money will be available maybe as
    plugins/extensions of some sort. Once it's up on github (soon, I
    promise!) then the community can build up on that, and companies can
    fork and create versions with subscription services.

    I like the look of all of these. I'll let you know when it's up so you
    can contribute! ;-)
    David Laing 7 August 2013 13:43
    Jamie,

    Its my team that is working on cityindex/logsearch-development-flow
    <https://github.com/cityindex/logsearch-development-flow>, and I think
    it would make a great basis for a CF dashboard.

    1. Logstash works *really* well as an agent fetching log events from
    somewhere; parsing them; and feeding them into another system for
    indexing. Look at using logstash as the basis for your worker agent.
    2. ElasticSearch is actually a JSON database + powerful searching
    with a nice REST interface. So, it both stores and indexes whatever
    JSON you feed it. And it scales fantastically - we're currently
    indexing 50,000 log events / minute within 15sec of the logs getting
    written with almost zero server load.
    3. Kibana 3 is a collection of AngularJS widgets backed by data
    fetched via REST calls (to ElasticSearch). It would be easy to
    integrate extra panels; although if you got your data into
    ElasticSearch correctly I'd imagine you wouldn't need to write much
    extra - see http://goo.gl/uDlyFc for a few examples of the kind of
    dashboards you can create. (I have no idea why it insists on starting
    on slide 4!)

    D

    On Wednesday, 7 August 2013, Jamie van Dyke wrote:


    --
    David Laing
    Open source @ City Index - github.com/cityindex
    <http://github.com/cityindex>
    http://davidlaing.com
    Twitter: @davidlaing
    --
    *Jamie van Dyke*: Chief Science Officer at PharmMD Inc.
    <http://www.pharmmd.com>
    phone: 615-713-2020 <callto:415-526-2339>
    web: www.pharmmd.com <http://www.pharmmd.com>
    twitter: @fearoffish <http://twitter.com/fearoffish>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupvcap-dev @
postedAug 7, '13 at 9:29a
activeAug 7, '13 at 12:59p
posts5
users4

People

Translate

site design / logo © 2017 Grokbase