FAQ
Kamaelia 0.2.0 has been released!

What is it?
==========Kamaelia is a collection of Axon components designed for network
protocol experimentation in a single threaded, select based environment.
Axon components are python generators are augmented by inbox and outbox
queues (lists) for communication in a communicating sequential processes
(CSP) like fashion.

The architecture is specifically designed to try and simplify the
process of designing and experimenting with new network protocols in
real environments. More background on the motivations behind Kamaelia
can be found here: http://kamaelia.sourceforge.net/Challenges/

The focus of this release adds in support for introspection, pygame based
interfaces, 4 new examples using these, visualisation tools, as well as
syntactic sugar to make building Kamaelia systems simpler. (Specifically
Graphline and pipeline systems. This build upon the existing base allowing
TCP and Multicast based client/server systems.

Other additions and changes include updated examples, variety of bugfixes
in existing components (some pre-emptively discovered by introspection), and
a variety of utility components. It is now also possible to write components
using threading as the concurrency model rather than generators - allowing
integration with thread/blocking only based systems.

The system is known to work under Linux, Mac OS X, Windows and a subset
has been tested on Series 60 mobiles.

General feedback is welcome either directly, mailing lists or via the
project weblog which is here:
* http://kamaelia.sourceforge.net/cgi-bin/blog/blog.cgi

What's new in version 0.2.0 ?
============================
Lots! Full release notes can be found here:
* http://kamaelia.sourceforge.net/Kamaelia-0.2.0-ReleaseNotes.html

Editted highlights...

Debian Packages! Many thanks are due to Gintautas Miliauskas, Programmers
of Vilnius, for assistance in building Debian packages. (The current
packages are based on his, and any errors are likely to be mine, not
his) These have been tested successfully on Ubuntu 5.04.

4 new examples have been added showing of various new subsystem:
* Example 5 : This creates a simple streaming system, and looks inside
to see what components are running/active, and passes the resulting
information an Axon Visualiser.

* Example 6 : This is a simple/generic topology visualisation server.
Accepts the following commands over the network
ADD NODE id label auto -
ADD LINK id id
DEL NODE id
DEL ALL
As this stands this is pretty useful, but that's pretty much everything
it does like this.

* Example 7 : This shows how the visualisation subsystem can be extended
to work in different ways. What this does by default when run is
randomly create new nodes and new linkages quite quickly, allowing
you to see how the system works.

* Example 8 : Sample slideshow/presentation tool. Unlike traditional
slideshow/presentation tools, you can modify this to run arbitrary
components. An example of how this can work is provided - allowing
stepping through some graph visualisations along with the presentation.

A Tools directoy has been added with the following tools:
* Axon Shell. (Requires IPython) Implements a simple command line shell
which allows experimentation with Axon systems - the shell runs a
scheduler as a background thread. For a tutorial of use, see:
* http://kamaelia.sourceforge.net/AxonShell.html
* Axon Visualiser. Implements a simple tool for looking inside (quite
literally) running Axon/Kamaelia systems. This allows a very different
style of debugging and can be extremely useful. Tutorial on its way!

Graphlines and Pipelines

These are probably the most useful additions to Kamaelia since 0.1.2.
They are essentially syntactic sugar for building and working with
systems of components, but make building interesting systems rapidly
out of pre-existing components fun and easy. The pipelines follow the
same sort of model as the Unix pipeline. Graphlines are something new,
and like pipelines and all linkages may take any data along them.

Please take a look at the release notes for a graphline example.

A couple of simple pipelines looks like this:

pipeline(
ReadFileAdaptor(file_to_stream, readmode="bitrate", bitrate@0000,
chunkrateP),
blockise(), # Ensure chunks small enough for multicasting!
Multicast_transceiver("0.0.0.0", 0, "224.168.2.9", 1600),
).activate()
pipeline(
Multicast_transceiver("0.0.0.0", 1600, "224.168.2.9", 0),
detuple(1),
VorbisDecode(),
AOAudioPlaybackAdaptor(),
).run()

A selection of other subsystems have been added - targeted at visualisation
of Kamaelia (and other graph structured) systems using pygame. The layout
mechanism is a simple physics engine.

Key packages of note added: Kamaelia.UI, Kamaelia.UI.Pygame,
Kamaelia.Physics, Kamaelia.Visualisation.Axon,
Kamaelia.Visualisation.PhysicsGraph

Requirements
=========== * Python 2.3 or higher recommended, though please do report any bugs
with 2.2.
* Axon (1.1.1 recommended)
* vorbissimple (if you want to use the vorbis decode component/examples)

(Both Axon and vorbissimple are separate parts of the Kamaelia project,
and available at the same download location - see below)

Platforms
========Kamaelia has been used successfully under both Linux, Windows and Mac OS
X (panther). A subset of Kamaelia has been successfully tested on Series 60
Nokia mobiles when used with the Axon SERIES 60 branch.

Where can I get it?
==================Web pages are here:
http://kamaelia.sourceforge.net/Docs/
http://kamaelia.sourceforge.net/ (includes info on mailing lists)

ViewCVS access is available here:
http://cvs.sourceforge.net/viewcvs.py/kamaelia/

Weblog
* http://kamaelia.sourceforge.net/cgi-bin/blog/blog.cgi

Licensing
========Kamaelia is released under the Mozilla tri-license scheme
(MPL1.1/GPL2.0/LGPL2.1). See http://kamaelia.sourceforge.net/Licensing.html

Best Regards,


Michael.
--
Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.

Search Discussions

  • Phil hunt at Aug 2, 2005 at 6:44 am
    Kamaelia seems it might be an interesting project. However, I don't
    think the project is well served by this announcement -- which I
    find vague and hard to understand. Which is a shame, because it
    means that other people probably don't understand it very well
    either, which means less people will use it.

    In a spirit of constructive criticism I'd like to give some detailed
    comments about this...

    On Mon, 01 Aug 2005 19:50:11 +0200, Michael Sparks wrote:

    Kamaelia is a collection of Axon components designed for network
    protocol experimentation in a single threaded, select based environment.
    Axon components are python generators are augmented by inbox and outbox
    queues (lists) for communication in a communicating sequential processes
    (CSP) like fashion.
    I really have very little idea what this means.
    The architecture is specifically designed to try and simplify the
    process of designing and experimenting with new network protocols in
    real environments.
    The information I can guess out of this is: "Kamaelia is a library
    for creating network protocols that sit on top of tcp and/or udp".
    Given that it's a BBC project, I imagine we're talking about
    protocols for transferring large amount of data, e.g. sound or
    motion pictures.
    More background on the motivations behind Kamaelia
    can be found here: http://kamaelia.sourceforge.net/Challenges/
    There's something interesting here:

    In Building Public Value, the BBC has committed to the delivery of
    BBC content to the British Public over this multitude of systems,
    in a way that is enjoyable, accessible and **repurposable** by the
    British Public in the way they choose.

    (my emphasis)

    This is a laudable aim, but if the BBC want to do this, the first
    step they could make is to open up their content under a Creative
    Commons licence. They could start with their news, current affairs
    and documentaries. They could also do this with music downloads: for
    example a few weeks ago they made available for free download
    all of Beethoven's symphonies, performed by the BBC Orchestra. But the
    download only staryed up for a week, and they didn't put the files
    under a CC license, so they could be redistributed. Why not? It
    would have cost them no more to do so than what they did, and
    would've provided better value for downloaders and music lovers.

    Of course, the music industry would've complained, but they did
    anyway. Frankly, fuck them. Their business model is obsolete, and
    they should either get a new one or crawl into a corner and die.

    BTW, the html is broken in that it's very poorly written, doedsn't
    use CSS, and some of the stuff that is intended to display in a
    monospaced font doesn't. For example, this HTML code from
    <http://kamaelia.sourceforge.net/Docs/Axon.html>:


    <div><span style="font-family:Courier 10
    Pitch;font-weight:600">def</span><span style="font-family:Courier
    10 Pitch"> main(self):</span></div>
    <ul type="disc"><div><span style="font-family:Courier 10
    Pitch;font-weight:600">yield</span><span
    style="font-family:Courier 10 Pitch"> 1<br /></span><span
    style="font-family:Courier 10 Pitch;font-weight:600">while</span>
    <span style="font-family:Courier 10 Pitch">(self.i):</span></div>

    could be better written as:

    <pre>
    def main(self):
    yield 1
    while (self.i):
    </pre>

    Much, much simpler, and actually works.
    General feedback is welcome either directly, mailing lists or via the
    project weblog which is here:
    * http://kamaelia.sourceforge.net/cgi-bin/blog/blog.cgi
    I guess the main question that needs to be answered is "Why would I,
    as a programmer, use Kamaelia?"


    --
    Email: zen19725 at zen dot co dot uk
  • Michael Sparks at Aug 2, 2005 at 10:05 am

    Phil Hunt wrote:

    Kamaelia seems it might be an interesting project. However, I don't
    think the project is well served by this announcement -- which I
    find vague and hard to understand. Which is a shame, because it
    means that other people probably don't understand it very well
    either, which means less people will use it.
    It is a shame, and thanks for mentioning this. Let me have another
    go :-) (ripping to shreds welcome :)

    OK, here's a better go. (It's always difficult to think where to pitch
    this sort of thing)
    ---START---
    The project aims to make it simple to build networked multimedia
    systems (eg audio, video, interactive systems), which are naturally
    componentised and naturally concurrent allowing quick and fast reuse in
    the same way as Unix pipelines do.

    It is designed as a practical toolkit, such that you can build systems
    such as:
    * Ogg Vorbis streaming server/client systems (via vorbissimple)
    * Simple network aware games (via pygame)
    * Quickly build TCP based network servers and clients
    * Quickly build Multicast based network servers and clients

    It runs on Linux, Window, Mac OS X with a subset running on Series 60
    phones.

    The basic underlying metaphor of a component us like an office worker
    with inboxes and outboxes, with deliveries occuring between desks,
    offices, and depts. The component can thus do work anyway it likes but
    only communicates with these inboxes and outboxes. Like office workers,
    components run in parallel, and to achieve this are generally
    implemented using python generators, but can also used threads.

    The rationale behind the project is to provide a toolkit enabling the
    development of new protocols, including streaming, for large scale
    media delivery. The license essentially allows use in proprietary
    systems without change, but all changes to the system itself must be
    shared.
    ---END---

    Is that clearer ?

    A short summary of all that could be:

    """Kamaelia is a networking/communications infrastructure for innovative
    multimedia systems. Kamaelia uses a component architecture designed to
    simplify creation and testing of new protocols and large scale media
    delivery systems."""

    Hopefully that's clearer than:
    Kamaelia is a collection of Axon components designed for network
    protocol experimentation in a single threaded, select based
    environment. Axon components are python generators are augmented by
    inbox and outbox queues (lists) for communication in a communicating
    sequential processes (CSP) like fashion.
    [ which you noted "I really have very little idea what this means." ]
    The information I can guess out of this is: "Kamaelia is a library
    for creating network protocols that sit on top of tcp and/or udp".
    Given that it's a BBC project, I imagine we're talking about
    protocols for transferring large amount of data, e.g. sound or
    motion pictures.
    Indeed. However, if you're having to guess I'm not doing the best job
    (despite trying!) at explaining this.
    More background on the motivations behind Kamaelia
    can be found here: http://kamaelia.sourceforge.net/Challenges/
    There's something interesting here:

    In Building Public Value, the BBC has committed to the delivery of
    BBC content to the British Public over this multitude of systems,
    in a way that is enjoyable, accessible and **repurposable** by the
    British Public in the way they choose.

    (my emphasis)

    This is a laudable aim, but if the BBC want to do this, the first
    step they could make is to open up their content under a Creative
    Commons licence.
    <not such a wise thing for me to reply to, but please read the
    disclaimer at the bottom, these are personal opinions only>

    I would suggest that you forward this comment directly to the team
    involved in working on the Creative Archive. You can do this via this
    URL:
    * http://creativearchive.bbc.co.uk/

    This is unfortunately not as simple as "just putting stuff up", and any
    comments you have, I'm sure would be welcome. However, the Creative
    Archive hasn't yet been put to a Public Value Test (including impact on
    industry), and rights is a much more problematic issue than you might
    expect. (eg background music on the radio in eastenders, posters on
    people's walls...)

    There's also http://www.bbc.co.uk/info/contactus/form.shtml which might
    be more appropriate.

    On a positive note, the Creative Archive has more organisations involved
    than just the BBC, it inlucde the BFI , Channel 4, Open University &
    Teachers'TV, so even if these things take longer than some people might
    like they do seem to be moving. Public organisations are weighed down
    whether the should do something, as well as whether they can.

    BBC R&D's role in this is to make these decisions possible.
    They could start with their news, current affairs
    and documentaries.
    BBC Backstage is a start here - at least with some of the online
    content. (http://backstage.bbc.co.uk)
    But the download only staryed up for a week, and they didn't put the
    files under a CC license, so they could be redistributed. Why not? It
    would have cost them no more to do so than what they did, and
    would've provided better value for downloaders and music lovers.
    I can pass on that request if you're interested in an answer - I'm not
    personally aware of the rationale behind it only being up for a week.

    </not such a wise thing for me to reply to, but please read the
    disclaimer at the bottom, these are personal opinions only>
    BTW, the html is broken in that it's very poorly written,
    It's written using a PyQT based WYSIWYG editor (written on my time,
    not the BBC's) that has the limitation that preformatted text gets
    squished (since the editor doesn't support the pre tag - or indeed
    a lot of other tags). At some point I'll fix the editor to handle
    pre-formatted text, but it's not a priority right now. (Should it be?)

    The HTML isn't really *that* bad as far as generated code is concerned
    IMHO, and is directly what the PyQT QTextEdit widget creates. (It's not
    that great either - you may notice it doesn't have proper headings
    either)
    doesn't use CSS,
    This isn't a problem IMHO, YMMV. The site's been tested on a variety of
    browsers/platforms and looks pretty much the same on all.
    I guess the main question that needs to be answered is "Why would I,
    as a programmer, use Kamaelia?"
    Probably the most interesting response I've had back from people who've
    picked it up is that it's FUN. You can write small sketches (throwaway
    code fragments), rewrite their inputs/outputs to take data from inboxes
    & outboxes, and then it's usable in a much wider system. It encourages
    reuse of code, and assimilating existing systems can be relatively
    trivial.

    For example, trivial sketches:
    * A program to display an image using pygame
    * A program that can understand when an area of space has been
    clicked. (call it a button)
    * A program that can print a list of filenames, pausing waiting for a
    user to press return. (call it a chooser)

    Trivial changes:
    * Take the filename of the image from an inbox
    * Rather than print the filenames to send the filename to an outbox.
    * Rather than wait for a user to press a key, wait for a message on
    an inbox.

    A useful composition linking some of these together:

    Graphline(
    CHOOSER = Chooser(items = files),
    IMAGE = Image(size=(800,600), position=(8,48)),
    NEXT = Button(caption="Next", msg="NEXT", position=(72,8)),
    PREVIOUS = Button(caption="Previous", msg="PREV",position=(8,8)),
    linkages = {
    ("NEXT","outbox") : ("CHOOSER","inbox"),
    ("PREVIOUS","outbox") : ("CHOOSER","inbox"),
    ("CHOOSER","outbox") : ("IMAGE","inbox"),
    }
    ).run()

    And you have a simple presentation tool !

    Another example. The visualisation framework was originally a stand
    alone piece of code and was essentially just a piece of eye candy. I
    threw in a few yields into certain locations. File reading & static
    graph construction was replaced with recieving data from inboxes, and
    then it was available for use (and networked after creating trivial
    protocol components).

    A piece of eye candy is now a useful tool. It's likely to gain the
    ability to send to an outbox a message saying "this blob is clicked"
    meaning we can have 'WYSIWYG' GUI based construction of pipelines
    easily.

    It's designed literally as a tool for making it simple to bolt things
    together with minimal change. (Which means you choose what you
    use, not the framework.)

    The system also lends itself to test driven and network independent
    development, something not that simple normally with network
    protocols.

    Why wouldn't you use it? When Twisted is appropriate (Twisted is a more
    mature framework).


    Michael.
    --
    Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) contains personal views
    which are not the views of the BBC unless specifically stated.
    (THIS IS NORMALLY AT THE BOTTON OF MY EMAILS, BUT IT'S REALLY, REALLY,
    REALLY THE CASE THIS TIME!!!)
  • Phil hunt at Aug 3, 2005 at 11:36 am

    On Tue, 02 Aug 2005 11:05:16 +0100, Michael Sparks wrote:
    Phil Hunt wrote:
    Kamaelia seems it might be an interesting project. However, I don't
    think the project is well served by this announcement -- which I
    find vague and hard to understand. Which is a shame, because it
    means that other people probably don't understand it very well
    either, which means less people will use it.
    It is a shame, and thanks for mentioning this. Let me have another
    go :-) (ripping to shreds welcome :)

    OK, here's a better go. (It's always difficult to think where to pitch
    this sort of thing)
    ---START---
    The project aims to make it simple to build networked multimedia
    systems (eg audio, video, interactive systems),
    There's plenty of software that facilitates networking, for example
    Python already has software for tcp and http clients/servers, and
    for xmlrpc remote procedure calls.

    So what does Kamaelia do that's extra? I imagine it's to to with
    streaming large amounts of data. For example, a streaming video or
    audio player. Or VoIP, perhaps.
    which are naturally
    componentised and naturally concurrent allowing quick and fast reuse in
    the same way as Unix pipelines do.
    OK, so what do the components in the pipelines do? What sort of data
    do they hold? Unix pipelines act on ascii files; I assume you are
    do this on audio and visual data. What langauage will the ele,ments
    in thne pipelines be written it? I assume some will be in C/C++ for
    speed.
    It is designed as a practical toolkit, such that you can build systems
    such as:
    When you say "you" who do you mean? Is the audience programmers or
    less technical people? A project that allows non-technical people
    to build complex network applications is an ambitious one, but not
    impossible (I'd find it very impressive and very exciting,
    particularly if it runs on devices such as mobile phones).
    * Ogg Vorbis streaming server/client systems (via vorbissimple)
    * Simple network aware games (via pygame)
    * Quickly build TCP based network servers and clients
    What sort of servers and clients? Serving what? Could I use it, for
    example, to build a
    * Quickly build Multicast based network servers and clients
    Serving what? Could I use it, for example, to build an n-player
    encrypted (I mean proper encryption here, the sort GCHQ or the NSA
    can't break) VoIP server to allow people to do conference calls over
    the Internet?
    It runs on Linux, Window, Mac OS X with a subset running on Series 60
    phones.

    The basic underlying metaphor of a component us like an office worker
    with inboxes and outboxes, with deliveries occuring between desks,
    offices, and depts. The component can thus do work anyway it likes but
    only communicates with these inboxes and outboxes. Like office workers,
    components run in parallel, and to achieve this are generally
    implemented using python generators, but can also used threads.
    That metaphor brings up an image (at least to me) that the sorts of
    data that can be communicated are things like documents,
    spreadsheets, business graphs, memos. Also the metaphor doesn't
    indicate any sense of real-time-ness; stuff on an in tray isn't
    immediately processed and moved to the out tray.I'm pretty certain
    that's not what you have in mind.

    May I suggest a different metaphor? Consider a hi-fi system where
    you plug cables in to control the dataflow. For example, an
    unamplified audio signal goes into an amp, the amp has various dials
    and buttons on it, and an amplified signal comes out. Note that the
    signal processing is done in real time.

    Is that the sort of thing you are working on?
    The rationale behind the project is to provide a toolkit enabling the
    development of new protocols, including streaming, for large scale
    media delivery.
    OK, I get the straming part of it. But what asbout non-streaming
    stuff? What other protocols are necessary?
    The license essentially allows use in proprietary
    systems without change, but all changes to the system itself must be
    shared.
    Fair enough.
    ---END---

    Is that clearer ?

    A short summary of all that could be:

    """Kamaelia is a networking/communications infrastructure for innovative
    multimedia systems. Kamaelia uses a component architecture designed to
    simplify creation and testing of new protocols and large scale media
    delivery systems."""

    Hopefully that's clearer than:
    Kamaelia is a collection of Axon components designed for network
    protocol experimentation in a single threaded, select based
    environment. Axon components are python generators are augmented by
    inbox and outbox queues (lists) for communication in a communicating
    sequential processes (CSP) like fashion.
    [ which you noted "I really have very little idea what this means." ]
    Sort of. But it's still a bit vague. For example:

    "networking/communications infrastructure" can mean a lot of things.
    It doesn't even specify whether it is hardware or software.

    "innovative". This actually has two meanings. One is "is new /
    allows new things to be built". Another is "meaningless marketing
    buzzword" (half :-)). I assume you mean the former, but some people
    may be put off by what sounds like buzzwordspeak.

    A good short summary would answer these questions:

    * What broad category does the system fall under? Software or
    Hardware? (the answer here is software). For what sort of
    problem-domain? (here the answer is audio and video)

    * What sort of software is it? An end product, useful in itself? Or
    a library, from which people can build applications?

    * Who is it aimed at? Programmers, less-technical users, the genral
    public?

    * At what stage of completion is it? This is a question sometimes
    not answered well by open source projects, which can cause
    disappointment if a project is at an earlier stage of development
    than some percieve. And also, a project might have immediate goals
    that are quite modest, and long-term goals that are more ambitious.
    For example, trivial sketches:
    * A program to display an image using pygame
    * A program that can understand when an area of space has been
    clicked. (call it a button)
    * A program that can print a list of filenames, pausing waiting for a
    user to press return. (call it a chooser)

    Trivial changes:
    * Take the filename of the image from an inbox
    * Rather than print the filenames to send the filename to an outbox.
    * Rather than wait for a user to press a key, wait for a message on
    an inbox.

    A useful composition linking some of these together:

    Graphline(
    CHOOSER = Chooser(items = files),
    IMAGE = Image(size=(800,600), position=(8,48)),
    NEXT = Button(caption="Next", msg="NEXT", position=(72,8)),
    PREVIOUS = Button(caption="Previous", msg="PREV",position=(8,8)),
    linkages = {
    ("NEXT","outbox") : ("CHOOSER","inbox"),
    ("PREVIOUS","outbox") : ("CHOOSER","inbox"),
    ("CHOOSER","outbox") : ("IMAGE","inbox"),
    }
    ).run()
    That's certainly less lines of code than it would take in Tkinter.
    And easier to follow.
    And you have a simple presentation tool !
    Now I'm confused. Is Kamaelia a GUI builder?
    Another example. The visualisation framework was originally a stand
    alone piece of code and was essentially just a piece of eye candy. I
    threw in a few yields into certain locations. File reading & static
    graph construction was replaced with recieving data from inboxes, and
    then it was available for use (and networked after creating trivial
    protocol components).
    Ah, so now I'm guessing it allows me to write networked PyGame
    applications, where with just a few lines of code I can have
    networked streaming video on my pyGame screen... am I right?
    A piece of eye candy is now a useful tool. It's likely to gain the
    ability to send to an outbox a message saying "this blob is clicked"
    meaning we can have 'WYSIWYG' GUI based construction of pipelines
    easily.
    That sounds useful.
    Why wouldn't you use it? When Twisted is appropriate (Twisted is a more
    mature framework).
    The problem with Twisted, IME, is I don't understand the
    documentation.

    --
    Email: zen19725 at zen dot co dot uk
  • Sergei Organov at Aug 3, 2005 at 1:30 pm
    zen19725 at zen.co.uk (phil hunt) writes:
    [...]
    Unix pipelines act on ascii files;
    No, they don't.

    --
    Sergei.
  • Phil hunt at Aug 3, 2005 at 2:15 pm

    On 03 Aug 2005 17:30:31 +0400, Sergei Organov wrote:
    zen19725 at zen.co.uk (phil hunt) writes:
    [...]
    Unix pipelines act on ascii files;
    No, they don't.
    Nitpicker.

    I would have thought it was perfectly obvious, in context, what I
    meant.

    --
    Email: zen19725 at zen dot co dot uk
  • Michael Sparks at Aug 3, 2005 at 3:57 pm
    I've reordered the q's slightly to avoid repetition... Also by answering
    this question first, it may put the rest of the answer into context
    better.

    phil hunt wrote:
    At what stage of completion is it?
    This is something we deliberately try to reflect in the version number.
    Yes, you can build network servers and new protocols relatively simply
    at the moment. Yes, you can integrate with pygame in a limited useful
    fashion at present. Yes we have audio playback.

    However we don't have yet... (some examples)
    * Decent GUI integration yet.
    * /Full/ pygame integration.
    * Nice integration with pymedia
    * Direct support for Dirac.
    Which aside from other things means you can't build (say) a video
    & SMIL playback system trivially, yet.

    As a result that's why the version number is 0.2 - whilst you /can/ do a
    lot, there's a lot more to do. Clearly that also naturally implies that
    we don't expect any end user to be looking at the site. (The low
    version number will normally scare them away)
    The project aims to make it simple to build networked multimedia
    systems (eg audio, video, interactive systems),
    There's plenty of software that facilitates networking, for example
    Python already has software for tcp and http clients/servers, and
    for xmlrpc remote procedure calls.
    There is indeed.
    So what does Kamaelia do that's extra? I imagine it's to to with
    streaming large amounts of data. For example, a streaming video or
    audio player. Or VoIP, perhaps.
    It's designed to make bolting things together to make these sorts of
    system simpler and easier. At the same time it's designed to encourage
    writing code in a way that makes it simpler. The natural side effect of
    this is the system might make it easier to take advantage of multiple
    CPU systems as they come online, since it makes a system naturally
    concurrent. As the original announcement said "Kamaelia is designed as
    a testbed". And by testbed I mean testbed as it testing out new ideas,
    see if they work and see if they pan out. (Not as in a testing suite)

    Probably the best way of describing the difference is this... After my
    talk about Kamaelia at Europython, I had an long chat with Tommi
    Virtinan about integration between Kamaelia and Twisted. I haven't had
    a chance to follow up with him yet regarding how this would work, though
    I have set a line in the sand aiming to have easy integration between
    Kamaelia and Twisted before Kamaelia hits version 1.0.0. The impression
    I got from Tommi was that he was most interested in the communications
    aspect - the fact we can bolt together systems in a manner directly
    akin to Unix pipelines, though I suspect he found the graphines aspect
    more interesting.

    Or as someone asking a similar question at Open Tech exclaimed after
    I finally managed to explain it better to them "Ooooh - you're trying
    to make concurrency EASY!".
    OK, so what do the components in the pipelines do? What sort of data
    do they hold? Unix pipelines act on ascii files; I assume you are
    do this on audio and visual data. What langauage will the ele,ments
    in thne pipelines be written it? I assume some will be in C/C++ for
    speed.
    Components are object instances of python classes. The data passed
    between components are object instances. Clearly these python classes
    can be written in python, C, C++, pyrex etc. Currently all of Kamaelia's
    components are python based. Some existing components make calls
    into some related C libraries via python bindings.. An example of
    writing a component using Pyrex can be found here:
    * http://kamaelia.sourceforge.net/PyrexComponents.html
    It is designed as a practical toolkit, such that you can build systems
    such as:
    When you say "you" who do you mean?
    Generally I expect the readership of c.l.p/python-list at python.org to be
    programmers. Python is generally easy to pick up and having asked
    someone who's not done much programming beforehand (beyond
    a small amount of VB and Access), and is pre-university to use the
    system to build a simple streaming system prototyping visualising PVR
    content on a mobile (and watching them succeed), they seem
    relatively reasonable examples.

    At some point, the ability to allow non-programmers to bolt together
    Kamaelia systems would be desirable, but a first step is making it
    simpler for programmers to bolt together systems. We currently have an
    interactive visualisation tool(*), and the logical extension of that is
    a tool that allows systems to be bolted together without knowing any
    code.
    (*) http://kamaelia.sourceforge.net/AxonVisualiser.html

    It'd be an interesting side project for someone to take forward, and
    might be low hanging fruit in terms of projects. (Especially if viewed
    initially as a tool for assisting development, rather than replacing
    development)
    Is the audience programmers or
    less technical people? A project that allows non-technical people
    to build complex network applications is an ambitious one, but not
    impossible (I'd find it very impressive and very exciting,
    particularly if it runs on devices such as mobile phones).
    It's a little ambitious at this stage, yes.
    * Ogg Vorbis streaming server/client systems (via vorbissimple)
    * Simple network aware games (via pygame)
    * Quickly build TCP based network servers and clients
    What sort of servers and clients?
    Whatever you feel like. If you want a server to split and serve audio,
    you could do that. if you want a server to spit out fortune cookies,
    you can do that. (Useful only really as an alternative to a chargen
    test protocol IMO)
    * Quickly build Multicast based network servers and clients
    Serving what? Could I use it, for example, to build an n-player
    encrypted VoIP server to allow people to do conference calls over
    the Internet?
    You could do that probably. (Though we don't have a component
    for audio capture (though a read file adaptor reading from /dev/audio
    might work depending on your platform I suppose) and audio
    encoding at the moment, so those would probably be the core
    components to integrate. If you want to use multicast over the wide
    area internet you'd also have to convince all the people using the
    system to use ISPs that support multicast......)
    (I mean proper encryption here, the sort GCHQ or the NSA can't break)
    I'd be impressed if that could be written, using anything really. (Can't
    implies never)
    The basic underlying metaphor of a component us like an office worker
    with inboxes and outboxes, with deliveries occuring between desks,
    That metaphor brings up an image (at least to me) that the sorts of
    data that can be communicated are things like documents,
    spreadsheets, business graphs, memos.
    They could indeed. The underlying framework doesn't differentiate
    between data nor have any realtime aspect embedded in the system
    at present. Just because we're focussing on systems that have a realtime
    element and are multimedia based, this does not mean the system is
    limited to that.
    May I suggest a different metaphor? [hifi]
    I'll think about it. It's closer to the model that pysonic seems
    to take, and it implies a functional transform - ie /just/ dataflow
    - rather than connected processing units that /might/ take time
    to do processing.
    OK, I get the straming part of it. But what asbout non-streaming
    stuff? What other protocols are necessary?
    One example is peer to peer mesh setup. People normally
    think of P2P as a distribution mechanism. However, the underlying
    approach also very good at setting up communications meshes.
    This could be of use in many areas, such as GRID based systems
    for distributed rendering, application layer multicast, and network
    multicast island joining.

    Due to the illegal /uses/ of P2P, much work in this area is difficult to
    reuse due to defensive coding. Decoupling development of protocols
    from use of those protocols is generally a wide idea IMO.
    Is that clearer ?

    A short summary of all that could be:
    [ new descscription followed by useful comments, we'll take
    them on board, I think I've answered most points inline ]
    "innovative". This actually has two meanings. One is "is new /
    allows new things to be built".
    Bingo. It could be argued the other is misuse as buzzword.
    That's certainly less lines of code than it would take in Tkinter.
    And easier to follow.
    We're looking at tkinter as well. (Some tests at tkinter integration
    are in CVS)
    And you have a simple presentation tool !
    Now I'm confused. Is Kamaelia a GUI builder?
    Multimedia systems have many aspects. They include taking in
    audio/video and spitting out audio/video/pictures/text/... Take
    a look a SMIL if you're curious - it's a system that Kamaelia would be
    incomplete if it made decoding/display/interpretation of SMIL
    difficult.

    We also have to be able to demonstrate system to other people
    inside the BBC in a way non-technical people understand. That means
    showing structures in a friendly dynamic way, showing pictures,
    playing sounds (hence visualisation - looking inside running systems).

    That means we need ways of integrating with systems like pygame &
    other toolkits. If however I'm talking outside the BBC I'll try to give
    examples which people might find interesting - such as building a
    presentation tool. The blocks are very much like Lego & K'Nex and
    adding in a new block enables all sorts of new applications.

    For example, we could take the text ticker, combine that with a text
    feed and have a personal autocue/teleprompter. Alternatively someone
    could use it to have subtitles (say) at the opera displayed on a Nokia
    770 (maemo) based device.
    Ah, so now I'm guessing it allows me to write networked PyGame
    applications, where with just a few lines of code I can have
    networked streaming video on my pyGame screen... am I right?
    As yet, no. When we hit version 1.0, yes. Why? Because we haven't
    built components for video decode & display (and that's the only
    reason). That's why I don't mention video normally - I prefer to focus
    on where we are when announcing things, rather than where we're
    going (leaving that to more appropriate forums).
    Why wouldn't you use it? When Twisted is appropriate (Twisted is a
    more mature framework).
    The problem with Twisted, IME, is I don't understand the
    documentation.

    I can't help there, sorry. There *is* a twisted book coming out shortly,
    which might be worth looking into (or getting from a library), if
    you're interested...

    Finally, please note: Kamaelia is part of an R&D project - not all of
    the results will pan out. Some might, some might not. Unless you aim
    for the Stars though you won't get to Mars.

    Best Regards,


    Michael.
    --
    Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
  • Phil hunt at Aug 3, 2005 at 6:13 pm

    On Wed, 03 Aug 2005 16:57:34 +0100, Michael Sparks wrote:
    I've reordered the q's slightly to avoid repetition... Also by answering
    this question first, it may put the rest of the answer into context
    better.

    phil hunt wrote:
    At what stage of completion is it?
    This is something we deliberately try to reflect in the version number.
    Yes, you can build network servers and new protocols relatively simply
    at the moment. Yes, you can integrate with pygame in a limited useful
    fashion at present. Yes we have audio playback.

    However we don't have yet... (some examples)
    * Decent GUI integration yet.
    * /Full/ pygame integration.
    * Nice integration with pymedia
    * Direct support for Dirac.
    Which aside from other things means you can't build (say) a video
    & SMIL playback system trivially, yet.
    Isn't SMIL something that's goinhg to go into web browsers? In which
    case, you'd presumably not want to build one yourself, I imagine?
    There's plenty of software that facilitates networking, for example
    Python already has software for tcp and http clients/servers, and
    for xmlrpc remote procedure calls.
    There is indeed.
    So what does Kamaelia do that's extra? I imagine it's to to with
    streaming large amounts of data. For example, a streaming video or
    audio player. Or VoIP, perhaps.
    It's designed to make bolting things together to make these sorts of
    system simpler and easier.
    What you say "bolting things together" do you mean writing Python
    code? Or will there be some other way?

    What I have in mind is something like a project I worked on some
    time ago, where a program could be written by drawing boxes on a GUI
    application, and drawing lines (representing data flow) to connect
    the boxes. So one half of a VoIP application might look like:

    +----------+ +------------+ +-----------+
    listen | | convert to | | volume |
    on udp |====>| audio |====>| control + |
    port 600 | | stream | | output |
    +----------+ +------------+ +-----------+

    With something like this novel multimedia apps could be prototyped
    quickly (though making them into useful apps would take extra work
    -- in the case of a VoIP application you'd want a phonebook, for
    example).
    At the same time it's designed to encourage
    writing code in a way that makes it simpler.
    Examples would be useful here. Both of what it can do now, and what
    it will be able to do eventually.
    The natural side effect of
    this is the system might make it easier to take advantage of multiple
    CPU systems as they come online, since it makes a system naturally
    concurrent. As the original announcement said "Kamaelia is designed as
    a testbed". And by testbed I mean testbed as it testing out new ideas,
    see if they work and see if they pan out. (Not as in a testing suite)
    So what it will eventually do is not cast in stone?
    Probably the best way of describing the difference is this... After my
    talk about Kamaelia at Europython, I had an long chat with Tommi
    Virtinan about integration between Kamaelia and Twisted. I haven't had
    a chance to follow up with him yet regarding how this would work, though
    I have set a line in the sand aiming to have easy integration between
    Kamaelia and Twisted before Kamaelia hits version 1.0.0. The impression
    I got from Tommi was that he was most interested in the communications
    aspect - the fact we can bolt together systems in a manner directly
    akin to Unix pipelines, though I suspect he found the graphines aspect
    more interesting.

    Or as someone asking a similar question at Open Tech exclaimed after
    I finally managed to explain it better to them "Ooooh - you're trying
    to make concurrency EASY!".
    OK, so what do the components in the pipelines do? What sort of data
    do they hold? Unix pipelines act on ascii files; I assume you are
    do this on audio and visual data. What langauage will the ele,ments
    in thne pipelines be written it? I assume some will be in C/C++ for
    speed.
    Components are object instances of python classes. The data passed
    between components are object instances.
    What sort of objects? For example, if an application is audio
    straming, what sort of objects will be passed? Will there be onre
    object passed for each 10 ms or so of sound, or will there be a
    continuously-set-up object that passes the data throught it a byte
    at a time?

    And if it's written in Python, will it run fast enough?
    Clearly these python classes
    can be written in python, C, C++, pyrex etc. Currently all of Kamaelia's
    components are python based. Some existing components make calls
    into some related C libraries via python bindings.. An example of
    writing a component using Pyrex can be found here:
    * http://kamaelia.sourceforge.net/PyrexComponents.html
    My thought on this example:

    cdef class test(Axon.Component.component):
    def mainBody(self):
    if self.dataReady("inbox"):
    data = self.recv("inbox")
    self.send(data, "outbox")
    return 1

    My first impressions are that it seems obvious to me what this does:
    it merely passes its input throught to its output (no doubt there
    are places where such a null component is useful). Hopefully all
    code written with Kamaelia will be so clear.

    Now I have some questions.

    1. mainBody() is evidently being called at intervals. At what
    frequency are these intervals? For example, if the data stram is
    video data, will mainBody() get called once for each frame?

    2. what data type is (data) in the code? Or can it be any type?


    --
    Email: zen19725 at zen dot co dot uk
  • Michael Sparks at Aug 3, 2005 at 9:18 pm

    phil hunt wrote:
    On Wed, 03 Aug 2005 16:57:34 +0100, Michael Sparks wrote:
    ...
    Which aside from other things means you can't build (say) a video
    & SMIL playback system trivially, yet.
    Isn't SMIL something that's goinhg to go into web browsers? In which
    case, you'd presumably not want to build one yourself, I imagine?
    I was using SMIL as an example to be illustrative of a class of networked
    applications that require visual support. A more direct example
    would be MHEG (not mpeg) decoding and display systems, which
    have similarities to SMIL systems in terms of capabilities required.
    It's designed to make bolting things together to make these sorts of
    system simpler and easier.
    What you say "bolting things together" do you mean writing Python
    code? Yes.
    Or will there be some other way?
    A clicky pointy approach would be nice. (There's two sorts of interface in
    the world IMO - "tappity tappity" and "clicky pointy" - if being equally
    disdainful :) Unix shell systems are very "tappity tappity", and whilst I
    can do "tappity tappity", and have been doing "tappity tappity" for my
    entiring working life, I do prefer "clicky pointy" systems.)

    Though building systems using AR Toolkit type systems would be the ideal.
    ( http://www.hitl.washington.edu/people/poup/research/ar.htm#artoolkit )
    That's probably a couple of years off though (at least).
    What I have in mind is something like a project I worked on some
    time ago, where a program could be written by drawing boxes on a GUI
    application, and drawing lines (representing data flow) to connect
    the boxes. So one half of a VoIP application might look like:

    +----------+ +------------+ +-----------+
    listen | | convert to | | volume |
    on udp |====>| audio |====>| control + |
    port 600 | | stream | | output |
    +----------+ +------------+ +-----------+
    From example 4:
    pipeline(
    Multicast_transceiver("0.0.0.0", 1600, "224.168.2.9", 0),
    detuple(1),
    SRM_Receiver(),
    detuple(1),
    VorbisDecode(),
    AOAudioPlaybackAdaptor(),
    ).run()

    Join multicast group "224.168.2.9", listen for packets on port 1600, throw
    away information relating to who sent it, pass it through a simple reliable
    multicast reordering subsystem, throw away the annotation, decode the
    vorbis, and playback.

    The visual representation of that isn't dissimilar to your diagram above.
    With something like this novel multimedia apps could be prototyped
    quickly (though making them into useful apps would take extra work
    -- in the case of a VoIP application you'd want a phonebook, for
    example).
    Agreed.
    At the same time it's designed to encourage
    writing code in a way that makes it simpler.
    Examples would be useful here. Both of what it can do now, and what
    it will be able to do eventually.
    Examples of what can be done now are here:
    * http://kamaelia.sourceforge.net/KamaeliaStatus.html

    I'll think about the latter part of that. (I personally loath technologies
    that promise much and deliver little. Hence why I like to focus on what
    we /can/ do)
    The natural side effect of
    this is the system might make it easier to take advantage of multiple
    CPU systems as they come online, since it makes a system naturally
    concurrent. As the original announcement said "Kamaelia is designed as
    a testbed". And by testbed I mean testbed as it testing out new ideas,
    see if they work and see if they pan out. (Not as in a testing suite)
    So what it will eventually do is not cast in stone?
    The high level goals are to solve the problems in the Challenges document.
    We're not particularly wedded to any one particular approach, and Kamaelia
    can/should be moulded to solve those problems, not the other way round.

    By definition along the way a variety of components will be generated, which
    can obviously be put together in an arbitrary fashion, assuming type
    adaptors as necessary, to form new interesting systems.

    This ability to explore new/interesting systems quickly is something
    generally of value to an R&D dept, so there's some people at BBC R&D
    interested in using Kamaelia for all sorts of interesting projects. (which I
    can't discuss here)
    Components are object instances of python classes. The data passed
    between components are object instances.
    What sort of objects?
    Python objects. Numbers, strings, chunks of audio in a string, chunks of
    audio in a binary object wrapped by a python class, etc. Precisely as I
    said.
    For example, if an application is audio
    straming, what sort of objects will be passed?
    That depends on the level to which you dive inside the audio being
    streamed. If you're trying to be a content agnostic streamer then you
    don't care what's inside the byte stream (from, say, a file) just what the
    bit rate is you're supposed to send at. (for that you can just chop up the
    file and send it chunks at a time)

    If however you're dealing with variable bit rate data you may wish to look
    inside the wrapper to figure out the where data blocks are for a given time
    period in order to send those. This (for example in the case of sending MPEG
    over RTP) can mean that you need to parse the data stream in order to send
    the data. At that point it makes sense to send over a structured object
    representing each chunk/block/frame (depending on codec/transport).
    Will there be onre
    object passed for each 10 ms or so of sound, or will there be a
    continuously-set-up object that passes the data throught it a byte
    at a time?
    Small messages turn out to be inefficient for what it's worth. (It's more
    efficient to send bundles of small message or larger messages)
    And if it's written in Python, will it run fast enough?
    I've seen overlay mesh networks implemented in TCL shipping 60Mbit/s
    (sustained) in the past, which if we're implementing an overlay joining
    multicast islands together would mean multicasting the equivalent of ~200
    channels through each node. Scaling is not always about throughput of
    course - Bit Torrent's initial implementation was also in python, and scales
    very well.

    If it doesn't scale sufficiently we can always optimise specific subsystems
    when we need to rather than prematurely optimise. Also we have done a
    naive (deliberately naive) translation of the ideas into C++, including
    generators, so if things don't pan out we have a natural migration path else
    where. (After knowing whether the overall approach is workable or not)
    cdef class test(Axon.Component.component):
    def mainBody(self):
    if self.dataReady("inbox"):
    data = self.recv("inbox")
    self.send(data, "outbox")
    return 1

    My first impressions are that it seems obvious to me what this does:
    A line of code is more often read than written, so if it's obvious, that's
    good.
    it merely passes its input throught to its output Correct.
    (no doubt there are places where such a null component is useful).
    An echo protocol is one example - a server you connect to that simply echoes
    back to you whatever you send it. (an echo server can be useful for all
    sorts of network testing in practice)
    Hopefully all code written with Kamaelia will be so clear.
    See here for a longer example - which should show that generally speaking
    things do tend to remain clear: http://tinyurl.com/dp8n7 (also shows how
    you can incrementally develop new components).
    Now I have some questions.

    1. mainBody() is evidently being called at intervals. At what
    frequency are these intervals? For example, if the data stram is
    video data, will mainBody() get called once for each frame?
    The scheduler is currently a dumb scheduler with essentially the following
    logic:
    for thread in microthreads:
    thread.next()

    As a result the time between calls is not guaranteed (nor can it be with
    this sort of scheduler). The system does however allow for multiple
    schedulers, so you could potentially have a realtime scheduler if it
    was found to be necessary, at which point information about how a
    component wished to be scheduled would become important.
    2. what data type is (data) in the code? Or can it be any type?
    Any data type python handles. (as indicated above)

    In case you're wondering "why doesn't it do everything yet", I'll
    just simply reiterate that we're at version 0.2, and we're getting
    to the crawling/walking stage first before the walking/talking/running
    stage :-)

    Best Regards,


    Michael.
  • Phil hunt at Aug 3, 2005 at 6:48 pm

    On Wed, 03 Aug 2005 16:57:34 +0100, Michael Sparks wrote:
    Is the audience programmers or
    less technical people? A project that allows non-technical people
    to build complex network applications is an ambitious one, but not
    impossible (I'd find it very impressive and very exciting,
    particularly if it runs on devices such as mobile phones).
    It's a little ambitious at this stage, yes.
    But it couldbe there eventually?
    * Ogg Vorbis streaming server/client systems (via vorbissimple)
    * Simple network aware games (via pygame)
    * Quickly build TCP based network servers and clients
    What sort of servers and clients?
    Whatever you feel like. If you want a server to split and serve audio,
    you could do that.
    This is streaming audio, right? For non-streaming I can just use an
    ftp or http server.
    * Quickly build Multicast based network servers and clients
    Serving what? Could I use it, for example, to build an n-player
    encrypted VoIP server to allow people to do conference calls over
    the Internet?
    You could do that probably. (Though we don't have a component
    for audio capture (though a read file adaptor reading from /dev/audio
    might work depending on your platform I suppose) and audio
    encoding at the moment, so those would probably be the core
    components to integrate.
    That's a slightly worrying answer for me, worrying because it seems
    I've misunderstood the nature of the project. I assumed that
    components for audio capture, and related activities, would be at
    the heart of the project.
    If you want to use multicast over the wide
    area internet you'd also have to convince all the people using the
    system to use ISPs that support multicast......)
    (or just sent the signal out multiple times)
    (I mean proper encryption here, the sort GCHQ or the NSA can't break)
    I'd be impressed if that could be written, using anything really. (Can't
    implies never)
    What -- good encryption? That's pretty much a well-known technique
    these days (unless the NSA has some *very* advanced hardware in
    their basement, which I strongly suspect they don't).
    The basic underlying metaphor of a component us like an office worker
    with inboxes and outboxes, with deliveries occuring between desks,
    That metaphor brings up an image (at least to me) that the sorts of
    data that can be communicated are things like documents,
    spreadsheets, business graphs, memos.
    They could indeed. The underlying framework doesn't differentiate
    between data nor have any realtime aspect embedded in the system
    at present. Just because we're focussing on systems that have a realtime
    element and are multimedia based, this does not mean the system is
    limited to that.
    Again, this makes me think I've misunderstood the project.
    OK, I get the straming part of it. But what asbout non-streaming
    stuff? What other protocols are necessary?
    One example is peer to peer mesh setup. People normally
    think of P2P as a distribution mechanism. However, the underlying
    approach also very good at setting up communications meshes.
    When you say a mesh, what do you mean?
    This could be of use in many areas, such as GRID based systems
    for distributed rendering, application layer multicast, and network
    multicast island joining.
    Unpack, please.
    Due to the illegal /uses/ of P2P, much work in this area is difficult to
    reuse due to defensive coding.
    Oh. Could you give an example?
    We also have to be able to demonstrate system to other people
    inside the BBC in a way non-technical people understand. That means
    showing structures in a friendly dynamic way, showing pictures,
    playing sounds (hence visualisation - looking inside running systems).
    Visualisation, if done properly, ought to be useful to technical
    people too.
    That means we need ways of integrating with systems like pygame &
    other toolkits. If however I'm talking outside the BBC I'll try to give
    examples which people might find interesting - such as building a
    presentation tool. The blocks are very much like Lego & K'Nex and
    adding in a new block enables all sorts of new applications.
    That's kind of the impression that I've got.
    For example, we could take the text ticker, combine that with a text
    feed and have a personal autocue/teleprompter. Alternatively someone
    could use it to have subtitles (say) at the opera displayed on a Nokia
    770 (maemo) based device.
    That would be useful.

    Or you could have subtitles in different languages, and the user
    gets to choose which one to display...

    --
    Email: zen19725 at zen dot co dot uk
  • Michael Sparks at Aug 3, 2005 at 8:16 pm

    phil hunt wrote:
    On Wed, 03 Aug 2005 16:57:34 +0100, Michael Sparks wrote:
    Is the audience programmers or
    less technical people? A project that allows non-technical people
    to build complex network applications is an ambitious one, but not
    ...
    It's a little ambitious at this stage, yes.
    But it couldbe there eventually?
    Could? Yes. Will? Can't say. I can agree it would be nice, and given
    time/resources (or someone sufficiently interested externally) then it
    may happen. (IMO there's no real reason that it couldn't happen aside
    from time/effort/resources)
    What sort of servers and clients?
    Whatever you feel like. If you want a server to split and serve audio,
    you could do that.
    This is streaming audio, right? For non-streaming I can just use an
    ftp or http server.
    There's more to network servers and clients than just audio & video, or
    unidirectional download.

    For example the visualisation/introspection tool is an example of a
    client server system. The server is the visualisation tool. It listens on a
    specified port waiting for a connection. The client connects and sends
    data to it about the internal structure, and the server displays this.
    * Quickly build Multicast based network servers and clients
    Serving what? Could I use it, for example, to build an n-player
    encrypted VoIP server to allow people to do conference calls over
    the Internet?
    You could do that probably. (Though we don't have a component
    for audio capture (though a read file adaptor reading from /dev/audio
    might work depending on your platform I suppose) and audio
    encoding at the moment, so those would probably be the core
    components to integrate.
    That's a slightly worrying answer for me, worrying because it seems
    I've misunderstood the nature of the project. I assumed that
    components for audio capture, and related activities, would be at
    the heart of the project.
    *Our* main interest, at the moment, is in /delivery/ of content.

    Dealing with capture would be largely re-inventing wheels before we know
    whether the framework is a suitable framework. We are looking at making it
    possible to use pymedia for dealing with capture/encoding/decoding.

    There's a number of things along the way we need to deal with this, but
    we're not starting from the perspective of capture.

    (After all for capture we can generally look at using dedicated encoder
    hardware that will often spit out it's encoded information in the form of a
    network connection. As a result capture and encoding hasn't been a priority
    as yet. Sometimes looking into a project from the outside I can appreciate
    that certain decisions might look strange, but consider that you don't need
    to worry about capture in order
    (I mean proper encryption here, the sort GCHQ or the NSA can't break)
    I'd be impressed if that could be written, using anything really. (Can't
    implies never)
    What -- good encryption? That's pretty much a well-known technique
    these days (unless the NSA has some *very* advanced hardware in
    their basement, which I strongly suspect they don't).
    You said *can't*. That says to me cannot ever be broken. If you have a large
    number of listeners, as your statement implied, that implies decryptable by
    many listeners - you then just need one compromised listener (essentially
    you're asking for the equivalent of implementing a DRM system that the NSA
    couldn't break...).

    If you can provide me with a library that you can guarantee that it will
    satisfy the following properties:

    encoded = F(data)

    and a piece of client code that can do this:
    decoded = G(encoded)

    Then yes, that can be wrapped. That's trivial in fact:
    ---(start)---
    from magic import unbreakable_encryption

    class encoder(component):
    def __init__(self, **args):
    self.encoder = unbreakable_encryption.encoder(**args)
    def main(self):
    while 1:
    if self.dataReady("inbox"):
    data = self.recv("inbox")
    encoded = self.encoder.encode(data)
    self.send(encoded, "outbox")
    yield 1

    class decoder(component):
    def __init__(self, **args):
    self.decoder = unbreakable_encryption.decoder(**args)
    def main(self):
    while 1:
    if self.dataReady("inbox"):
    data = self.recv("inbox")
    decoded = self.decoder.decode(data)
    self.send(decoded, "outbox")
    yield 1
    ---(end)---

    If you believe you can implement F&G for general use such that F&G
    can //never// be decrypted by anyone other than authorised recipients, I
    suggest you cease this conversation - you have some highly marketable
    technology.
    The basic underlying metaphor of a component us like an office worker
    with inboxes and outboxes, with deliveries occuring between desks,
    That metaphor brings up an image (at least to me) that the sorts of
    data that can be communicated are things like documents,
    spreadsheets, business graphs, memos.
    They could indeed. The underlying framework doesn't differentiate
    between data nor have any realtime aspect embedded in the system
    at present. Just because we're focussing on systems that have a realtime
    element and are multimedia based, this does not mean the system is
    limited to that.
    Again, this makes me think I've misunderstood the project.
    Realtime systems are a subset of all systems that are interesting in terms
    of network delivery & multimedia systems. Realtime scheduling is a well
    known area and if/when this becomes an issue, we'll look at adding it into
    the mix. The real problem is dealing with concurrency and making it simple
    to work with. Making realtime concurrent systems easy to work with strikes
    me as running before you can walk.

    (Network systems are naturally concurrent, so if you're aiming to make
    network systems easy to build you're really talking about making concurrent
    systems easy to build.)
    OK, I get the straming part of it. But what asbout non-streaming
    stuff? What other protocols are necessary?
    One example is peer to peer mesh setup. People normally
    think of P2P as a distribution mechanism. However, the underlying
    approach also very good at setting up communications meshes.
    When you say a mesh, what do you mean?
    I mean a mesh. (see below)
    This could be of use in many areas, such as GRID based systems
    for distributed rendering, application layer multicast, and network
    multicast island joining.
    Unpack, please.
    They're all big systems, all of which utilise networks of collaborating
    systems for different purposes.

    Grid starting point:
    * http://en.wikipedia.org/wiki/Grid_computing

    *short* introduction to application level multicast (has many names,
    including overlay multicast):
    * http://www.mnlab.cs.depaul.edu/seminar/fall2002/Overcast.pdf
    * Also puts the term "mesh" in context.

    Multicast island joining is a special case and is exactly what it says -
    joining multicast islands together.
    Due to the illegal /uses/ of P2P, much work in this area is difficult to
    reuse due to defensive coding.
    Oh. Could you give an example?
    How many RFCs have you seen documenting the protocols used by (say)
    Napster, Bit Torrent, Gnutella, Limewire, Freenet? The legitimate uses of
    Bit Torrent for example tend to get ignored by certain large companies
    when trying to shut down systems.
    We also have to be able to demonstrate system to other people
    inside the BBC in a way non-technical people understand. That means
    showing structures in a friendly dynamic way, showing pictures,
    playing sounds (hence visualisation - looking inside running systems).
    Visualisation, if done properly, ought to be useful to technical
    people too.
    It is (as mention on the page describing the visualisation tool).

    Regards,


    Michael.
  • Wolfgang Keller at Aug 8, 2005 at 9:49 am

    The project aims to make it simple to build networked multimedia
    systems (eg audio, video, interactive systems),
    Why limit the scope of application artificially?

    Axon/Kamaelia imho is perfectly applicable to any kind of application
    that does some kind of data processing.

    Think workflow applications, business applications, application
    integration...

    J. Paul Morrison has used the concept he named "Flow-Based-Programming"
    for financial applications since the 60's, any control systems engineer
    knows function-block-diagrams etc. and so on...

    Sincerely,

    Wolfgang Keller
  • Matt Hammond at Aug 8, 2005 at 11:02 am

    On Mon, 08 Aug 2005 10:49:32 +0100, Wolfgang Keller wrote:
    The project aims to make it simple to build networked multimedia
    systems (eg audio, video, interactive systems),
    Why limit the scope of application artificially?

    Axon/Kamaelia imho is perfectly applicable to any kind of application
    that does some kind of data processing.
    You're absolutely right - Axon/Kamaelia hopefully will not just be limited
    to those areas. It just so happens that those kinds of systems are the
    underlying reasons why we started building Axon/Kamaelia. That's therefore
    where we're concentrating our resources at the moment.
    Think workflow applications, business applications, application
    integration...

    J. Paul Morrison has used the concept he named "Flow-Based-Programming"
    for financial applications since the 60's, any control systems engineer
    knows function-block-diagrams etc. and so on...
    We have ... alot! I really do hope that this system is used for more. I'd
    be very interested to know if anyone has had a go at building any
    components - for whatever purpose. The potential for component re-use
    might mean less work for us! :-)


    regards


    Matt Hammond
    --
    Matt Hammond
    R&D Engineer, BBC Research and Development, Tadworth, Surrey, UK.
  • Tim Golden at Aug 2, 2005 at 11:53 am
    [Michael Sparks]
    Phil Hunt wrote:
    Kamaelia seems it might be an interesting project. However, I don't
    think the project is well served by this announcement -- which I
    find vague and hard to understand. Which is a shame, because it
    means that other people probably don't understand it very well
    either, which means less people will use it.
    It is a shame, and thanks for mentioning this. Let me have another
    go :-) (ripping to shreds welcome :)

    OK, here's a better go. (It's always difficult to think where to pitch
    this sort of thing)
    [... snip ...]

    I just wanted to say that I find the ideas behind Kamaelia
    interesting, and I only wish I had an application for it!
    Because I'm not especially into media-streaming, I'm more
    interested in it from the point of view of the generator-based
    architecture.

    What prompted me to write now was in appreciation of Michael's
    taking the effort to recast his words into a form which might
    well be understood better by others (including me) when he
    might well have simply huffed a bit and told the other poster
    to read the words on the site and stop complaining about the HTML!

    Thanks, Michael. Hope the project continues apace and if I
    can ever find a need for it, I'll know where to look.

    Tim Golden

    ________________________________________________________________________
    This e-mail has been scanned for all viruses by Star. The
    service is powered by MessageLabs. For more information on a proactive
    anti-virus service working around the clock, around the globe, visit:
    http://www.star.net.uk
    ________________________________________________________________________
  • Phil hunt at Aug 3, 2005 at 11:40 am

    On Tue, 2 Aug 2005 12:53:29 +0100, Tim Golden wrote:
    [Michael Sparks]
    Phil Hunt wrote:
    Kamaelia seems it might be an interesting project. However, I don't
    think the project is well served by this announcement -- which I
    find vague and hard to understand. Which is a shame, because it
    means that other people probably don't understand it very well
    either, which means less people will use it.
    It is a shame, and thanks for mentioning this. Let me have another
    go :-) (ripping to shreds welcome :)

    OK, here's a better go. (It's always difficult to think where to pitch
    this sort of thing)
    [... snip ...]

    I just wanted to say that I find the ideas behind Kamaelia
    interesting, and I only wish I had an application for it!
    Because I'm not especially into media-streaming, I'm more
    interested in it from the point of view of the generator-based
    architecture.

    What prompted me to write now was in appreciation of Michael's
    taking the effort to recast his words into a form which might
    well be understood better by others (including me) when he
    might well have simply huffed a bit and told the other poster
    to read the words on the site and stop complaining about the HTML!
    I'd second that!

    I'm not trying to be critical, honestly, I'm trying to understand
    something that looks like it could be very cool indeed.

    (And on a more general level to improve the average quality of
    documentation for open source projects. Something which is bvery
    often deficient.)

    --
    Email: zen19725 at zen dot co dot uk
  • Michael Sparks at Aug 3, 2005 at 10:07 pm

    Tim Golden wrote:
    I just wanted to say that I find the ideas behind Kamaelia
    interesting, and I only wish I had an application for it!
    Because I'm not especially into media-streaming, I'm more
    interested in it from the point of view of the generator-based
    architecture.
    It's nice to know that people find it interesting even without a direct
    need. (Indeed I suspect this is the case with many open source projects.)

    One thing we are learning as we go along is how to use generators to
    refactor out commonly used code out of loops like this. One example is
    status updating:

    import time
    def long_running_thing(foo):
    t = time.time()pplication for it!
    Because I'm not especially into media-streaming, I'm more
    interested in it from the point of view of the generator-based
    architecture.
    It's nice to know that people find it interesting even without a direct
    need. (Indeed I suspect this is the case with many open source projects.)

    One thing we are learning as we go along is how to use generators to
    refactor out commonly used code out of loops like this. One example is
    status updating:

    import time
    def long_running_thing(foo):
    t = time.time()
    while running:
    << do something long running>>
    if time.time() - t > update_interval:
    print update_message

    An alternative to this is this:

    def updater(interval, message):
    t = time.time():
    while 1:
    if time.time() - t > interval:
    print message

    def long_running_thing(foo):
    myUpdater = updater(5, "Still running")
    while running:
    << do something long running>>
    myUpdater.next()

    You can do similar things for frame rates and so on. Obviously in our system
    we generally go one step further and translate the resulting generators into
    components, but sometimes simple conversion to generators alone
    can make the code much clearer.

    Tim Golden wrote:
    What prompted me to write now was in appreciation of Michael's
    taking the effort to recast his words into a form which might
    well be understood better by others (including me) when he
    might well have simply huffed a bit and told the other poster
    to read the words on the site and stop complaining about the HTML!
    I tend to generally think that for each poster to a newsgroup there's often
    several lurkers who share the same opinion, and given text is a very harsh
    medium, I try to make allowances for that.

    Answering questions can sometimes also help others to assist with
    documentation patches of course. Technical writing not necessarily being
    the same skill set as code writing or architecture design after all :-)

    I must admit I do remember the earlier days of usenet when people would ask
    a question, get several answers, and the original poster would summarise
    the responses (something that doesn't happen as often these days
    unfortunately, possibly due to services like google news making it easier
    to go back and see the discussion).

    phil hunt wrote:
    (And on a more general level to improve the average quality of
    documentation for open source projects. Something which is bvery
    often deficient.)
    You have to remember as well that most open source projects rely on
    volunteers, and anyone can write docs. If you want to improve the average
    quality of docs for open source projects there is only one way to do this -
    to be prepared to write the documentation yourself or pay someone to do
    it. (In our case documentation is driven by internal need, which doesn't
    always match the needs of people externally. You might not've seen the
    white paper giving a detailed overview of the project from a technical
    perspective here: http://tinyurl.com/8543e)

    Tim Golden wrote:
    Thanks, Michael. Hope the project continues apace and if I
    can ever find a need for it, I'll know where to look.
    Thanks for your kind words.

    Best Regards,


    Michael.
  • Terry Reedy at Aug 4, 2005 at 1:11 am
    "Michael Sparks" <ms at cerenity.org> wrote in message
    news:42f14e42$0$3493$ed2619ec at ptn-nntp-reader03.plus.net...
    An alternative to this is this:

    def updater(interval, message):
    t = time.time():
    while 1:
    if time.time() - t > interval:
    print message
    yield None # add this ????
    def long_running_thing(foo):
    myUpdater = updater(5, "Still running")
    while running:
    << do something long running>>
    myUpdater.next()
  • Michael Sparks at Aug 4, 2005 at 3:47 pm

    Terry Reedy wrote:
    "Michael Sparks" <ms at cerenity.org> wrote in message
    news:42f14e42$0$3493$ed2619ec at ptn-nntp-reader03.plus.net...
    def updater(interval, message):
    t = time.time():
    while 1:
    if time.time() - t > interval:
    print message
    yield None # add this ????

    Yes. (I can't believe I missed that out! :) I'd normally use yield 1,
    but that's force of habit. I tend to use a true value to indicate a
    never ending generator - partly because we use a false value to
    indicate clean shutdown of a component to our scheduler.

    Thanks :)


    Michael
    --
    Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedAug 1, '05 at 5:50p
activeAug 8, '05 at 11:02a
posts18
users8
websitepython.org

People

Translate

site design / logo © 2022 Grokbase