media tools (was Re: [AG-TECH] AccessGrid 3: What information is available)

[Andrew Swan]

Rhys Hawkins wrote:
> I'm going to try and avoid language wars as best as possible, but I
> need a starting point to explain what I would like to happen. I believe
> that the lower levels should be C, my reasons are:
>  - libcommon is C and it works
>  - most free/OS codecs are in C/C++
>  - capturing interfaces are in C
>  - better access to platform specific services for performance (eg
> Quicktime on Mac, DirectShow in win)

right, i don't think there is any disagreement about this and
this is indeed how the tools are currently organized.

> So what I would like to see developed is a C library(ies) of rtp et al,
> capture interfaces, codecs. This can then be wrapped into whatever
> higher level language you want. For the AG this would probably be
> python, with the GUI in wxPython. In terms of the VIC code, this could
> be achieved by ripping out the TCL/TK stuff from the codecs in VIC, ie
> so the decoders no longer derive from TclObject and modify the
> interfaces as appropriate. How feasible is this?
>
> The benefit of this is that anybody could use this library, wrap it in
> another language if they so desire(python, java, tcl/tk) and write their
> own consumer or producer app and *know* that they will be compatible
> with other video applications used in the AG. And if we have a common
> stable base, then we might finally get away from H.261 only AG sessions!

it is feasible but i don't think it is very desirable.  to dive
down slightly deeper, there are really 3 software "layers" in
vic.  at the lowest level are the codecs, network stack, etc.
written in c++.  at the top level is the user interface, written
in tcl/tk.  between these layers is code that handles the
creation, configuration, and cleanup of the low level objects.
this is the code that, for instance, creates the pipeline of
grabber-encoder-transmitter-network needed to send video and
the pipeline of network-decoder-renderer needed to receive and
display video.  this code reacts to events from the low level
(e.g., a new source begins transmitting or an existing source
changes formats).  it also reacts to events from the ui (e.g.,
the user clicks on a thumbnail to begin displaying a sender
or the user changes the resolution at which a stream is to be
displayed).

the issue is that both this middle layer and the ui layer are
written in tcl.  if you want to throw tcl out completely and
rewrite user interfaces in python, you also lose that middle
layer and have to re-implement it in python as well.  assuming
you do this, you now have two implementations of the middle
layer, one in tcl and one in python.  any experienced software
developer will agree that that is a situation to be avoided if
possible.

of course, another compromise would be to leave the tcl middle
layer alone and to have apps that include c++, tcl, and python,
all in the same address space.  can we agree that is something
that nobody wants?

-Andrew