FW: AG audio

[Ivan R. Judson]

This is very cool; we need to corner bob at the AHM and figure out how to
get this functionality into AG2 since it is one of our feature requests :-)

--Ivan

PS -- just highlighting windows would be good for us for now.

> -----Original Message-----
> From: Robert Putnam [mailto:putnam at tonka.bu.edu] 
> Sent: Tuesday, April 22, 2003 12:00 PM
> To: jtvh at bu.edu
> Cc: papka at mcs.anl.gov; glenn at bu.edu; stevens at mcs.anl.gov; 
> judson at mcs.anl.gov; disz at mcs.anl.gov
> Subject: Re: AG audio
> 
> 
>    Date: Tue, 22 Apr 2003 08:53:12 -0400
>    From: Jennifer Teig von Hoffman <jtvh at bu.edu>
>    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; 
> en-US; rv:1.2.1) Gecko/20021130
>    X-Accept-Language: en-us, en
>    CC: "'Glenn Bresnahan'" <glenn at bu.edu>,
>       "Rick L. Stevens" <stevens at mcs.anl.gov>, Ivan Judson 
> <judson at mcs.anl.gov>,
>       "'Terry Disz'" <disz at mcs.anl.gov>, putnam at bu.edu
>    Content-Type: text/plain; charset=us-ascii; format=flowed
> 
>    Hi Mike,
> 
>    Robert Putnam is developing this, and will be 
> demonstrating it next week 
>    at the AHM. He's currently running this with AG 1.x, but 
> developing it 
>    in such a way that it should be no problem to move it up 
> to 2.x once the 
>    various programming hooks are in place. He's planning to 
> eventually rely 
>    on AG2 authentication services to match vic and rat 
> streams appropriately.
> 
>    I'm cc-ing him here; Robert, can you say a bit more about it?
> 
> Sure.  Here's an abstract, plus a little internal memo I 
> wrote about it:
> 
>       This is a demonstration of a prototype AG system that 
> does left-right
>       panning of incoming Rat audio streams to their 
> corresponding Vic window
>       locations.  A modified version of Vic transmits window 
> locations to a
>       custom software application, which, in turn, issues 
> stereo positioning
>       commands to a standard (unmodified) Rat.  It is hoped 
> that the test
>       implementation will shed some light on the following 
> questions: What are
>       the hardware and software requirements of a positional 
> audio system?
>       What is the impact on system performance (e.g., 
> increased packet loss,
>       impaired echo-cancellation, etc.)?  Would another 
> mechanism, e.g.,
>       visually highlighting vic windows, be more effective at 
> helping users
>       identify the current speaker?  Does the system provide 
> an increased
>       sense of immersion?  Does it increase (or decrease) 
> intelligibility and
>       overall audio quality?  Is it distracting?  And do the 
> benefits of
>       positional audio justify the system's increased cost 
> and complexity?
> 
> Vic and Rat have a built-in message scheme called the 
> "conference bus" or "mbus."  Rat uses it to do interprocess 
> communication between the user interface and the audio 
> engine, but the standard AG Vic pretty much ignores mbus 
> messages.  My agent uses the mbus to communicate with both 
> Vic and Rat.
> 
> When the agent starts up, it reads a little database file 
> that has associations between the Vic and Rat capture machines: 
> 
>    site: Purdue University
>    rat_name: Purdue University
>    vic_name: Purdue University MAIN
>    vic_name: Purdue University PRESENTER
>    vic_name: Purdue University AUDIENCE
> 
> As I said at the meeting, I expect that this part of the 
> implementation will go away when I use AG 2.0 services to get 
> these associations.
> 
> The agent then starts monitoring the stream of messages Rat 
> sends from its audio engine to its user interface, including 
> periodic (every second or so) updates about the incoming 
> audio streams.  It puts into a array of structs the names and 
> IP addresses of the remote Rats and the local ID that Rat 
> uses for them.  Here's the current struct definition:
> 
>    struct db {
>      int site_id;
>      char site_name[STANDARD_STRING_LEN];
>      char vic_names[MAX_VICS][STANDARD_STRING_LEN];
>      char rat_name[STANDARD_STRING_LEN];
>      int last_xi;
>      int num_vics;
>      unsigned int data[3];
>    } Db[MAX_SITES];
> 
> My modified Vic puts a message on the mbus whenever the 
> operator adjusts a window location.  The message contains the 
> window XY location (scaled to floats between 0.0 and 1.0) and 
> the IP address and name of the remote Vic capture machine.  
> The agent reads the message off the mbus and tries to find 
> the corresponding rat capture machine, using a heuristic if 
> necessary. (Again, this will change in the future.)  If 
> there's a match, the agent puts a "3D audio" positioning 
> command (part of Rat's standard equipment) on the mbus.  The 
> message contains the local Rat ID for the stream, the azimuth 
> (-90 to 90 degrees) and whether to do simple panning or HRTF 
> (I'm using the former because I think it sounds better over 
> loudspeakers).
> 
> Note that this test implementation is made a little simpler 
> because interprocess communication is built into Vic and Rat, 
> and they are running on the same machine.  If this were a 
> more typical AG setup, I'd have to write a little socket code 
> for messages between Vic and the agent.
> 
> 
> Robert Putnam
> putnam at bu.edu
> (617) 353-8295
> 
> 
> 
> 
>    (P.S. Congrats on such a successful AG retreat!)
> 
>    - Jennifer
> 
>    Michael E. Papka wrote:
> 
>    >Glenn and Jennifer,
>    >
>    >Can I get more info on this?
>    >
>    >Enhanced AG Audio
>    >-	Developing positional audio for the AG
>    >-	Associate rat and vic streams
>    >-	Locate vic window on display surface, Uses a modified vic
>    >-	Send positional audio info to rat, Uses rat 3d audio 
> capabilities
>    >-	Will demonstrate proof of concept at AHM
>    >
>    >Thanks
>    >Mike
>    >  
>    >
> 
>