[AG-TECH] Audio and Video Syncronization

[Jim Farrar]

We had this or a similar problem at NCSA at one time.  There was a lag in 
rat that grew larger the longer rat ran.  Every time  we re-started rat the 
sync problem would get better, but then as rat ran the delay would come 
back and keep getting larger.  We never could find and solve the exact 
problem, but as we upgraded cards and drivers, the problem went away.


At 06:01 AM 6/27/01 -0400, Will, Rodger (R.) wrote:
>I am very new to this technology, but I assumed that with vic and rat 
>being independent programs, that sync would be a challenge.
>
>One of our nodes employs an automatic camera/microphone system that 
>switches vic's input to the active speaker's camera and microphone. I 
>would assume that this effect would be akin to high-motion.
>
>I will watch the stats today, and pass along the information.
>
>I agree that someone urgently needs to solve this problem in software.
>
>-----Original Message-----
>From: Markus Buchhorn [mailto:Markus.Buchhorn at anu.edu.au]
>Sent: Tuesday, June 26, 2001 11:22 PM
>To: ag-tech at mcs.anl.gov
>Subject: Re: [AG-TECH] Audio and Video Syncronization
>
>
>At 08:21 PM 26/06/2001 -0500, Marty Hoag wrote:
> >However, I'm not sure how much
> >effect [ntp] will have on the actual audio/video syncronization.  I've had a
> >feeling that rat or vic can drift, especially if there is packet loss or 
> other
> >weirdness but others would know more about why that is.  I'd also check 
> things
> >like the rat settings to make sure everyone is at 16khz audio and look 
> at the
> >playout and decode stats for remote sites
>
>
>I'd always understood/assumed (I may be wrong) that the sync problem was
>due to the apps being totally independent and sizing their own playout
>buffers (or delays) independently. Hence if jitter and/or loss get larger
>for one stream over another (packet size issues?) the playout buffer is
>"stretched" (or delay increased) and sync goes out the window. I think rat
>and vic try to keep this to a reasonably constant level, and drop the
>playout delay whenever they can - but they are doing it independently.
>
>Audio can catch up during silence, video during "static" periods - so it
>can also be a function of the content
>
>So Marty's suggestion to watch the playout delay stats is spot on. You need
>to watch both audio and video and how they relate to each other.
>
>"Professional"/commercial software does audio and video in the same app, or
>tighly linked apps, and I'd understood it to be (partly) for that reason.
>
>Has anybody looked at MBUS hooks for vic and rat? Synch is one of the areas
>that it is meant to address.
>http://www.ietf.org/internet-drafts/draft-ietf-mmusic-mbus-transport-06.txt
>
>Perhaps one dirty option for now is to force vic/rat to drop late packets
>in preference to variable buffers - set the playout delay to a constant...
>
>Just my two bits worth. However, I can't stress enough how important it is
>to ultimately get this right - we have to put up with enough crap from the
>ISDN VC guys without giving them more ammunition :-) (not that ISDN is
>always fully synched either...). There's no reason why we couldn't get the
>synch perfect. Bundled MPEG streams would be another way we could do it,
>with a little effort...
>
>Cheers,
>          Markus
>
>P.S. Lip synch can be lost at the *source* if the audio and video encoders
>aren't time synched to each other (and ditto the decoders at the receiving
>end). So it is worth doing ntp or similar. Tying it to a specific ntp
>server on the Net I don't believe is necessary though, a local ntp server
>should suffice.
>
>P.P.S. I'm not sure why Rodger leaving the room should make it better
>(presuming it is not a personal reflection :-) ). If the encoding and
>decoding is on the same machine it may be that it reduces the pressure on
>the CPU on the encoding side or NIC on the machine. If they are on
>different machines it may be reflecting network problems somewhere upstream
>(sending less/smaller packets if the video is static and the audio is
>quiet). You could try watching a one-way stream, with nothing sent in the
>other direction and see how it behaves, and then turn on your transmission.
>
>Markus Buchhorn, Faculty of Engineering and IT,          | Ph: +61 2 61258810
>email: markus.buchhorn at anu.edu.au, mail: CSIT Bldg #108  |Fax: +61 2 61259805
>Australian National University, Canberra 0200, Australia |Mobile: 0417 281429
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/ag-tech/attachments/20010627/d869f4e5/attachment.htm>