[AG-TECH] Audio and Video Syncronization

[Rick Stevens]

The camera/mic switch is at the analog input level right ?  How long of a 
delay is there for the camera/mic system to detect that it needs to switch 
sources ?  My intuition is that the sync problem is probably only slightly 
related to this, but more likely related to networking (i.e. dropped 
packets) or CPU loading problems. Of course if it is easy to turn off then 
it would be worth doing the experiment to eliminate it from the set of 
possible causes.

At 12:03 AM 6/28/2001 -0400, Will, Rodger (R.) wrote:
>No echo promlems. Just sync....
>
>If a sufficiently advanced camera\mic switching system became available, 
>would this improve things?
>
>-----Original Message-----
>From: Rick Stevens [mailto:stevens at mcs.anl.gov]
>Sent: Wednesday, June 27, 2001 10:43 PM
>To: Will, Rodger (R.); Will, Rodger (R.); 'Ag-Tech (E-mail) '
>Cc: ''Markus Buchhorn' '
>Subject: RE: [AG-TECH] Audio and Video Syncronization
>
>
>ah.
>
>I would be concerned that it might confuse the echo cancellation too have
>you noticed any problems with that ?
>
>At 11:12 AM 6/27/2001 -0400, Will, Rodger (R.) wrote:
> >The existing room already had this installed.  We can disable.
> >
> >-----Original Message-----
> >From: Rick Stevens
> >To: Will, Rodger (R.); Ag-Tech (E-mail)
> >Cc: 'Markus Buchhorn'
> >Sent: 6/27/01 10:57 AM
> >Subject: RE: [AG-TECH] Audio and Video Syncronization
> >
> >
> >At 06:01 AM 6/27/2001 -0400, Will, Rodger (R.) wrote:
> > >I am very new to this technology, but I assumed that with vic and rat
> > >being independent programs, that sync would be a challenge.
> > >
> > >One of our nodes employs an automatic camera/microphone system that
> > >switches vic's input to the active speaker's camera and microphone. I
> > >would assume that this effect would be akin to high-motion.
> >
> >I should point out that we dont recommend this type of camera/microphone
> >
> >coupling.  What is the problem
> >you are trying to solve with it ?
> >
> >
> >
> > >I will watch the stats today, and pass along the information.
> > >
> > >I agree that someone urgently needs to solve this problem in software.
> > >
> > >-----Original Message-----
> > >From: Markus Buchhorn [mailto:Markus.Buchhorn at anu.edu.au]
> > >Sent: Tuesday, June 26, 2001 11:22 PM
> > >To: ag-tech at mcs.anl.gov
> > >Subject: Re: [AG-TECH] Audio and Video Syncronization
> > >
> > >
> > >At 08:21 PM 26/06/2001 -0500, Marty Hoag wrote:
> > > >However, I'm not sure how much
> > > >effect [ntp] will have on the actual audio/video syncronization.
> >I've had a
> > > >feeling that rat or vic can drift, especially if there is packet loss
> >or
> > > other
> > > >weirdness but others would know more about why that is.  I'd also
> >check
> > > things
> > > >like the rat settings to make sure everyone is at 16khz audio and
> >look
> > > at the
> > > >playout and decode stats for remote sites
> > >
> > >
> > >I'd always understood/assumed (I may be wrong) that the sync problem
> >was
> > >due to the apps being totally independent and sizing their own playout
> > >buffers (or delays) independently. Hence if jitter and/or loss get
> >larger
> > >for one stream over another (packet size issues?) the playout buffer is
> > >"stretched" (or delay increased) and sync goes out the window. I think
> >rat
> > >and vic try to keep this to a reasonably constant level, and drop the
> > >playout delay whenever they can - but they are doing it independently.
> > >
> > >Audio can catch up during silence, video during "static" periods - so
> >it
> > >can also be a function of the content
> > >
> > >So Marty's suggestion to watch the playout delay stats is spot on. You
> >need
> > >to watch both audio and video and how they relate to each other.
> > >
> > >"Professional"/commercial software does audio and video in the same
> >app, or
> > >tighly linked apps, and I'd understood it to be (partly) for that
> >reason.
> > >
> > >Has anybody looked at MBUS hooks for vic and rat? Synch is one of the
> >areas
> > >that it is meant to address.
> > >http://www.ietf.org/internet-drafts/draft-ietf-mmusic-mbus-transport-06
> >.txt
> > >
> > >Perhaps one dirty option for now is to force vic/rat to drop late
> >packets
> > >in preference to variable buffers - set the playout delay to a
> >constant...
> > >
> > >Just my two bits worth. However, I can't stress enough how important it
> >is
> > >to ultimately get this right - we have to put up with enough crap from
> >the
> > >ISDN VC guys without giving them more ammunition :-) (not that ISDN is
> > >always fully synched either...). There's no reason why we couldn't get
> >the
> > >synch perfect. Bundled MPEG streams would be another way we could do
> >it,
> > >with a little effort...
> > >
> > >Cheers,
> > >          Markus
> > >
> > >P.S. Lip synch can be lost at the *source* if the audio and video
> >encoders
> > >aren't time synched to each other (and ditto the decoders at the
> >receiving
> > >end). So it is worth doing ntp or similar. Tying it to a specific ntp
> > >server on the Net I don't believe is necessary though, a local ntp
> >server
> > >should suffice.
> > >
> > >P.P.S. I'm not sure why Rodger leaving the room should make it better
> > >(presuming it is not a personal reflection :-) ). If the encoding and
> > >decoding is on the same machine it may be that it reduces the pressure
> >on
> > >the CPU on the encoding side or NIC on the machine. If they are on
> > >different machines it may be reflecting network problems somewhere
> >upstream
> > >(sending less/smaller packets if the video is static and the audio is
> > >quiet). You could try watching a one-way stream, with nothing sent in
> >the
> > >other direction and see how it behaves, and then turn on your
> >transmission.
> > >
> > >Markus Buchhorn, Faculty of Engineering and IT,          | Ph: +61 2
> >61258810
> > >email: markus.buchhorn at anu.edu.au, mail: CSIT Bldg #108  |Fax: +61 2
> >61259805
> > >Australian National University, Canberra 0200, Australia |Mobile: 0417
> >281429