[AG-TECH] Audio and Video Syncronization
Will, Rodger (R.)
rwill1 at ford.com
Wed Jun 27 05:01:33 CDT 2001
I am very new to this technology, but I assumed that with vic and rat being independent programs, that sync would be a challenge.
One of our nodes employs an automatic camera/microphone system that switches vic's input to the active speaker's camera and microphone. I would assume that this effect would be akin to high-motion.
I will watch the stats today, and pass along the information.
I agree that someone urgently needs to solve this problem in software.
-----Original Message-----
From: Markus Buchhorn [mailto:Markus.Buchhorn at anu.edu.au]
Sent: Tuesday, June 26, 2001 11:22 PM
To: ag-tech at mcs.anl.gov
Subject: Re: [AG-TECH] Audio and Video Syncronization
At 08:21 PM 26/06/2001 -0500, Marty Hoag wrote:
>However, I'm not sure how much
>effect [ntp] will have on the actual audio/video syncronization. I've had a
>feeling that rat or vic can drift, especially if there is packet loss or other
>weirdness but others would know more about why that is. I'd also check things
>like the rat settings to make sure everyone is at 16khz audio and look at the
>playout and decode stats for remote sites
I'd always understood/assumed (I may be wrong) that the sync problem was
due to the apps being totally independent and sizing their own playout
buffers (or delays) independently. Hence if jitter and/or loss get larger
for one stream over another (packet size issues?) the playout buffer is
"stretched" (or delay increased) and sync goes out the window. I think rat
and vic try to keep this to a reasonably constant level, and drop the
playout delay whenever they can - but they are doing it independently.
Audio can catch up during silence, video during "static" periods - so it
can also be a function of the content
So Marty's suggestion to watch the playout delay stats is spot on. You need
to watch both audio and video and how they relate to each other.
"Professional"/commercial software does audio and video in the same app, or
tighly linked apps, and I'd understood it to be (partly) for that reason.
Has anybody looked at MBUS hooks for vic and rat? Synch is one of the areas
that it is meant to address.
http://www.ietf.org/internet-drafts/draft-ietf-mmusic-mbus-transport-06.txt
Perhaps one dirty option for now is to force vic/rat to drop late packets
in preference to variable buffers - set the playout delay to a constant...
Just my two bits worth. However, I can't stress enough how important it is
to ultimately get this right - we have to put up with enough crap from the
ISDN VC guys without giving them more ammunition :-) (not that ISDN is
always fully synched either...). There's no reason why we couldn't get the
synch perfect. Bundled MPEG streams would be another way we could do it,
with a little effort...
Cheers,
Markus
P.S. Lip synch can be lost at the *source* if the audio and video encoders
aren't time synched to each other (and ditto the decoders at the receiving
end). So it is worth doing ntp or similar. Tying it to a specific ntp
server on the Net I don't believe is necessary though, a local ntp server
should suffice.
P.P.S. I'm not sure why Rodger leaving the room should make it better
(presuming it is not a personal reflection :-) ). If the encoding and
decoding is on the same machine it may be that it reduces the pressure on
the CPU on the encoding side or NIC on the machine. If they are on
different machines it may be reflecting network problems somewhere upstream
(sending less/smaller packets if the video is static and the audio is
quiet). You could try watching a one-way stream, with nothing sent in the
other direction and see how it behaves, and then turn on your transmission.
Markus Buchhorn, Faculty of Engineering and IT, | Ph: +61 2 61258810
email: markus.buchhorn at anu.edu.au, mail: CSIT Bldg #108 |Fax: +61 2 61259805
Australian National University, Canberra 0200, Australia |Mobile: 0417 281429
More information about the ag-tech
mailing list