[AG-TECH] Packet Loss

[Markus Buchhorn]

At 10:53 AM 24/08/2006 +1000, Chris Kendrick wrote:
>For some reason I was getting ~5sec patches of huge packet loss (as
>shown in the RAT reception quality matrix) roughly every 3 minutes from
>them (voice went extremely choppy), but they were always receiving good
>audio from me.

Was it a regular interruption? i.e. every 180 seconds or whatever?

Packet loss can be very uni-directional, and is usually unlikely to be the software or tools (until you stress the bandwidth or cpu - e.g. vic claims packet loss when it fails to service the network buffer quickly enough due to cpu load; it's local packet loss, not network packet loss). The bridge might be suspect, so testing with another bridge would be good. Also, check by time of day.

>My question is what tools tecniques can I use to further diagnose what
>is going wrong if it happens next time?

I'd start with monitoring the traffic very carefully. Did vic show similar drops? You can watch its traffic graphs and generate a nice picture to take to your network folks. You can run things like iperf if you're really keen.

What it sounds like to me is a cpu-bound router on the path. It might or might not be actual congestion, but if it's regular then somebody is either generating a significant burst of traffic at that interval, or a burst of cpu load. I've had some experiences like that, most recently with the AARNet router in Seattle, which was being probed (legitimately) for its routing tables every 60 seconds. That SNMP query tipped the poor router over the edge :-/ so I saw massive packet loss every 60ish seconds. Took me a while to convince the network folks to chase it down, but they monitored it at the edge of campus and saw what I saw, and I could then show it was happening off-campus.

You really need to get friendly with the network engineers on the path. Odds are it'll be close to Melbourne or Bristol, so I'd start there. It's still possible that it's on the broader backbone, but that means getting even more people looking at it. If it's not on the campus network at either end, I'd start with the AARNet folks in Oz, and the JANET folks in the UK.

Cheers,
        Markus


Markus Buchhorn, ANU Internet Futures             |Ph: +61 2 61258810
Markus.Buchhorn at anu.edu.au                        |Fx: +61 2 61259805
The Australian National University, Canberra 0200 |Mob: 0417 281429