[Swift-devel] coaster IO buffer management

Mihael Hategan hategan at mcs.anl.gov
Thu Mar 10 12:38:10 CST 2011


cog trunk r3057 contains an automatic coaster service to worker TCP
buffer size management.

There is a (now hardcoded) range of shared buffer sizes and the code
divides that up for all the worker sockets. As new sockets come or go,
the size is adjusted accordingly. This is done to ensure that:

1. If few workers are present, they get a sufficiently large buffer size
to improve I/O performance.
2. If many workers are present, there are some attempts at limiting the
amount of memory consumed by TCP buffers. This should theoretically be
OK since it is assumed that many parallel transfers on small TCP buffers
will still result in good IO performance since latencies are now
parallelized.

Mihael




More information about the Swift-devel mailing list