[petsc-users] Open-MX: a low-latency alternative to IP over 1 and 10 Gigabit Ethernet

[Jed Brown]

This may be interesting to those solving with relatively small
subdomains on ethernet clusters.  Open-MX is an open source
implementation of the Myrinet Express (MX) protocol which operates over
generic ethernet hardware.  It is a driver for Linux (>= 2.6.15) and
does not interfere with IP traffic.  It works with the native MX backend
of several MPI implementations including Open MPI and MPICH2.

The big feature for PETSc users is that compared to TCP/IP, it offers
30% lower latency over GigE and half the latency with 25-30% higher
bandwidth over 10 GigE.  This could make a big difference for those
observing poor scalability on ethernet clusters.  In particular, if
-log_summary shows a lot of time in events like VecDot, VecNorm, and
VecScatterEnd.

More info here:

  http://open-mx.org


Jed