MX-10G Performance Measurements

The user-level performance metrics ("micro-benchmarks") most often cited for cluster interconnect are:

• Latency for short messages,
• Sustained unidirectional data rate for large messages, and
• Sustained bidirectional data rate in which hosts send and receive messages concurrently.

For Myri-10G hardware with MX-10G software, the following performance is typical with one NIC per host:

MX-10G can support multiple bonded NICs per node. With two bonded NICs per node (see the link below), the data rates are nearly double those shown in this table, but the latency is still 2.2µs.

There are other performance metrics that are directly or indirectly predictive of application performance:

• The message length at which half of the sustained data rate is achieved is predictive of the performance of applications that use short messages. However, a graph showing data rate as a function of message length (follow the links below) is more useful.
• The host-CPU utilization per message is a very important metric for application performance, but is very small for MX and for MPI over MX. Host-CPU utilization is significant only for TCP/IP over MX (IPoM).

The links below will take you to detailed performance measurements for MPICH-MX over MX-10G.

The links below will take you to detailed performance measurements for Lustre-MX and PVFS2-MX over MX-10G.

Notes:

The latency measurements presented on these pages were, unless otherwise noted, between NICs connected point-to-point (switchless).

For performance of Myrinet-2000 under MX-2G software, please refer to the MX-2G Performance web page.