Myricom: Extreme Performance Ethernet

Myri-10G "Gen2" (5 GT/s) PCI Express Network Adapter with two 10GBase-CX4 network ports for performance and failover (20Gb/s throughput)

Special-order Product: This low-profile (half-height) PCI Express add-in card is not fully compliant with the PCI Express Card Electromechanical Specifications. The lower CX4 connector intrudes into a "keep-out area" of the circuit board and may interfere mechanically with motherboard components in some hosts. Thus, this Network Adapter is a special-order product that should be used only for applications in which a low-profile card is required, and in which it has been determined that there is no mechanical interference in the host. If a low-profile card is not required, the standard-height 10G-PCIE2-8B2-2C Network Adapter with a standard PCI faceplate should be used.

Product Codes:

• 10G-PCIE2-8B2L-2C: Network Adapter with a low-profile PCI faceplate (not available with a standard-height PCI faceplate)

Key Features:

• Low-profile (half-height) PCI Express x8 add-in card
• Dual-protocol network ports, 10-Gigabit Ethernet or 10-Gigabit Myrinet
• Wire-speed performance from both network ports concurrently
• "Gen2" (5 GT/s) PCI Express x8 host port
• Firmware-controlled stateless offloads
• 14 Watts typical

Introduction: This Network Adapter appears to the host operating system as two independent PCI Express devices, one for each network port. Each of the two devices (Lanai-Z8ES chips) can carry traffic at rates approaching 10Gb/s (~9.9 Gb/s with a 9KB MTU, or ~9.5 Gb/s with a 1500B MTU). Together, the two devices can carry traffic between the host and the network at rates approaching 20Gb/s.

If both devices are using the bundled Myri10GE Ethernet software distribution and if the hosts have sufficient memory bandwidth, the netperf TCP_STREAM test with two streams shows an aggregate bandwidth of ~19.8 Gb/s with a 9KB MTU, and ~18.9 Gb/s with a 1500B MTU. These netperf 2.4.3 tests were between Intel Nehalem-EP hosts running the RHEL5 kernel 2.6.18-92.el5. Performance will vary with the host and operating system.

Specifications:

Myri-10G network ports (2): 10GBase-CX4, 10+10 Gbit/s data rate, full-duplex. The ports are fully compliant with the IEEE 802.3ak specifications for the 10GBase-CX4 10-Gigabit Ethernet Physical layer (PHY), and can operate with either Ethernet or Myrinet protocols at the Data Link layer. When operating in Ethernet mode, the ports support Ethernet flow control as defined by IEEE 802.3x. The ports may be connected with standard 10GBase-CX4 cables up to 15m in length. (See this Guide to Myri-10G PHYs (pdf).)

PCI Express host port: This Network Adapter is an x8 (8 lane) "Gen2" (5 GT/s) PCI Express Add-in Card. It is capable of exchanging data with a host computer at up to 4 GBytes/s (500 MBytes/s per lane) data rate in each direction, full-duplex. This Network Adapter is fully compliant with the PCI Express Base Specification Rev. 2.0 at 5 GT/s or 2.5 GT/s. The circuit-board edge connector of the Network Adapter will fit mechanically in x8 or x16 physical slots in host computers. The Network Adapter auto-negotiates operation in the widest available mode (x8, x4, x2, or x1) and highest data rate (5 or 2.5 GT/s) supported by the slot it is plugged into.

As noted above and under "Physical Characteristics" below, this product is not fully compliant with the PCI Express Card Electromechanical Specification Rev. 2.0.

A PCI Express switch chip connects the host port to two Lanai-Z8ES-based PCI Express endpoints via two x8 (8 lane) PCI Express "Gen1" (2.5 GT/s) links. The Lanai Z8ES is fully compliant with the PCI Express Base Specification Rev. 2.0 (2.5 GT/s only).

Optional PCI Express capabilities supported: Advanced Error Reporting, Function-Level Reset, Device Serial Number, up to 16 outstanding read requests, up to 2KB MaxPayloadSize for all packet types, MSI and MSI/X, SMBus access.

These "Gen2" PCI Express Network Adapters will operate correctly in "Gen1" (2.5 GT/s) PCI Express slots in a host, but require "Gen2" (5 GT/s) PCI Express slots to achieve full performance.

Network Adapter processors and memories (2): The Network Adapter is based on two Myricom Lanai Z8ES chips bridged to the host port with a PCI Express switch chip. Each Lanai Z8ES includes a programmable RISC, a set of packet engines, and 2MB of fast SRAM. The RISC, packet engines, and SRAM inside the Lanai-Z8ES chip operate at a clock rate of 364.6MHz. Byte parity is generated and checked on all on-chip memories.

EEPROM (2): 1MB, which includes the firmware required for PCI device initialization and an Etherboot (UEFI- and PXE-compatible) driver. The driver loads the matching firmware as required by the mode in which the Network Adapter is operating. The EEPROM can be re-programmed in-place by the Lanai Z8ES.

LEDs (4): The yellow LEDs on the PCI faceplate are controlled by the firmware running in the Lanai for this port. The interpretation of the yellow LEDs is different for different firmware. There is also a green LED for each port: off indicates that the link is down, on indicates that the link is up, and blinking indicates traffic.

Physical characteristics: The circuit board is a low-profile (half-height) add-in card as defined in the PCI Express Card Electromechanical Specification Rev. 2.0, but does not comply fully with these specifications. There is not enough space within these specifications to include two CX4 connectors on a low-profile faceplate. As is common practice in other commercial Network Adapters with two CX4 ports, the lower CX4 connector intrudes into a "keep-out area" of the circuit board and may interfere mechanically with motherboard components in some hosts. Thus, Myricom does not include this Network Adapter in the regular product list, but only as a special-order product.

Height 68.9mm (exclusive of the PCI faceplate), length 142mm (exclusive of the PCI faceplate and connectors), total thickness 22mm, weight 74g (0.163 pound) including the PCI faceplate. The Network Adapter can be supplied only with a low-profile PCI faceplate.

Power: Under conditions of maximal bidirectional traffic between both network ports and the host, the Network Adapter consumes 14.0W typical, 14.7W maximum.

Power details: The Network Adapter is powered from both 3.3V and 12V from the PCI Express port. The 3.3V current is 2.34A (7.72W) typical, 2.46A (8.12W) maximum. The 12V current is 0.52A (6.24W) typical, 0.55A (6.60W) maximum. Total power is thus 13.96W typical, 14.72W maximum.

Environmental: Operating: Temperature 0C to 55C up to 10,000 foot altitude with 100LFM minimum airflow. Relative humidity 15% to 80% @ 50C, non-condensing. Storage: Temperature -40C to 70C. Relative humidity 90% @ 65C.

Regulatory Approvals: Fully compliant with EN55024 (1998 w/A1: 2001 & A2: 2003), EN55022 (1998) Class A, AS/NZS 3548 (1005 W/A1 & A2: 97) Class A, CISPR 22 (1997) Class A, FCC Part 15 Subpart B Section 15.109 Class A, VCCI (April 2000) Class A, & ICES-003 Class A (ANSI C63.4 1992). See the Index of Myri-10G Regulatory Reports.

Reduction of Hazardous Substances: These network adapters are RoHS-compliant under the server exemption.

Myricom-supported software: These network adapters may use the included (bundled) Myri10GE software for 10-Gigabit Ethernet operation, or optional software distributions including MX-10G™, VideoPump™, Sniffer10G™, or DBL™. This software is distributed from the Myricom Software & Customer Support page.