The Great Gigabit Ethernet Shootout

[Stereodude]

For some reason I wasted most of my day running iperf tests on all my Gigabit cards and every possibly combination of them. I ran three 30 second tests in each direction for the following TCP window sizes; 4k, 8k, 16k, 32k, 64k, & 128k. One NiC (in the chart title) was in my Windows XP x64 Server in the basement and the other NiC was in my Windows XP x86 (32-bit) Desktop (legend). In the "Server" test the NiC in the chart title sent data to the NiCs in the legend. In the "Client" test the NiC in the legend sent data to the NiC in the chart title.

Here are the consolidated results consisting of the minimum, average, and maximum of the 8k, 16k, 32k, and 64k tests. The 4k and 128k results were omitted because they are basically a duplication of the 8k and 64k results respectively.

Intel MT Desktop:

Intel CT Desktop:

Intel PT Server:

Intel PT Desktop:

Realtek 8111C:

 

[Stereodude]

I guess there are a few conclusions to draw from the data.

1) The Realtek PCIe x1 NiC's soldered on most motherboards offer best in class throughput performance regardless of the NiC on the other end.

2) 32-bit 33MHz PCI Gigabit NiC's do not perform well compared to their PCIe brothers.

3) An Intel Server and Desktop card with the same chipset do not offer interchangeable performance.

• For a good example compare the Intel PT Server and Intel PT Desktop numbers when connected to the Realtek 8111B or Intel CT Desktop where the Intel PT Server had about 5% better performance than the Intel PT Desktop.

4) The OS made a significant difference with many of the Intel cards.

• For example, the Intel PT Server card under XP x64 sending data to an Intel PT Desktop card under XP x86 delivered a minimum of 354Mbps, average of 504MBps, and max of 730Mbps. By swapping the cards, placing the Intel PT Server card under XP x86 sending data to an Intel PT Desktop under XP x64, performance grew to a minimum of 442Mbps, average of 688MBps, and max of 875MBps. That's an overall performance improvement of 26% simply by swapping the OS's.
  
  
• Another dramatic example is the Intel PT Desktop card under XP x86 sending data to an Intel PT Server card under XP x64 delivered minimum of 350Mbps, average of 527MBps, and max of 816MBps. When the cards were swapped, with the Intel PT Desktop card under XP x64 sending data to an Intel PT Server under XP x86, performance grew to a minimum of 533Mbps, average of 730MBps, and max of 918Mbps. An overall performance improvement of 29% simply by swapping the OS's.

 

[ddrueding]

Thank you for all the hard work Stereodude, and a very interesting conclusion! Those results are so non-intuitive, that I would have to wonder about their statistical significance. Any insight about the standard deviation of your numbers?

 

[Stereodude]

Standard deviation of which aspect?

In general, the iperf results are pretty tightly grouped. There's also a lot more data behind the numbers in the above graphs. There's 18 minutes of iperf testing and 36 tests to produce 6 numbers on the graphs. Here's an example.

Realtek 8111C - Realtek 8111B test case:

 

[Stereodude]

I should also point out that the above number throughput only. CPU usage is not taken into account. In general the Intel cards offer lower CPU usage, but iperf has no means to record CPU usage, so you end up watching Task Manager and try to guesstimate the average during the test (it bounces around) or running a much longer iperf test and watching the average in the Performance monitor.

Doing the latter with iperf set to max out the NiC, the RT8111B received an average of 939Mbps of 8k TCP window traffic and used an average of 27.2% of my Q6600 @3.0gHz. Sending an average of 935Mbps used an average of 25.9% CPU. Repeating the test on the Intel CT Desktop card received a average of 833Mbps and used an average of 23.6% CPU. Sending it did an average of 929Mbps and used an average of 16.4% CPU.

So the Realtek was able to get 34.5Mbps for each percent of CPU usage when receiving and 36Mbps for each percent of CPU usage when sending. In comparison the Intel CT Desktop did 35.3Mbps receiving, and 56.6Mbps sending.

 

[Mercutio]

I'm mildly fascinated by these results, but it does very little to convince me that the Realtek Ethernet chipsets are anything other than garbage hardware, given the real-world difficulties I have had with them.

 

[Stereodude]

I made no claims to the quality of the hardware, just the throughput.

On a related rant, I would have a 2nd PT Server card, but the ebay seller shipped me a PT Desktop card after having a listing for a PT Server card. So, I tested it above, but I want to see what PT Server to PT server performance looks like. I intend to use the PT Server card in my main server.

 

[sechs]

I might suspect that drivers and settings would have a greater sway on "performance" than the actual piece of hardware.