PCIe x1 NiC's (gigabit)???

[Stereodude]

Anyone have an opinion on PCIe x1 gigabit Ethernet cards?

Some people on another forum swore that a good NiC like a gigabit Intel add in card (even an older PCI one) offer better performance than the cheapo Realtek stuff found soldered onto motherboards (even though the Realtek's are PCIe x1).

I'm skeptical, but curious. I see Intel has a few single port PCIe x1 compatible cards like the Intel Pro/1000 PT Server and Intel Pro/1000 PT Desktop adapters. Those two have the same chipset, but have different features and prices. The latter has been replaced by the Intel Gigabit CT Desktop adapter. I have no idea how the newer CT compares to the PT.

I would think the server adapter would be more suited for my server, but I've got no idea if the extra features it has will be at all useful for my situation, or if the CT server adapter will offer any better performance than the CT desktop adapter.

 

[ddrueding]

I have heard the same thing, though all my machines (even my servers) just use the onboard NIC. If I were to get an add-on NIC, I would be looking for one with hardware offloading of stuff and perhaps encryption acceleration.

 

[LunarMist]

I have one of the Intel models - not sure which one. Performance seems about the same as the onboard port, but I'm just copying files between machines.

 

[Mercutio]

Biggest problem I have with Realtek NICs is that some of them just won't operate with some switches at speeds higher than 10Mbit, no matter what the link light is reporting. I've tested and verified this several times with 3Com and SMC switches against supposedly 100Mbit or 1000Mbit cards.

I stick an Intel Pro/100 or 3C905 in and everything magically works the way it's supposed to.

This is particularly aggravating since not even Intel puts Intel NICs on all its boards; consumer boards get Realtek.

Net result is that I buy lots of system pull NICs off ebay.

 

[sechs]

Some people on another forum swore that a good NiC like a gigabit Intel add in card (even an older PCI one) offer better performance than the cheapo Realtek stuff found soldered onto motherboards (even though the Realtek's are PCIe x1).

I think that this depends on your definition of "better performance."

If using even less processor time and improved compatibility with stuff you probably won't use is "better," then definitely. You could probably get some gain by playing solitaire with a high-end graphics card rather than built-in, as well.

 

[CougTek]

Support for Jumbo frames?

 

[Stereodude]

Biggest problem I have with Realtek NICs is that some of them just won't operate with some switches at speeds higher than 10Mbit, no matter what the link light is reporting. I've tested and verified this several times with 3Com and SMC switches against supposedly 100Mbit or 1000Mbit cards.

I have a 24 port 3com ProCurve 1800-24 (gigabit) in the basement and haven't had any issues with the Realtek 8111B / 8111C's on my mobos connecting at less than 1000Mbit/sec and based on the transfer speeds they really run at 1000Mbit/sec in real use. I did see a little quirkiness with one of my old systems (w/ Broadcom gigabit on the motherboard) when using a really old 25' CAT5 cable. It would only connect at 100Mbit until I replaced the cable with a newer one.

 

[Stereodude]

Support for Jumbo frames?

I think every gigabit NiC supports jumbo frames now days. However, I can't use them because I have a few 100Mbit/sec devices and one 10Mbit/sec device on the network that of course don't support jumbo frames.

I tried using VLANs and creating a gigbit only jumbo frame supporting VLAN, but didn't really see any performance improvement.

 

[Stereodude]

I guess I will do some benchmarks with the Intel NAS Performance Toolkit and see what shakes out. I have some PCI Intel Pro/1000 MT cards I can benchmark against the PCIe Realtek's.

 

[Stereodude]

Well, I bought a used Intel Pro/1000 PT Server Adapter off ebay and a new Intel Gigabit CT Desktop Adapter to do some testing with.

I did a little experimenting with iPerf today and I was surprised at how high the CPU usage was with the on-board Realtek 8111C [PCIe] ~48% (server) / ~67% (client) with an E5200 CPU. My other server with an on-board Intel 82547GI [CSA] was on the other side of the iPerf test had this CPU usage ~28% (server) / ~70% (client) with a 3.0gHz P4 w/ HT (Prescott). Bandwidth was >900Mbit/sec each way (sequential not simultaneous).

I wouldn't have expected the P4 to have lower CPU usage than the E5200 in the server role. I'm curious now to see how the two new Intel PCIe cards will compare in the server to the on-board Realtek 8111C.

 

[Stereodude]

So, I monitored CPU utilization while running iperf (with 5 parallel threads to max out the connection) with the two NiC's.

RT8111C

Server:
4k - 67%
8k - 66%
16k - 64%
32k - 64%
64k - 64%
128k - 62%

Client:
4k - 45%
8k - 33%
16k - 51%
32k - 51%
64k - 48%
128k - 48%

Intel Pro/1000 PT Server

Server:
4k - 45%
8k - 47%
16k - 49%
32k - 52%
64k - 53%
128k - 52%

Client:
4k - 35%
8k - 34%
16k - 37%
32k - 34%
64k - 35%
128k - 34%

Intel Gigabit CT Desktop

Server:
4k - 59%
8k - 59%
16k - 48%
32k - 48%
64k - 55%
128k - 52%

Client:
4k - 37%
8k - 33%
16k - 31%
32k - 36%
64k - 31%
128k - 38%

 

[Stereodude]

Here are the iperf results of the various cards in my basement server connected to my desktop and it's RT8111B. There's also an additional test with the Intel Gigabit CT Desktop adapter in my desktop in place of the Realtek 8111B.



I have another Intel Pro/1000 PT Server adapter on the way and am curious to see the results of the Intel Pro/1000 PT Server to Intel Pro/1000 PT Server combination.

 

[P5-133XL]

I read the numbers and say there's some value to the Intel Nic's compared to RealTek at least in a server environment where there is significant network traffic and CPU usage matters.

In a low usage environment like consumer or normal business usage the cheaper is better because it just won't make a significant difference..

 

[Stereodude]

There's definitely a CPU usage improvement. The iperf BW tests are little more muddy. The Intel cards are not consistently the fastest. There's also no correlation between the iperf BW tests and actual performance in Windows File Sharing.

I should also point out that in the chart the two identical colors are in the order of the legend. So for example the first yellow for 4k bar is Intel CT Desk. to RT8111B the second is the reverse combination RT8111B to Intel CT Desk. (there's a typo in the chart).

 

[Stereodude]

Here's the replacement graph for post #12 with the above mentioned typo fixed.

 

[Stereodude]

So the fastest NiC in one scenario is also the slowest in the other three scenarios. Classic...



Results are the average of 3 copies of a 5.5GB file using Windows Explorer.