Network the NAS

[LunarMist]

Here is my current configuration.



The 10G SFP+ in the three NAS and two tower computers are all from 2-port direct attach PCIe cards (mostly X520-DA2 or clones).
I'm trying to the improve the NAS connectivity without slowing down the performance by having two NAS on one switch with one 10G line to each computer. The Main NAS (12-bays) is capable of 800-1000MB/sec. speeds since it has the Xenon CPU, 48GB ECC RAM, and modern UltraStar array. Unfortunately it only has a single PCIe slot. The 2nd NAS (newest 8-bay) has the ZFS and encryption, so it's limited to ~400-500 MB/sec. and the 3rd NAS is older (internal 8-bay with 8-bay expansion) but with similar speeds.

My feeling is that the 2nd and 3rd NAS should be on a switch with all three computers and the two desktop computers should also be wired directly to the Main NAS if this is possible. I'd really like the NUC to see the main NAS at 2.5G speeds. (That NAS only has natively GbE ports.)

If it makes any difference, none of the NAS or computers are connected to the internet through the above mentioned networks. I have a separate
1GbE network for that.

 

[Mercutio]

This guy give your 8x 10Gb SFP+ ports that'll negotiate down to 2.5Gb if you just want to deal with what you have now. For 100% more money, you can get one that has a brand name on it.

The Trendnet DOES support management functions, so if you wanted to have one VLAN for your internet connections and another for back-end, it'll handle that.

 

[LunarMist]

2.5Gb has to be the normal connector, not the SFP style.
I was looking at QNAP, but it seems all the switches with more than two 10G SFP+ have fans and are noisy.

Amazon.com

 

[Mercutio]

2.5Gb has to be the normal connector, not the SFP style.
I was looking at QNAP, but it seems all the switches with more than two 10G SFP+ have fans and are noisy.

Amazon.com

You run the 2.5s to a 2.5Gb switch and SFP them to the bigger guy.

 

[LunarMist]

So I need two switches. Is having only one switch slowing down the overall speeds when moving data around from NAS to NAS? It seems like it would be the opposite.

I'm not finding any cables from SFP+ to 2.5GbE. There are some SFP+ to 10GbE adapters, but many complaints of them running very hot and burning out early.

If it makes any difference I'd like to keep the budget around $600 or so.

 

[Mercutio]

You can use a media adapter to get your cables all on one standard and you'll probably have to manually assign the port speed on the switch rather than have it auto-negotiate. 10GbbaseT is kind of notorious for high power draw compared to 1Gb or 2.5Gb. Media adapters can be REALLY picky about when they work but it looks to me like a pair of adapters will be an inexpensive experiment.

 

[Handruin]

You could use this switch for the 2,5GB RJ45 and uplink to one of the other switches Merc mentioned

Amazon.com

 

[LunarMist]

I'm pretty sure this is getting too complicated. I'm reading that my old X520-DA2 SFP+ cards are not liking modern switches and also needing two switches is not worth it.

Maybe I can put a quad port X710-DA4 in the main computer and have 10G access to all 3 NAS directly. Then I would get one of these simpler switches. https://www.trendnet.com/products/2-5g-switch-sfpplus/6-port-multi-gig-switch-TEG-S562 The 6W power drain is appealing.
If I understand correctly the SFP+ output of one computer would go through that switch and into the Main (Synology) NAS. Then a 2.5gBe port would be connected to the NUC and/or laptop.

 

[Handruin]

Is this what you are describing?

 

[LunarMist]

I have this now.

 

[LunarMist]

This is what I proposed.

 

[LunarMist]

I obtained the X710-DA4. Despite having no Win 10 drivers, it does work with the Intel Win 11 drivers.
Each NAS is connected to the Main computer separately with 10G so I can copy from any to another at full speed of the NAS.
The Main and 3rd NAS are connected to the 2nd computer with 10G, but the 2nd NAS is connected to the 2nd computer with 2.5GbE. I could get another X710-DA4 for the 2nd computer, but it is such a PITA to access and install that it is probably not worth the hassle.
I would probably be OK with swapping the 2.5GbE/NAS #2 and 10G/NAS #3 on the 2nd computer since the 2nd NAS is much more deserving of 10G than the slower 3rd NAS, but I would have to add a 2-port 2.5GbE on the 3rd NAS that only has a single port card in it now. (The 2nd NAS has built in 2.5GbE.)
Overall, each computer is now accessing all NAS with at least 2.5GbE.

 

[CougTek]

You draw network diagrams like coaches draw football plays :

 

[LunarMist]

I was trying to use the Handruin diagram. Is there a good tool for it that I can use easily and is not too expensive?
I never understood the "football" diagrams, but I know the NAS connections are too convoluted.
I've already changed the connections again. Ideally I would use 25G and connect everything to one fancy switch, but I have real budget limits nowadays and copying/moving from one NAS to another is way slowed down with only one line.

 

[Mercutio]

You can get a Visio product key from g2a.com for about $20.

 

[Handruin]

I used some free web based diagram tool to make that initially. I liked Visio back in the day but haven't used it in a hot minute

 

[CougTek]

For those fortunate enough to work for a networking device manufacturer or a reseller, one tool that's quite useful to create network diagram is Intangi Iris.

You can see a demo of the program when used to create a network with Ruckus (former Brocade switches) devices.

Several other manufacturers lists their products in that tool too. You can try to open an account to obtain it for free if your company isn't already subscribed. https://www.intangi.com/

 

[LunarMist]

It's not pretty, but will work for now.

 

[Mercutio]

This is probably relevant to some interests. 4x2.5GbE, 2x10GbE SFP+. $50 and runs on under 10W.

 

[Handruin]

Damn they really are getting cheaper. Any bets if it can hit full port speeds on all?

 

[LunarMist]

This is probably relevant to some interests. 4x2.5GbE, 2x10GbE SFP+. $50 and runs on under 10W.

The similar TrendNet switch (TEG-S562, shown in the previous posts) was only $89 and uses 6W. It's not made in PRC.

I'm not seeing any performance drop compared to having no switch. Transfer rates are 1100-1200MB/sec. to the NAS cache.
It runs about 285-300MB/sec. on the 2.5GbE. Copying from one NAS to another is about half speed, but that's what the Main computer is for.

 

[Mercutio]

Damn they really are getting cheaper. Any bets if it can hit full port speeds on all?

If it's around over over 1GB/s, I don't think people can quibble much. Mellanox NICs like the ones I use at home are around $50 each, so $150-ish to get 10GbE point to point seems pretty damned reasonable to me.

 

[sedrosken]

I have a variant of that switch -- seems there's one main design floating around and myriad chinese companies are using it. Mine is, for what it's worth, a Davuaz -- I paid 48 dollars shipped for mine a few weeks ago. I don't know about hitting the full 10Gb from the uplink SFP ports, but I'm certainly doing far, FAR more than 2.5 or even 5Gbps is capable of, that's for certain. I've clocked certain transfers at around 800MB/s if I'm hitting the file cache on the NAS. Maybe it's not what it's rated for, but for abusing what are supposed to be uplink ports and being a sub-fifty dollar switch, it's not bad.

 

[LunarMist]

Practically the only point of that type of switch with 2x 10Gb SFP+ is to bridge from 10Gb to 2.5Gb or to extend the twinyax length. It's too bad that there are not 4x 2.5Gb + 4x 10Gb switches under 15W and fanless.

 

[LunarMist]

AssRock 4x4 2.5GbE transfer speed is quite slow through the 10Gb SFP+ switch to the 10Gb SFP+ of the NAD.
The read speeds from NAS therough the switch to the 4x4 are good. The 4x4 has 2.5GbE Realitek. I don't recall this same issue with the NUC 11 which has an iNtel 2.5GbE. What would be causing this? Is there a compatibility defect of the switch?

 

[ddrueding]

In Windows you should be able to see what speed the network connection has negotiated. Just make sure it hasn't settled on 1Gb. If it has, I'd start by swapping out the cable (just cause it is easy), then mess with drivers.

 

[LunarMist]

There is only one speed of 2.5GbE. How can I see the up and down speeds separately?
I don't know what the drivers want. Everything I install loses the connection.

 

[ddrueding]

I'm running Win11, so my screenshots won't help you.

Right-click on the network icon in the taskbar (down near the clock) and choose "properties" or something similar to that. There should be something called "link speed" or similar.

 

[LunarMist]

Yes, it is listed as 2.5Gb and I'm getting >200MB/sec. read/write to NAS #2 and NAS #3.
The drivers are from 2019, but the Realitek one that AMD suggested to use.
In the diagram above the NUC is now the 4x4 8840U with the Realitek 2.5GbE and the afflicted NAS is Main NAS.
I'm not sure if there is an issue with the chips in the 10Gb to 2.5Gb Switch, but don't have a clue on the switchly design. Is there a chip for the 2.5GbE to other 2.5GbE ports and then a separate chip for the 2.5GbE to 10Gb? The other 2.5GbE are obviously working fine. The 10GbE through and through is also fine, i.e., Computer #2 to Main NAS.