1TB RAID 5 array

[Stereodude]

I'm in the process of building a RAID 5 setup with 6 WD2000JB drives. So far I have 4 of the drives. I've got my eye on a 3ware 7500-8 card, but 3ware just announced a 7506-8, which is supposed to have up to a 25% performance increase.

Now I'm going to be stuffing this thing in a Soyo Dragon mobo with a AMD 1400 that doesn't have a 64bit PCI bus, so it'll be bus limited. It will function as a server on my home network. So I figure the 25% speed increase doesn't matter to me any. I plan on upgrading my network to gigabit at the same time. Linksys's $175 8 port gigabit switch is exactly what the doctor ordered.

Anyhow getting around to my question, I wonder if the 3ware card is even really need, or if software RAID5 under Linux or W2k server will be good enough. I don't really like the idea of having my RAID setup dependent on the OS.

Any tips/suggestions?

 

[CougTek]

Hmmm...a 6 drives RAID 5 array on a 32bit PCI bus plus a gigabit NIC. You'll be more than bus limited, you'll be bus choked.

If you look at the overall setup cost, adding a higher-end motherboard with 64bit PCI slots wouldn't increase the total price in % term. Unless you badly need 1GB of storage, you should consider trading a drive for a more serious motherboard IMO.

The Tyan 2466N recently had a ~15% price drop on my list during the last month. And I have seen some places where the GigaByte MPX board was sold for less than 200$ several months ago. The MPX chipset is quite old, but so is the Athlon 1.4GHz you intend to use.

 

[Stereodude]

I already have the rest of the hardware. Copper gigabit is only good for about 25-30Mbytes/sec anyhow unless you have a really expensive NIC with a co-processor on it to offload the load. So drawing 25-30 meg a sec from the RAID array to the ethernet card shouldn't really overload it. At least that's my theory. I can copy fast than that from HD to HD (using the 33Mhz 32bit PCI bus and two High point controllers).

 

[Mercutio]

In my experience, even with high-end 64bit ICP SCSI RAID controllers, RAID5 is slow. Software RAID5 on a pure file server (NT4/2000 or Netware - I haven't done soft RAID5 on Linux) is somewhat better, particularly for write speed. It comes down to balancing loads. Since you're talking "workstation", I'm guessing your CPU will largely be busy with other things, so you'll be better off with hardware (anecdotally, I've seen SQL Server machines go from 70% CPU load to 40% by moving to hardware RAID from software).

 

[Stereodude]

According to the last reviews at StorageReview of a 3ware card, and the benchmarks on 3ware's site the card should be good for over 40Mbytes/sec write and beyond what the 32bit 33MHz PCI bus can handle for reads RAID-5. See http://www.3ware.com/products/benchmarks.asp

 

[Stereodude]

Well, the hardware is all ordered. I have 5 of the 6 HDs in my posession. The last is on it's way from NewEgg and the 3ware 7506-8 is on it's way from Hypermicro. Hopefully I'll get all the stuff by friday so I can set it up over the weekend.

 

[The JoJo]

Stereodude, what gigabit NIC are you using? Could you please post some performance numbers for it?

Please post numbers about your raid-system when you get it up and running! Nice system.

I'm thinking about upgrading to gigabit at home too, for faster access to my file server.

 

[Stereodude]

Stereodude, what gigabit NIC are you using? Could you please post some performance numbers for it?

Right now I only have the built on gigabit on my Asus board. It's a Broadcom NetXtreme adapter. I'll add a linksys 8 port gigabit switch, and gigabit cards to the other PCs soon.

 

[Handruin]

If you have the time, I'd also be curious to see some numbers from the gigabit setup along with the RAID5. I'd like to see how bad small file transfers would be.

Maybe I have my info messed up, but a GBE has to send thousands of frames because the linksys switch doesn't support jumbo frames? This causes a large amount of CPU usage, doesn't it?

 

[Mercutio]

Yes.

But if you're doing gigabit xfers, it's not like you'll be spiking for very long. Unless you're like me, and regularly need to send 30GB from one comptuer to another.

 

[Handruin]

Also, let me know if you can find drivers for that broadcom card. (hopefully your asus board came with a driver CD) We received some new Dell 6650's at work that came with dual broadcom cards and I couldn't find drivers for them, so we used whatever Dell had on their site.

 

[Handruin]

I have about 18GB worth of valuable data I would transfer nightly along with all my DB backups. It might be better to buy two cards and skip the switch. I'll have to look into the intel 1000MT cards, maybe they support jumbo frames and offload some of the CPU work. I can't really justify the cost, but it would sure speed up the copying of all those database backups.

Then the next question is, how do I make my computer use the GBE adapter instead of my 10/100 card during data transfers? My two main machines are connected together via a 10/100 switch. If I were to add another NIC into each of them, how would I specify my backups to use that adapter? Would I map a drive using their private IP, or is there an easier way?

 

[Mercutio]

I'd suggest, if your routing equipment supports it, putting the PCs' 100mbit cards on different subnets and the GbE cards on the same subnet.

 

[Handruin]

The GBE would be a direct connection so I could put those into any subnet.

 

[Mercutio]

I'm suggesting that the two GbE cards share a subnet so there's only one route for packets to take.

 

[Howell]

Since there would be only one physical route for the packets to take anyway, wouldn't Layer-2 work it out?

 

[Mercutio]

If you've got GbE - GbE on one network, and 100Mb - switch - 100Mb on another, there would be two possible routes. Which would be used in preference would then be an issue of binding order and in that circumstance, whichever connection was at the top of the heap would more-or-less always be the route.

I guess you could remove the Client for Windows Networks from the 100Mbit adaptors, too. Then you'd only have the GbE route. So that's one way.

Assuming Handruin doesn't have a WINS or DDNS server sitting around (or at least that he's not using them) to act as a bridge, Windows networking basically only works on the local subnet (you can specify connectivity to an IP, but the hosts won't find each other with broadcasts and won't populate Networ Places etc). That's why I'm suggesting subnetting as a possibility.

I'm assuming Windows hosts here. Let me know if that's not a correct assumption.

 

[Stereodude]

Thanks for the information guys. I've since found that SMC has just put out (this month) 5 and 8 port Gigabit switches that have jumbo frame support. Models SMC8505T and SMC8508T. Pricing is supposed to be $135 and $195 respectively.

 

[Stereodude]

Thanks for the information guys. I've since found that SMC has just put out (this month) 5 and 8 port Gigabit switches that have jumbo frame support. Models SMC8505T and SMC8508T. Pricing is supposed to be $135 and $195 respectively.

I found them at E-cost for $109 and $148 shipped. They looks like a better deal. Here's the link for the 5 port model on SMC's page. I can't find the 8 port on their site. http://www.smc.com/index.cfm?sec=Products&pg=Product-Details&prod=310&site=c Let me know what you think.

Thanks.

 

[Stereodude]

Found this review on Digit Life http://www.digit-life.com/articles2/gigeth32bit/gig-eth-32bit-2.html Looks like Intel's cards are the way to go.

 

[Handruin]

Are you planning on buying the pro 1000MT with the SMC8505T? Even though 1000Mb isn't feasible in a 32bit PCI, the increase over 100Mb is at least 3 times the amount if you use the default frame size. I would notice the difference...it's enough to make your RAID5 break a sweat.

 

[Mercutio]

Actually as long as we're talking about the way to go, it probably involves an i875 motherboard with GbE over CSA.

But that's only if you swing in a P4ish sorta direction.

As far as the SMC switch goes, I wouldn't be pissed off if someone threw on at me.

I work with SMC managed switches from time to time. They've been impressive from a features standpoint, but they're also the only switches that I've installed that I've ever had fail for no good reasons (er, once out of maybe 15 installations).

 

[Mercutio]

The bigger limitation with GbE is cabling. Even with compliant Cat6, the best I've seen over copper - even with 64bit PCI - was something like 400Mbit, and that was with disk subsystems that should've been fast enough to saturate gigabit connections.

 

[Handruin]

Do they even make a true cat7 or cat8...if so would it help any, or is the only real way to get the speed is with fibre optics?

The graphs on the link stereodude posted look like they had higher than 400Mbit transfer rates using a 32bit PCI. Am I reading it incorrectly?

 

[Stereodude]

The graphs on the link stereodude posted look like they had higher than 400Mbit transfer rates using a 32bit PCI. Am I reading it incorrectly?

I think you're reading it correctly. I saw some reviews where they hit nearly 1gig with 64bit cards and large frames.

 

[Mercutio]

Hafta wonder where they got their cable.
There's no such thing as cat 7 or 8. Most Cat 6 is really 5e with a different label.

 

[Stereodude]

Hafta wonder where they got their cable.
There's no such thing as cat 7 or 8. Most Cat 6 is really 5e with a different label.

I think a lot of it depends where the cable is run and what interference it is subject to. Flourescent lights are disasters.

 

[Howell]

Hafta wonder where they got their cable.
There's no such thing as cat 7 or 8. Most Cat 6 is really 5e with a different label.

I think a lot of it depends where the cable is run and what interference it is subject to. Flourescent lights are disasters.

There is a physical limitation to the cable design. And then there is a physical limitation to the cable design once interference is taken into account. I'm sure the cat tests (5,6,7) allow for some interference but how much I don't know. So, a cat 5 cable in a pristene environment may be as good as a cat 6 cable in a non-pristene environment.

It's the ballasts in the lights that are the problem. I suspect the electronic ballasts jtr is fond of would not have the problem.

 

[Stereodude]

Well... I have the array built and formatted. Both of which took a long time to do. Now I guess I can start putting data on it.

 

[Stereodude]

Also, let me know if you can find drivers for that broadcom card. (hopefully your asus board came with a driver CD) We received some new Dell 6650's at work that came with dual broadcom cards and I couldn't find drivers for them, so we used whatever Dell had on their site.

You must not have looked very hard...

http://www.broadcom.com/docs/driver-download.html

 

[Handruin]

I promise you, that link was not there and their page was different only a few months ago. I haven't been back since. I wasn't the only one having problems finding the drivers on their site...

 

[Stereodude]

I promise you, that link was not there and their page was different only a few months ago. I haven't been back since. I wasn't the only one having problems finding the drivers on their site...

I believe you. I was just giving you a hard time.