Fiber Channel help, SAN or JBOD?

[charmedmeat]

Well I'm interested enough in Fiber Channel and SAN's to start building a small one here in my home. Here's some of the reading material I've been digesting:

http://www.amdzone.com/articleview.cfm?articleid=1254&page=2

http://www.informit.com/isapi/product_id~{4DA8291C-1F61-41FD-BE00-FC12A7687DD6}/element_id~{F71E8B4A-42A2-4277-A79B-EFBE285785AE}/st~{D2A629B3-0A2F-4A11-B595-07175F7CEF60}/session_id~{521753BA-404C-43F6-B8A7-DB9C3332CDD3}/content/articlex.asp

Sorry for the long URL! Anyway I was wondering what the difference is between the AMDzone method of just hooking up a couple drives directly to the HBA and actually using a hub/switch and how would that work. Is it possible to use the AMDzone method of cabling a few drives together, then hook it up to a FC hub/switch and have access to it? Also I've noticed there are quite a few storage arrays from various manufacturers and I was wondering what it would take to make one myself. I'm interested in having a group of 10-20 drives in a RAID array attatched to an FC hub/switch building it myself. Are there enclosures available that can do this and is it possible to build a hardware RAID enclosure or am I limited to software RAID? Also is each drive in the loop seen as it's own drive?

FYI I'm a cheap bastard and I'm doing this as a fun project so 5-10K boxes are out of the question! I'm also not adverse to getting my hands in a little PCB design/soldering. Thanks for any help you might be able to give!

Chad

 

[Mercutio]

Do you know anything about networking, charmedcat?

A SAN can be conceptualized a lot like an ethernet network. Running one or two drives in a daisy-chain fashion (HBA to drive to drive to HBA) is pretty close to your old-fashioned 10base2 LAN (or xbaseT on a hub). Shared media, only one node talking a a time (and yes, the drives can be addressed seperately or as an array, at least on the controllers I'm familiar with).

In short, not something that's terribly appealing. The FC stuff I've worked with has all been switched arrays.

There are some other stumbling blocks. One is that true fiber optic cables for FC arrays are REALLY expensive. $30 - $50 in ebay-land for 1M cables, as are "real" enclosures and backplanes.

So, you're stuck with a bastardized copper-based adaptor board. The adaptors run about $50 apiece (or $50 per drive, anyway). You're looking at $75 or so for 10k 36GB drives, which really isn't bad at all, except that they're loud enough to wake the dead in a decent-sized group (so says a man with 10 SCA barracudas in cabinet in his spare bedroom). Copper cable is cheap, but by the time you've shelled out for enough adaptors to run copper to your 10 drives, you probably pushed the cost for the project into outrageous-land ($1350 without cables, enclosure or the HBA, for 360GB of power-hungry FC drives)

Also note that it isn't hard, when working with the kinds of FC controllers that you can find on Ebay, to build something much cheaper and much faster with regulation SCSI drives. The extras associated with FC, plus the mediocre performance of those old Qlogic controllers make a project like yours an expensive proposition, and of somewhat limited value, given that your setup won't be much like the goody boxes that "Mr. works for EMC" Handruin gets to play with every day.

 

[Bartender]

Welcome aboard Charmedmeat, drinks on the house!

 

[charmedmeat]

Yes I have worked on networks believe it or not! What I'm interested in is whether or not you can take a loop of drives (such as your 10base2 analogy) and hook it up to a FC hub/switch. Imagine 2 boxes with power supplies and 10 drives each hooked into an 8 port FC hub with an HBA machine hooked up as well. Would this work? I was a little confused from the overview discussing loop vs fabric methodologies.

You mention the bastardized copper-based adapter boards costing $50 a piece. Which boards are you referring to? I noticed on the amdzone article that he was using a backplane (which if you go to the manufacturer's page he also has tcards as well). Is it possible to make one of these yourself without too much difficulty?

BTW I have a friend that works for EMC so I might be able to get some hand-me down gear if need be

 

[charmedmeat]

Welcome aboard Charmedmeat, drinks on the house!

heh! Thanks long time lurker, first time poster!

 

[Mercutio]

Small board. We all know each other pretty well. New guy shows up, maybe he knows networking, maybe he doesn't. That's why I asked.

If you're a reformed lurker, may I ask where you found us?

You can run multiple loops off a hub or switch.

I'm not an electronics guy, but my understanding of the copper-based TCards is that they're a very simple product to build (this according to EE-types on slashdot). My understanding of the amdzone "backplane" is that it's essentially a pair of tcards on a single interface. You can get similar products elsewhere on the web, but everyone seems to sell them for around the same price.

 

[CougTek]

heh! Thanks long time lurker, first time poster!

If you hold a Unix shell to your ear can you hear the C?

I've seen that signature before. Sivar?

 

[Handruin]

I think you'll find some good information in this document about the differences between a basic arbitrated loop like you suggested, and a hub/switch based FC-AL.

Hooking up a loop of drives into a hub may still suffer if one of the drives in the loop goes bad. Where as if they are all communicating using a hub, one can fail and the rest will continue to operate because the hub will loop out the failing device.

Generally when I think of FC-AL hubs, they are used more so for host connectivity to an array rather than connecting individual drives. I haven't seen a backplane type hub you can readily buy. Many of the hubs have GBIC's in them that you would have to convert to copper in order to connect to a single drive.

If you can find a hub using DB9 it would be more cost effective. The advantage of using a GBIC would allow you to stick the array in your neighbor’s house and run an SC cable from the hub to your HBA.

A fabric is much different than a hub setup. With a fabric, you create zones and place those in a zonesets to virtually manage host and array access. With a fabric, you generally have more than one switch and all the switches share the zoneset.

As Mercutio said, The tcards are for individual drive connections and the backplane allows you to connect more than one drive to a single card. I think both operate the same with the latter being a longer board with more connections on it. If you went with a backplane, you could probably mount it in a case and use removable drive cages for hot-swap ability.

 

[charmedmeat]

heh! Thanks long time lurker, first time poster!

If you hold a Unix shell to your ear can you hear the C?

I've seen that signature before. Sivar?

I saw it somewhere before, might've been on here!

 

[charmedmeat]

If you're a reformed lurker, may I ask where you found us?

You can run multiple loops off a hub or switch.

I'm not an electronics guy, but my understanding of the copper-based TCards is that they're a very simple product to build (this according to EE-types on slashdot). My understanding of the amdzone "backplane" is that it's essentially a pair of tcards on a single interface. You can get similar products elsewhere on the web, but everyone seems to sell them for around the same price.

Well I've been reading storagereview for some time and I found this site when the whole forum thing happened over there. I've never posted because generally if there's a question everyone answers it before I do or I just really don't have anything to say! Thanks for the info, I'll start looking for some specs or buy a tcard and RE one.

 

[Explorer]

Well I'm interested enough in Fiber Channel and SAN's to start building a small one here in my home. ~~~<snip>~~~ FYI I'm a cheap bastard and I'm doing this as a fun project so 5-10K boxes are out of the question!

You talk about being cheap, but then mention F-C switches. There's no way that you're going to be able to afford even a "low end" F-C switch. Even F-C hubs aren't cheap. Besides, the F-C compatibility bugaboo hits the second you mention the word F-C Switch.

The cheapest way to do a F-C array will be JBOD on a local loop. Anything more will not be cheap. As for F-C hubs, they are a better way to go when you have a slightly complex setup because you can eliminate or greatly reduce the potentially unreliable cabling problem associated with a F-C loop. You can also use multi-port F-C host bus adaptors, such as LSI Logics very nice 4-port F-C HBA to reduce potential F-C cabling problems.

Just a side note: You should wait it out and evaluate SAS (Serial Attached SCSI) once it becomes available (late this year). It will have every bit of Full Duplex channel performance of F-C for less money. And, you will have the added benefit of using cheap SATA drives as well.