iSCSI SAN

[Handruin]

Who's done basic iSCSI setups? I'm new to them and want to figure out how to get this working for a learning project.

I'd like to setup a home test environment of VMWare ESXi with SAN storage so that I can use vmotion. Since I'm somewhat familiar with fibre channel from past experiences, I'd like to branch our to learn iSCSI as well.

Does it seem reasonable that I could build a system running an OS or freenas that has an iSCSI initiator to allocate shared storage to a LUN? In order for vmotion to work, I need two physical machines to see the same storage device.

I've put together a conceptual diagram of what I'm thinking might work. Does this make sense? Is it difficult to get iSCSI working? I've read that iSCSI traffic should be seperate from normal TCP traffic which is why I have two conceptual switches in my layout. I realize I could probably get away with one switch using VLAN which is also desirable (so that I can learn that also).

Are there certain NIC hardware I should consider for iSCSI? I was reading that the intel pro 1000 adapters should work for this just fine.

 

[ddrueding]

Funny you should post this right now. At this very moment I have the Microsoft iSCSI initiator open on my main file server, preparing it to host some stuff to my ESXi machines. I have made no progress so far, but it is quite a coincidence.

 

[Handruin]

What OS is your target system (file server), is it server 2003/2008? Are you going to us it as VM storage or are you just mounting it for OS storage? There is a iSCSI initiator doc if it helps?

I'm going to try some stuff tomorrow using either the MS target/initiator software or freenas and see if I can get vSphere to mount a drive. I'd like to build a full environment in my apartment to do some performance testing and other general configuration and learning. I'm really curious to see how iSCSI performs.

 

[ddrueding]

The target will be Server 2003. I'll just allocate a chunk of the RAID array to it and use the same GbE link for both. It will be a backup target for the VMs, they will continue to live local to the ESXi server.

 

[Handruin]

Are you going to connect the iSCSI initiator to each VM to have one common backup location?

I was browsing around looking up information on Windows 7 and iSCSI initiator and I was lead to Openfiler. Have you played with this software at all? Looks like they have a free version to play with.

 

[ddrueding]

I was going to mount the iSCSI to the ESXi server itself, and backup entire VMs to the store.

 

[ddrueding]

OpenFiler looks cool, but it looks like a bare-metal package? All I want to do is turn some storage space on my Server 2003 machine into an iSCSI target.

 

[Mercutio]

I've set up targets on Centos for my Windows machines to use, but it doesn't sound like that's terribly helpful for you, ddrueding.

 

[ddrueding]

No, but I did just download CentOS for some testing...I might start there if it's easier.

 

[Handruin]

Do you think something like the VisionTek 900296 Killer Xeno Network Card for $100 might work as a decent iSCSI TOE card under windows? I know it's a shot in the dark, but considering a traditional TOE card is probably expensive.

Slightly related, I am quite surprised to read that Linux has a lack of support for the TCP Offload Engine.

 

[Handruin]

I did some basic tests with iSCSI. I'm still not sure what constitutes good performance. I have some other IO meter testing to do with this.

Test:
machine A to machine B

machine A:
Core i7 860
win7 with microsoft iSCSI Initiator

Machine B:
AMD X2 4600 +
OpenFiler
1TB Samsung 7200 RPM

I test with and without jumbo frames (9K) using ATTO. The results improved with Jumbo Frames enabled.

I've done basic file transfer tests and I'm seeing about 760Mb/sec. During some of the ATTO tests I would see 990Mb/sec.

 

[ddrueding]

Looks good to me. Of course, my question would be what is the latency of a seek while a file copy is ongoing?

 

[MaxBurn]

IO meter should be telling on that, is it going through a switch? I wouldn't know what a good number for this would be though. As I understand it in theory at any rate the switch can slow down IOPS, I think that's one of the reasons they want a dedicated LAN for iSCSI. Actually one of you posted a highly enlightening link here on that not long ago somewhere here.

 

[Handruin]

I agree, the latency of the drive while a file copying is a decent test for me to try. I did run IO meter several times with different tests, but I don't really know which of the tests will give me meaningful results. I also don't know how to best interpret them. There are also 20+ parameters that I can mangle in the iSCSI configuration. I've done googling on them, but haven't noticed any huge differences with my changes.

This is going through a simple netgear GS108 8 port GigE switch. All basic IP traffic is shared so I do know that a dedicated switch is ideal for iSCSI traffic. I have plans to invest in one of the HP ProCurve 1810G so that I can isolate the traffic with VLANs and also try trunking.

I did some iperf testing the other night from my laptop to my desktop using the same switch and I was getting throughputs of 990Mb/sec which I thought was really good. I know that measurement in and by itself doesn't paint the entire picture, but I figured there would be more overhead slowing it down. My laptop was also quite noisy when the transfers were that intense. I could hear high pitch electrical noises inside more than I've ever heard before.

 

[Santilli]

Hi
I can't get sound right now that is correct, but there is a webinar on this very subject going on today, right now:
https://www2.gotomeeting.com/en_US/island/webinar/registration.tmpl?Action=rgoto&_sf=1

 

[Handruin]

I clicked the link, but it said there was no webinar available.

 

[Santilli]

http://www.stonefly.com/webinar/

 

[Santilli]

Took me awhile to find that link. The Webinar went over downloading iscsi, setting it up, step by step in 2003, and was generally good stuff, but, apparently the sound isn't working, at least for me.

 

[Handruin]

Thanks for letting me know about it. I was already able to get iSCSI working on windows XP and Windows 7 using the free Microsoft initiator. If you want to learn how to get it working, I can try to help. You'll need to create something that works as an iSCSI target (such as freeNAS or OpenFiler) for this to work.

 

[Santilli]

My pleasure.
http://www.stonefly.com/resources/videos.asp

The above link has the prior webinars, with sound, and works.

May not be great for you, however.

Which program did you use, or should I use, if I'm going to setup space on my server as a target for Iscsi?

Probably use the server as the target, running Windows 7 Ultimate. I could also perhaps use VMPlayer, and setup a target using XP, 2003, or 2000, or Ubunutu?

Don't know how that would work...