Remote disks...

[ddrueding]

Anyone seen performance comparisons between local storage and iSCSI for SSDs and the like? I'm evaluating moving the storage on my ESXi out of the server and into a SAN/NAS/whatever. The vendor (of course) says it is fine, but I'd like to see some numbers elsewhere.

 

[Pradeep]

Decide your IOPS requirements. Add room for growth. Make sure vendor agrees that their proposed solution will meet those reqs, in writing. And of course a SAN is not a replacement for a tested backup solution with offsite storage.

 

[ddrueding]

Decide your IOPS requirements. Add room for growth. Make sure vendor agrees that their proposed solution will meet those reqs, in writing. And of course a SAN is not a replacement for a tested backup solution with offsite storage.

Man, I don't know my IOPS requirements. I stick SSDs in the local machine, and it works great. I want to know what performance hit it will take moving the drives out of the box.

 

[Pradeep]

Moving to iscsi will limit you to gigabit speeds and below. Prob not an issue unless you are capturing uncompressed video etc. Also SSDs will reveal limitations in controllers that magnetic hdds have trouble hitting. Safe to say things will be "slower" external than direct attached, of course if that can be noticed by the users is the key question.

 

[ddrueding]

Moving to iscsi will limit you to gigabit speeds and below. Prob not an issue unless you are capturing uncompressed video etc. Also SSDs will reveal limitations in controllers that magnetic hdds have trouble hitting. Safe to say things will be "slower" external than direct attached, of course if that can be noticed by the users is the key question.

Thanks Pradeep. Any idea of the impact on access time?

 

[time]

SAN is always slower - that should be obvious. Anyone who says otherwise is not to be trusted.

Decent SAN controllers use big caches to try to ameliorate this. However, this only offsets latency in the disk subsystem, which would be very considerably less with SSDs anyway.

What Pradeep is pointing out, is that SANs have a very finite limit on the number of transactions per second that they can process, thanks to the fact that they're sending and receiving packets on a network. This is why vendors want you to focus on streaming performance instead.

Database servers typically fetch data in 8kB blocks (subject to OS lookahead). It doesn't take an awful lot to hit the SAN transaction limit if you have more than one database server (or even multiple processes on the one server) sharing the same LUN.

Of course, this assumes you have enough spindles to keep up. If you have SSDs, it's like multiplying your spindles (reduced latency), so you'd expect to hit that limit much more easily.

 

[Stereodude]

*cough* FC *cough*

 

[ddrueding]

What is the price tag on FiberChannel? And what happened to infiniband?

Time for more research.

 

[sechs]

FC?
I realise this is well out of my bailiwick, but wouldn't 10g ethernet be cheaper?

 

[time]

I found this, linked to by Sun that supports my position. It talks explicitly about ddrueding's proposal. The figures are theoretical calculations, so are best case.

 

[ddrueding]

I found this, linked to by Sun that supports my position. It talks explicitly about ddrueding's proposal. The figures are theoretical calculations, so are best case.

Nice article, time.

 

[Pradeep]

FC?
I realise this is well out of my bailiwick, but wouldn't 10g ethernet be cheaper?

It's still not possible to get the port density of FC in 10g ethernet. You might be looking at 5 10G ports per switch card, versus upto 48 ports and beyond in 4/8Gbit FC.

What's happening is that you can use 10g ethernet in your core network and director switches, and then have FC switches in the network cabinets closer to their associated racks.

 

[ddrueding]

In all my research I've come to the conclusion that I still want to keep the primary storage local, but I will be using iSCSI to mount my backup locations.

Thanks for all the advice!