Quiet server

[Mercutio]

Here's what I'm dealing with:

One of my coworkers has developed an application that relies on MSSQL + IIS (and it's an asp .net thing, so it's going to stay on IIS) that targets a specific market, which is health care related.

The offices I've visited in attempting to sell this setup don't have dedicated data closets or even much soundproofing. And, because this stuff is medical in nature, the people we're selling to actually want this system in-house.

This system in part automates a lot of paper workflow into .pdfs. We're not sure about long-term storage requirements.

So I need a server system with decent-ish expansion that's also exceptionally quiet. I take this to mean that actual "server" parts are out. No redundant PSUs or 15krpm fans.

My starting point will probably be some kind of i5-based machine, 4GB RAM, 2x250GB drives for OS (SBS 2003 or 2008; I need MSSQL, IIS and Sharepoint to implement everything), and at least 2TB additional internal storage, preferably in a form that supports online expansion. I'll probably add one or two 2TB USB3 drives to cover backup needs. I'm also going to make sure each customer gets a nice, big UPS.

I'm looking at a Silverstone HTPC case and a 500W Seasonic PSU. The case is a bit of a sore spot. I could go with a generic rackmount enclosure but none of these customers have racks. These machines will at best be sitting on a shelf in a regular closet. Very possibly the one where employees are hanging up their coats.

Because I won't be able to use "real" server hardware, I'm planning to purchase an extra motherboard, two spare hard drives and a power supply to leave on-site for the customers; these people seem to be really interested in having full ownership of the systems.

My proof of concept machine is a 4GB Q6600, which is why I think 4GB is plenty of RAM and an i5 is likely enough CPU for the application. In all honesty, I could probably get away with an i3.

I'm still trying to decide if I want to build these systems on top of VMware. I think it makes some sense to do things that way, but I'm getting the impression that doing so is going to confuse anyone else involved in the maintenance of these systems. I'm not sure I'll be around here forever; setting it up now might make it easier to migrate that system later and it might confuse the hell out of another tech brought in to work on it.

 

[blakerwry]

I'll be happy to plug Dell here. They make some decent tower servers (or servers that can be either racked or towered). All of their servers for the last 5 or more years have had fans that spin down appropriately to match ambient/system temperature needs. I wouldn't consider these up to the standards of quiet PC's, but they are relatively quiet servers. My Dentist has one in his front office, for example.

 

[Mercutio]

I thought about just doing a Dell Twhatever. The extremely low-end Celeron "servers" are indeed fairly quiet but my proof of concept machine is, too, and it has a much more flexible configuration than Dell's.

 

[Mercutio]

... and getting a Poweredge T110 to a even kinda-similar space in terms of options pushes the price up to $3000 before tax and shipping. My Newegg shopping cart is $2300 including shipping and my spare parts.

 

[Handruin]

At least with the T110, you can leave support to Dell if you decide to move on. I remember reading some complaints of noise back when I was considering them for my own ESXi project. I know everything is relative to the individual, but you might want to investigate a bit more if noise is a concern. I also believe with the right CPU, ESXi is on the support matrix for the T110 if you wanted to go that route (though it doesn't sound like it based on your concerns).

 

[Bozo]

An Antec P183 or P192 wouldn't work?. They are quiet, and lots of room for expasnsion. You could even install a Supermicro 5 slot hotswappable unit in the 5.25" bays.

 

[Mercutio]

I don't like the interior layout of the P180-series cases. It turns the lower right-hand corner of the case into a total clusterfuck.

 

[time]

The Silverstone La Scala series is a nice choice. Apart from the good functional design, it looks a lot more upmarket than a P180.

Any reason you're not using SSDs for the boot drive?

 

[ddrueding]

Sounds like you have it under control. I would run it on top of ESXi, just to make backup and restoration easier. It makes remote admin and troubleshooting easier if that is a consideration.

 

[blakerwry]

... and getting a Poweredge T110 to a even kinda-similar space in terms of options pushes the price up to $3000 before tax and shipping. My Newegg shopping cart is $2300 including shipping and my spare parts.

I'm not seeing that stark of a difference. I spec'd a T110 with an i5 (Xeon X3430), 4GB RAM, 2x 250GB OS for $900. Add your own 2TB hard drives through newegg and your hardware costs are < $1200. For comparison, my newegg cart with similar performing desktop class components is $700, add the spare parts you mentioned and you're at $1000. I would think your software costs for MS SQL and Windows Server should be similar through Dell or anyone else, and you're not required to purchase software from the same place, so I didn't include them in the comparison.

Now, a $200 margin is still enough to make me think twice, and in some situations I might lean one way over the other, but I do think it's a more fair comparison.

Also, don't discount the Dell outlet. I saw several servers (T110, T310) matching or exceeding your requirements for ~$650-$700. Standard Outlet warranty for servers is 1 year, upgrade to 3 year for ~$100. Again, for mass data storage you're going to get a better deal on 1-2TB drives through the normal retail channels. This all but eliminates the price difference between a server through Dell or desktop class components + spares. At this point you're just down to how well the components have been tested, support, etc.

I should also remind that when shopping Dell you can typically get a few hundred $$ off the price of a new server by contacting a sales rep. We typically see ~$200 off the list price, plus upgraded RAM/CPU over our initial request. However, you may not see as much discount on a low end machine.

 

[time]

Is Dell going to reinstall/repair software for you? No.

Are they going to drop everything to replace a critical part the same day? Even if their agent had it on hand - which they won't? No and no.

What is the point in selecting Dell, but then sticking in your own equipment that they won't touch with a barge pole?

And debating minor savings on the initial purchase price is really quite academic.

They say Apple has a reality distortion field, but time and again I see this mindless devotion to Dell without any rational basis whatsoever. Fact is, if you buy everything from them, stick to a standard configuration, and have top-level corporate support, they're good. Change any of that, and they're more trouble to deal with than any initial saving is worth.

Mercutio's approach is the only sensible one. If he was certain he was going to be around, he could cover the customer for parts and service himself. In his possible absence, he's providing a rapid response capability - just add generic expertise and stir.

 

[Mercutio]

blakerwry, I configured a Dell server with all Dell parts, since that's the only way they're going to fully guarantee and support it. Their pricing for high capacity disk drives is particularly insane and appears to be the main cause of pricing disparity.

Dell also isn't going to support the software setup, whatever it winds up being. Hardware issues are probably going to be pretty minor. Anybody can replace a power supply or mirrored drive pretty easily. Getting a different tech to come in an manage software is the tricky part and at that point I don't think it's going to matter that I go Dell or not.

I'll go play around with their configurations some more but prices still look high to me.

Regarding SSDs, I just don't think it's going to matter for the application. These machines will probably be bottlenecked by the 100MBit switches they're connected to long before the I/O from 7200rpm drives becomes an issue. I'm also concerned about long-term reliability.

Regarding VMware: I've felt for a good long while that putting everything in a guest is probably the right way to build a modern server system, but in the small business environments where I work, nobody seems to be working that way. It could be a very smart decision to set my system up properly now, but I'm running the risk that someone else brought in to do work won't know what to do with it. I guess that's not really any worse than deploying a Linux server in 1999, but it still bothers me.

 

[time]

Are you sticking with 100% software RAID? Is that one of the reasons you split the HDDs, i.e. no special partitioning required?

 

[Handruin]

They say Apple has a reality distortion field, but time and again I see this mindless devotion to Dell without any rational basis whatsoever. Fact is, if you buy everything from them, stick to a standard configuration, and have top-level corporate support, they're good. Change any of that, and they're more trouble to deal with than any initial saving is worth.

Mercutio's approach is the only sensible one. If he was certain he was going to be around, he could cover the customer for parts and service himself. In his possible absence, he's providing a rapid response capability - just add generic expertise and stir.

Hardly devotion or distortion. My recommendation was based on Mercutio buying the system configured from Dell. I don't think that's an irrational basis if he feels he won't be around to support the system. I also think if you mix in a little bit of the info that blakerwry suggested, he might be able to bring the price down while talking with a sales rep for small business. I know this is possible because we just ordered another R710 and the web prices was ~$15K, but after going through our rep, the price was just about $10K. Granted he may not see that kind of savings, but there may be some.

I'm sure Mercutio can build a better machine at a similar cost, but what's the backup support plan in his absence?

 

[Mercutio]

Are you sticking with 100% software RAID? Is that one of the reasons you split the HDDs, i.e. no special partitioning required?

My current plan is to use software for a RAID1 for OS stuff and a controller-based RAID for the actual database. I was thinking about a Perc5 as a starting point since I have a spare one. I'd like to make an allowance for storage expansion.

Personally I think 2TB of storage is plenty for anything they're going to do in the lifetime of this system. They're storing .pdf files and Word documents, not video. The clients are worried about running out of disk space based on things they've read about past efforts to build paperless systems.

 

[LiamC]

Storage size:

If the people are using OpenOffice or MS Office 2007 +, then document storage won't be a problem If they are using Office 2003 or earlier, I've seen many 20MB+ single files. In this case, storage volume could be an issue.

Online extension. Why not Freenas or OpenFiler? Use rsync to back up the OS/data to another dedicated drive in the box or office? Schedule it to do so every 15 minutes. If the backup is to another machine in the office, then the only issues likely to affect data retention are theft of the box, or the place burns down or floods. Out of the box, I'm seeing 80~90MB/s read speeds and about half that for writes, so performance isn't lacking. Backing up to another machine in the office might be better than to a dedicated internal drive in the box or USB attached storage because it removes a point of failure if a power surge takes out the box--it's likely to take out your backup drive as well. Not saying it can't happen to the whole office, just trying to minimise the effect.

Why i5? Nothing I've read in the thread indicates gobs of horsepower is required. And I've been playing around with microATX boxes/AMD Athlon II 250's and with an Earthwatts 380, and a two drive config, I can get idle power under 50W. And you wouldn't need 4GB of RAM with Freenas/Openfiler. In fact, you'd be hard pressed to break 1 GB of actual RAM usage. If you go with AMD, then it's pretty easy to get boards with six or more SATA ports, so your expansion capabilities are taken care of--and for reliability you mentioned a spare board on site.

Why a 500W p/s? See above for idle power and that would put your Seasonic at under 10% load for probably 90% of it's uptime.

VMWare. An extra layer of complexity for what sounds like set-and-forget systems. Probably counter productive. How well is VMWare abstracting NICs these days? last I looked, the performance was poor, unless you had dedicated NICs per VM and extra for the physical host.

 

[time]

How well is VMWare abstracting NICs these days? last I looked, the performance was poor, unless you had dedicated NICs per VM and extra for the physical host.

Handy to know, thanks LiamC. Makes sense.

 

[time]

Speaking generally, does it make sense to use a dedicated RAID controller in this sort of case? My impression is that RAID 5 tends to suck on all 'affordable' controllers, and you now have the ludicrous situation where you can buy 6TB of storage for less than the cost of an entry level PERC card. Why not just use RAID 10 and be done with it?

Does anyone know if fake RAID 5 now offers comparable performance? Surely it's got to be a lower risk down the track in the event of a controller failure? No good for Linux, but Linux can do bootable soft RAID anyway - my understanding is that Windows still can't.

 

[LiamC]

Another + for RAID 10 over RAID 5. Sure, you get more usable disk from RAID 5, but if you've ever had to put up with accessing a RAID 5 share when a disk has failed, you'll know that the thing is practically unusable while the array rebuilds. Disk is cheap, multiple workers sitting around twiddling their thumbs isn't.

 

[Stereodude]

Not sure what controllers you've played with, but the better ones let you specify thresholds for utilization by different tasks and are still quite fast while rebuilding.

 

[time]

"... RAID 5 tends to suck on all affordable controllers ..."

 

[Handruin]

How well is VMWare abstracting NICs these days? last I looked, the performance was poor, unless you had dedicated NICs per VM and extra for the physical host.

Handy to know, thanks LiamC. Makes sense.

They abstract it very well. We consistently have a 5:1 or something 7:1 ratio per NIC. As long as you aren't using the default E1000 virtual adapter, you can get some decent performance using the VMXNET 3 provided you install the VMware tools. The days of having tons of NICs are in the past unless you have a specific application. I have not dedicated a NIC for the physical host in probably 3 or more years. The only component to get a dedicated NIC is for vmotion. Our layout has 4 NICs per blade. Two are teamed for public addressing, the other two are teamed for vmotion usage.

 

[Stereodude]

"... RAID 5 tends to suck on all affordable controllers ..."

Ok, what's your definition of affordable? A Dell PERC5i does what I described and are readily available used & working for ~$100 (or less).

 

[time]

$700 from Dell (looks like SAS only, can't find the SATA compatible, but you get the point).

That's more than 14TB of storage you could have bought by now.

 

[BingBangBop]

SAS is SATA compatible. With SAS you can use either SATA or SAS drives interchangeably.

 

[Mercutio]

Re: OpenFiler - My requirements are for one server. Space is a factor, because these offices weren't laid out to support any sort of data closet or area for any network infrastructure more serious than a cable modem and AP screwed to a wall. Because I need .ASP and Sharepoint, I'm forced to do Windows Server.

Re: VMware - I'd probably install an Intel PRO NIC anyway. In my experience, some motherboards' onboard NICs aren't supported by ESX in the first place. I'm going to evaluate how well that actually works for this application sometime next week. The idea of being able to divorce hardware from the software installation is very appealing and might become my common practice.

Re: CPUs. I don't think current commodity AMD motherboards are robust enough. CPU-wise, I'm thinking that socket 1156 is the current mainstream chip, whether it's an i3, an i5 or an i7, and because of that replacement parts should remain available somewhat longer than other things I could pick. I don't know how long these machines will be in use or what other purposes they might one day serve, so I'm willing to over-specify a bit in that department.

500W Seasonic PSU - Because I already have several of them sitting here. I know they're a quality, high efficiency part.

The storage situation - this is giving me the most cause for concern, actually. I'm thinking that having an expandable storage option would be a good idea, but of course current hardware RAID is going to have issues with future high-capacity storage devices. My initial thought was a mirrored 2TB drive and a further 2TB set aside to store backups. My sample set of documents is mostly PDF files ranging from 7MB up to 40MB, with some Word documents mixed in for good measure, but I don't know who those samples came from or how representative they are of the actual workload. The clients can't give me any kind of estimate on how much paper they're actually using right now, but it's easily dozens of pages per patient per year.

Anyway, I'm vacillating between just using the biggest drive I can in some kind of mirrored arrangement + 2 drives for alternating backups + another mirror set for OS + programs, or going to a hardware RAID where storage can be expanded, but only for drives that aren't advanced format types.

I have a spare Perc5. It's easy and cheap enough to obtain more of those. No, they're not ideal from a performance standpoint, but I still think my main bottleneck will be the network connection, not disk I/O.

The Dell salesdroid I talked to was friendly enough, but after talking to him for a while about my needs, he passed me off to a different part of Dell. The organizations I'm dealing with are Not for Profit, so the funding for these systems will be coming from Grants and the software licenses will be coming from TechSoup, and apparently the Small Business guys think that Not for Profit services are some kind of crazy moon language. So I have to talk to other Dell people. Bleh.

 

[Handruin]

$700 from Dell (looks like SAS only, can't find the SATA compatible, but you get the point).

That's more than 14TB of storage you could have bought by now.

There are a bunch here, and one for $80.

As BingBangBop mentioned, you can use either SAS or SATA. I'm using 8 SATA drives on my 6i in RAID 5.

 

[Bozo]

Handy to know, thanks LiamC. Makes sense.

The last time I install VMware (free edition) it didn't like IE7 or IE8. It worked but the GUI was kinda gooey.

 

[Stereodude]

As BingBangBop mentioned, you can use either SAS or SATA. I'm using 8 SATA drives on my 6i in RAID 5.

I've got 8 SATA drives on both of my PERC6i cards, but I'm using RAID-6.

 

[LiamC]

Socket 1156 is going to be supplanted by Socket 1155 in about 1 month (Jan 5), so I'm not sure how much longevity you'll get out 1156 chips/mobos.

Handy, thanks for the info on VMWare and NICs., but it's probably worth noting that this would apply to Intel (and I assume Broadcom) NIC's but not the on-board Marvell/Atheros/Realtek/Via stuff that seems prevalent

 

[Handruin]

Socket 1156 is going to be supplanted by Socket 1155 in about 1 month (Jan 5), so I'm not sure how much longevity you'll get out 1156 chips/mobos.

Handy, thanks for the info on VMWare and NICs., but it's probably worth noting that this would apply to Intel (and I assume Broadcom) NIC's but not the on-board Marvell/Atheros/Realtek/Via stuff that seems prevalent

Yes, both Intel and Broadcom are very common NICs to be included in our sever configs. Broadcom usually comes as default with many of the Dell and HP systems. The Dell T110 that Merc mentioned also has a Broadcom default option, but I know that using a VM solution isn't on his plate and likely not the Dell either.

I've never actually tried a VMWare ESX/ESXi product with a motherboard using any of the Marvell/Atheros/Realtek/Via like you mentioned. They could very well be marginally supported by the product and the workstation/server version which runs on top of an OS might work better. The only one I can think of that might be using his own homebrew solution would be ddrueding, but I suspect he may have an add-in Intel NIC in his ESXi setup(s).

Their product is also geared for higher end equipment (and NICs). They are pushing forward with 10Gb Ethernet and 40 Gb Infiniband solutions for things like fault tolerance and higher capacity vmotion. These things usually coincide with much larger budgets which can spend on the 10Gb and higher infrastructure.

 

[ddrueding]

The only one I can think of that might be using his own homebrew solution would be ddrueding, but I suspect he may have an add-in Intel NIC in his ESXi setup(s).

At the moment all of my production ESXi machines came with Intel or Broadcom NICs in them, but I'm in the process of building a smaller machine for a smaller company. It will run 3 2003-2008 servers for about a dozen people. Quiet is important. It will be in a P183 chassis and run a 3Ghz Quad-core CPU with about 12GB of RAM and a 180GB SSD backed by a pair of 2TB drives in RAID-1 for OS Backups and data. Haven't finished the spec yet.

 

[time]

Did this project go ahead, Mercutio?

 

[Mercutio]

Yes, but it was scaled back somewhat. What I ended up building was basically an i5 760 in a Silverstone chassis and a pair of 2TB Hitachi drives in RAID1.

It did turn out to be functionally silent with a Scythe HSF and a Seasonic PSU.