Quiet server build

[ddrueding]

The demand for a decent ESXi server is there, but the server room isn't. This will be sitting in a room with 8 people who talk on the phone all day for a living. I've already done all I can with the acoustics of the space in general, but this thing will be within 6 feet of a couple people no matter where I stick it. This rules out any pre-made servers, as they are all designed to be in a concrete bunker. Normally in such a situation I'd stick it in the bathroom, but these guys and their cordless headsets keep talking in there as well, and the acoustics are hard enough to manage already.

I figure we need a pair of low-to-middling Xeons and 64GB of RAM (expandable to 128GB). There will be a pair of 4TB drives along with a half-dozen SSDs. A tower is preferable, and redundant PSUs are not necessary.

I really like the Corsair 900D and Corsair AX1200i PSU. But I am really interested on your opinions on the motherboard and CPU coolers in particular.

Thanks!

 

[time]

To me, none of this makes any sense.

1. Why would you deploy ESXi when you only have one server?
2. Why do you need dual Xeons to service 8 users?
3. Why do you need dual Xeons when you're running virtual servers to service 8 people?
4. Why do you need 6 SSDs in this day and age? Will you be using hardware RAID?
5. If you're not using low-profile rack-friendly CPU fans, why would their noise be relevant compared to 8 people talking?
6. Why the f... would you need a 1200W PSU? Yet you don't need redundancy?

 

[ddrueding]

1. Due to a large number of recent acquisitions, there are currently 10 servers. Until they are able to integrate the data and users from the other companies, this is how it will be.
2. The number of users is much higher, but the other users work from home and remote in to WinXP/7/8 VMs running on the ESXi machine.
3. See #2.
4. No RAID, just capacity and IO. I'll try to keep each server and its clients on their own drive.
5. I'm not using low-profile CPU fans specifically because their noise would be relevant. But compatibility of server motherboards with workstation cases and CPU coolers is not a guaranteed thing.
6. I don't need redundancy because I can't get redundancy in a quiet PSU. That 1200W PSU will not have the fan even turn on during normal load, and I have spares of that model handy.

As more of an explainer, this company is going through a bit of a growth phase; they are in the process of buying out a number of their competitors. When they do, the head of that organization gets a desk in the office and the rest of their employees work from home. Each of the employees' workstations is converted to a VM that they remote into, as is their server. I can't do a terminal server because some of these servers have different requirements for the workstation, and some of the servers are actually different versions of the same application where multiple versions of the client software cannot coexist on the same machine. I also have all these licences that I may as well keep using.

Does that help?

 

[CougTek]

I've setup both ESXi 5.1, 5.5 and Hyper-V servers (2012 R2). I prefer to manage the Hyper-V 2012 R2 servers...a lot. ESXi 5.5 is buggy. If you go the ESXi road and if you can get by with just 32MB of RAM, opt for version 5.1. However, I would strongly consider Hyper-V server if I was you. You can get the Hyper-V manager for free with any Windows 8 computer and the remote server management tools are freely available from M$' web site. All you need is the licenses for the Winders Server inside your VMs.

Regarding the hardware, the silent requirement means that you'll end up with a half-shit server, IMO. You'll have to use mainstream or enthousiast hardware that isn't server-grade. Don't expect iron-clad reliability with that.

About the power supply, two things :

1. I've heard from someone I know in a big computer store that the Corsair AX non-i, Seasonic-made, have much lower return rate than the Corsair AX-i, Flextronics-made models.
2. No matter if you have plenty of replacement units on hands, when a power supply dies, it doesn't always die alone. Don't forget that you're not dealing with server-grade hardware and you might have to replace more than just the PSU if it goes tits up.

Lastly,

I can't do a terminal server because some of these servers have different requirements for the workstation, and some of the servers are actually different versions of the same application where multiple versions of the client software cannot coexist on the same machine.

Different versions of a client software can usually exist on the same machine if they are in different virtual machines on the same system. What's harder to do in TS is working on multiple monitors, but if you don't have that issue, I fail to see why they can't just work on a remote server in a rack far from their desk. Backing up their data would also be simplified. Or maybe it's just me...

 

[ddrueding]

I've setup both ESXi 5.1, 5.5 and Hyper-V servers (2012 R2). I prefer to manage the Hyper-V 2012 R2 servers...a lot. ESXi 5.5 is buggy. If you go the ESXi road and if you can get by with just 32MB of RAM, opt for version 5.1. However, I would strongly consider Hyper-V server if I was you. You can get the Hyper-V manager for free with any Windows 8 computer and the remote server management tools are freely available from M$' web site. All you need is the licenses for the Winders Server inside your VMs.

I've been meaning to play with Hyper-V for quite some time. You and Merc have both said nice things about it, and I believe it could be better than ESXi. But I have no experience with it at this point and it is time to start the project. 32GB of RAM won't be enough; 48 might do it, but I'd prefer 64. The load on the systems will be low, but there will be a lot of them running all the time. I might even be able to get away with a single CPU, but the RAM is important.

Regarding the hardware, the silent requirement means that you'll end up with a half-shit server, IMO. You'll have to use mainstream or enthusiast hardware that isn't server-grade. Don't expect iron-clad reliability with that.

About the power supply, two things :

1. I've heard from someone I know in a big computer store that the Corsair AX non-i, Seasonic-made, have much lower return rate than the Corsair AX-i, Flextronics-made models.
2. No matter if you have plenty of replacement units on hands, when a power supply dies, it doesn't always die alone. Don't forget that you're not dealing with server-grade hardware and you might have to replace more than just the PSU if it goes tits up.

Hence the request for recommendations. A good server board that will fit in a normal (but large) tower case? Tower-style heatsinks that will fit said motherboard? Thanks for the tip on the PSUs, that is good to know.

Lastly,

Different versions of a client software can usually exist on the same machine if they are in different virtual machines on the same system. What's harder to do in TS is working on multiple monitors, but if you don't have that issue, I fail to see why they can't just work on a remote server in a rack far from their desk. Backing up their data would also be simplified. Or maybe it's just me...

The users whose workstations will be virtualized will be working from home, so no multi-monitor for them. Each user will only need access to one of the servers (the one that was part of their former company), so this makes it easy to transition. The workstation they are remoting into was their former desktop (documents, favorites, and all), and the server they are connecting to will be the same as well. This way we can absorb a company within a couple days (including forwarding phones, etc).

 

[ddrueding]

Looking at a pair of Intel Xeon E5-2420, and I'd love to use a Tyan S7045 but I'm not sure it would fit.

 

[Bozo]

The Tyan motherboard is EBB (12" x 13"). The case says E-ATX. Is that the same thing?
The one thing that jumped out at me was there are only 2-6GB SATA slots. You are using 6 SSDs. Are you using some type of add-in card?

 

[CougTek]

Looking at a pair of Intel Xeon E5-2420, and I'd love to use a Tyan S7045 but I'm not sure it would fit.

Don't buy any E5-24xx ; they are about to change generation in the next two months. At a similar price, go for the E5-2620v2. Faster and cooler. There are more LGA2011 boards on the market than LGA1356 anyway and some dual-socket are even standard ATX. Heatsinks are easier to find too.

 

[ddrueding]

Thanks for that. I knew I was getting it wrong and I knew you'd correct me. Much appreciated.

So the board for that would be a S7056 or an S7066?

 

[Handruin]

Would a noise-reducing rack unit be possible? Rather than make the server as confined and under-performing as possible in order to meet your noise envelope, what about making the rack to some of the work for you in reducing the noise? We used one for a conference/demo for a work-related event when we needed 3-4 2U rack servers. Something like one of these...but I don't remember the exact brand.

http://www.xrackpro.com/
http://www.rackmountsolutions.net/Soundproof_Server_Rack_Cabinet_Ucoustic_AcoustiQuiet.asp
http://www.acoustiproducts.com/en/index.asp

 

[ddrueding]

Thanks for those links, I didn't know such things existed. But I still don't think I need to make things that complicated. This computer does not need five nines reliability; one nine would be plenty. And I don't think the requirements are pushing any performance envelopes either; the only reason I'm looking at the two socket server boards are for all the DIMM slots and a bunch of SATA ports.

 

[timwhit]

Could the server go in a datacenter instead?

 

[CougTek]

Thanks for that. I knew I was getting it wrong and I knew you'd correct me. Much appreciated.

So the board for that would be a S7056 or an S7066?

One or the other. I would use the S7066 if it was me, just because it supports higher-end CPU in case of a future upgrade.

Since you will probably go for standard case and power supply, you could do worse than the Silverstone RV03. The orientation of the motherboard and case fans would be ideal (bottom-to-top) and that enclosure supports SSI-EBB and SSI-CEB motherboard, plus E-ATX ones, so all your bases are covered. I have that exact enclosure here in my basement. While it isn't the most quiet case I've owned, its airflow is remarkable and it would be a perfect match for your project.

 

[CougTek]

SuperBizz has it cheaper than Newegg. That store is also one of the best place to buy server-grade RAM too.

 

[ddrueding]

Could the server go in a datacenter instead?

They'd rather not; the local users would prefer the lowest latency they can get.

 

[ddrueding]

One or the other. I would use the S7066 if it was me, just because it supports higher-end CPU in case of a future upgrade.

Since you will probably go for standard case and power supply, you could do worse than the Silverstone RV03. The orientation of the motherboard and case fans would be ideal (bottom-to-top) and that enclosure supports SSI-EBB and SSI-CEB motherboard, plus E-ATX ones, so all your bases are covered. I have that exact enclosure here in my basement. While it isn't the most quiet case I've owned, its airflow is remarkable and it would be a perfect match for your project.

Thanks again Coug. Any recommendation on heatsinks? I'd love to put some tower-style units in there, but the RAM bays look pretty close...

 

[CougTek]

You don't have to be exotic on the heatsinks : these CPUs only have an 80W TDP. Anything compatible with the LGA2011 and with active cooling will do. Also, regarding the RAM, you'll buy server-grade RAM (YOU WILL!) so the sticks won't have tall RAM-heatsinks. They will fit below most tower style coolers.

I've noticed that the Tyan board has a Narrow ILM. You like to complicate your life. Fortunately, Noctua has a solution for you. Here's the link to buy the 120mm model from Newegg. The 92mm is available too. Please don't tell me it's too expensive. You're building a server.

 

[ddrueding]

Like most of my projects, price hasn't been mentioned until you brought it up. That isn't an issue.

When you say server-grade RAM, are you specifically saying registered? Or simply RAM on the HCL for the board?

 

[CougTek]

ECC, Registered AND on the HCL for the board. It means this one if you want 16GB sticks (it will run at 1600MHz with the E5-2620v2 because the CPU doesn't support 1866MHz RAM) and this one if you plan to use 8GB sticks. You need 4 sticks per processor (or 8) if you want to fully use the bandwidth.

 

[ddrueding]

Yes sir.

16GB sticks seem a no-brainer as they are significantly cheaper in $/GB. That is unusual.

 

[Mercutio]

How quiet is quiet?

My file server at home has a pair of Scythe Ninja HSFs (LGA1366 Xeons) on an Asus workstation board inside a giant Norco chassis. I'm using a standard Seasonic Gold-class power supply because there's simply no way to get a quiet rackmount PSU. I can tell that the machine is on and it's not silent, but it's really not any louder than a typical desktop and most of the sounds it makes are drive noise.

 

[ddrueding]

Definitely not aiming for silent. If the two HDDs are the loudest thing I will be happy.

 

[CougTek]

I've been meaning to play with Hyper-V for quite some time. You and Merc have both said nice things about it, and I believe it could be better than ESXi. But I have no experience with it at this point and it is time to start the project.

So, bought the parts yet? If not, what are you waiting for? It is time, you wrote it yourself.

 

[Mercutio]

There's a certain case to be made for inertia. If you already know how to work one thing and it does what you want, it's really hard to just move on from that. VMware works just fine and dd has already addressed the deficiencies it has (in his case by running it on a desktop rather than ESXi, IIRC), and moving to Hyper-V will create a new set of headaches and gotchas that he's probably too busy to deal with, especially for a system with some expectation of high availability. I'm using Hyper-V now, but not to the extent that I'm going to go around and replace existing ESX implementations.

 

[ddrueding]

So, bought the parts yet? If not, what are you waiting for? It is time, you wrote it yourself.

It is all ordered. I'll let you know how it goes during the build.

 

[CougTek]

One thing you'll want to be careful with, if using ESXi 5.5, is if you have the habit of creating your VM on your own workstation and then converting them to be sent on the ESXi server, be sure not to convert them in version 9 or 10, as the free VMWare client won't be able to modify them once on the server. Always convert to version 8.0.

You'll have to use ESXi 5.5 if you want to use more than 32GB of RAM.

I'm assuming you are using the "free" tools here. If you pay for the full VMWare VSphere licence and management client, don't pay attention to what I've just wrote.

 

[ddrueding]

I have a paid license, thanks

 

[Mercutio]

Something that might've escaped your attention, dd: Intel does make Eight-core Atom CPUs meant for low power servers. This Supermicro board includes a C2750 and has a TDP of 20W. The whole thing only gives about half the horsepower of an E3-series Xeon but if you're consolidating legacy servers that's probably pretty similar on a per-core basis to what the old CPUs were.

 

[ddrueding]

Something that might've escaped your attention, dd: Intel does make Eight-core Atom CPUs meant for low power servers. This Supermicro board includes a C2750 and has a TDP of 20W. The whole thing only gives about half the horsepower of an E3-series Xeon but if you're consolidating legacy servers that's probably pretty similar on a per-core basis to what the old CPUs were.

That had completely escaped me, thanks for the tip! The A1SRi-2758F looks pretty sweet!

 

[Mercutio]

d'oh! Forgot the link.

 

[Mercutio]

This AsRock board is the Atom C2000-series available on Newegg. I have no idea how it works as a server-class product, but it supports 64GB of desktop RAM and has 12(!) SATA ports on an ITX form factor. If nothing else that's pretty drool worthy for a storage server.

 

[ddrueding]

Not a problem, you gave enough info

Just ordered a Supermicro SYS-5018A-FTN4, 32GB of 1600 ECC LV, and 4 SSDs to play with....I'll just figure out how to fit in some larger, slower fans (probably at an angle, maybe with some homemade air guides of some kind...

This might be my Hyper-V learning rig if it supports all the right extensions.

 

[Handruin]

This AsRock board is the Atom C2000-series available on Newegg. I have no idea how it works as a server-class product, but it supports 64GB of desktop RAM and has 12(!) SATA ports on an ITX form factor. If nothing else that's pretty drool worthy for a storage server.

Holy shit. That would be awesome for a LAN/storage server. Server 2012 is listed as supported and it has room for an 8x PCIe. I can't tell from the description but does the CPU come with it for $393? I'm assuming not. I find it odd that is has a comm port on it especially when it has two dedicated LAN ports and a third specifically for IPMI.

 

[ddrueding]

...and cancelled the order. That ASRock looks like a great board, just need to get the right RAM for it. That would almost run on a PICO PSU with 4 SSDs and 64GB...

 

[sdbardwick]

Holy shit. That would be awesome for a LAN/storage server. Server 2012 is listed as supported and it has room for an 8x PCIe. I can't tell from the description but does the CPU come with it for $393? I'm assuming not. I find it odd that is has a comm port on it especially when it has two dedicated LAN ports and a third specifically for IPMI.

The C2750 is a BGA processor, so it comes soldered to the motherboard.

 

[ddrueding]

Just ordered:

ASRock C2750D4I Mini ITX Server Motherboard FCBGA1283 DDR3 1600/1333

4x Kingston 16GB 240-Pin DDR3 SDRAM ECC Registered DDR3 1600 (PC3 12800) Server Memory Model KVR16LR11D4/16

2x SAMSUNG 840 EVO MZ-7TE750BW 2.5" 750GB SATA III TLC Internal Solid State Drive (SSD)

Total: $2100

Once I settle on a PSU and a chassis (really like that Supermicro front-loading half-depth one) we'll be in business. And Hyper-V Server 2012 R2 just finished downloading

 

[Chewy509]

This might be my Hyper-V learning rig if it supports all the right extensions.

The Atom C2750 supports VT-x (as do all the C2xxx Atoms), so Hyper-V should work...

http://ark.intel.com/products/77987/Intel-Atom-Processor-C2750-4M-Cache-2_40-GHz

 

[Handruin]

Just ordered:

ASRock C2750D4I Mini ITX Server Motherboard FCBGA1283 DDR3 1600/1333

4x Kingston 16GB 240-Pin DDR3 SDRAM ECC Registered DDR3 1600 (PC3 12800) Server Memory Model KVR16LR11D4/16

2x SAMSUNG 840 EVO MZ-7TE750BW 2.5" 750GB SATA III TLC Internal Solid State Drive (SSD)

Total: $2100

Once I settle on a PSU and a chassis (really like that Supermicro front-loading half-depth one) we'll be in business. And Hyper-V Server 2012 R2 just finished downloading

Just in time. That board now shows out of stock at newegg! Let us know how it works out. I may be interested in updating my NAS with this.

 

[ddrueding]

Between Merc and I we may have cleared them out. I'm surprised that the board isn't on Amazon.

 

[Mercutio]

I'm all set for ridiculous file servers, but you might be interested in this chassis.
The damned thing has minimally twice the internal volume of a typical mITX enclosure but it supports nine 2.5" drives if you need 'em.