New Server Systems

[Mercutio]

Several of my customers are at a point where they need new server systems.
Well, OK, maybe not need, but the machines in question are five and six years old and I would like to move them on to new hardware.

None of them have particularly huge workloads. They might run a database system, in-house E-mail and typical file server sorts of things. They are a mix of RHEL or Windows 2000/2003 server, with everything between 1GB and 4GB RAM.

I'd like to buy several of the same thing so I have easy access to replacement hardware. Also so I can build and install them all during the vacation time I am being forced to take.

My initial thoughts are to specify 8GB ECC RAM, 64bit OS, some kind of hardware RAID, 4 or preferably 6 internal SATA hard disks and at least quad core CPU support if not a dual CPU configuration. I generally use a hardware RAID1 for the volume containing the OS.

I'm not married to any particular chassis; the existing servers in question are mostly in 3 or 4U enclosures. The existing servers mostly don't have power supplies that support the full range of crap modern server boards seem to need plugged in.

I'm trying to get these machines built for somewhere around $2000 without the disk drives.

 

[P5-133XL]

If you are dealing with servers with high I/O and reliability requirements then would it not be appropiate to be using SAS drives and controllers rather than consumer-oriented SATA drives?

Depending on how many servers and their uses for a specific client, I believe I would over specificy the MB so that the max amount of RAM is very high. This is so the customer can combine servers using VM technology either now or at some later date with minimal HW changes. The biggest issue is making sure that different servers use seperate disk drive arrays for performance reasons and case-size limitations. If you can combine servers using VM's than that can be a major cost savings over multiple physical machines.

 

[ddrueding]

I've had really good luck with SATA drives in server environments. IMHO, SAS is only needed in the top 10% of performance cases (and in those cases, just make the jump to SSD).

I've found that as quad-core CPUs became available, multi-socket motherboards (and the chips that run them) have become more difficult to justify. A Q6600 is really capable of a lot of processing for very cheap. I'm currently running 8 Tyan 1U barebones servers with Q6600s and 4-8GB of regular desktop RAM and 4x 1TB drives on the onboard soft-RAID. These things have been very good to me, and as soon as I installed a stable OS (ESXi) on them I haven't had to reboot any of them yet.

 

[Mercutio]

If you are dealing with servers with high I/O and reliability requirements then would it not be appropiate to be using SAS drives and controllers rather than consumer-oriented SATA drives?

No one I'm dealing with has the kind of needs that would justify that cost at the moment. Looking at the workloads on the machines presently in place, none of them at present are even using 2GB of RAM and all of them have fairly short disk queues. These are machines that are handling limited workloads for 10 or 15 people, not 50.

That said, I may end up with SAS anyway; Intel S5000-series boards are looking like an appealing overall choice.

I have deployed virtual machines for some of my customers, but these people are all in a class where I do not expect it will be necessary over the life of these systems.

 

[Mercutio]

Those Tyan units seem a bit limited for my needs. The price is right, but the features aren't there, particularly since that motherboard doesn't do hardware RAID or even registered ECC memory, and I was looking at 8GB as a minimum supported configuration, not maximum.

 

[ddrueding]

Sorry, I must have underestimated their computing needs. I couldn't imagine an office of 10-15 people putting a dent in even what I listed, considering just one of them is handling exchange, AD, and an accounting system for 300 in seperate VMs.

 

[Mercutio]

I am most concerned about RAM capacity, actually. 8GB seems limiting when building for five years in the future. I'm thinking 2xquad core CPUs won't be a limiting factor, and as long as I have at least SATA2 I'm probably good in terms of disk I/O, since that's what SSDs seem to be using.

The Tyan you recommended does have a slightly bigger brother which does support 32GB RAM. It's $800 for the baseline product, but that would take care of a lot of my concerns. Those things take 3.5" hard disks, right?

And are they a total PITA to open on the rack?

 

[ddrueding]

All the Tyan barebones I've seen use 3.5" SATA disks. Do you have a link for the 32GB variant? I might have to take a look at those...

They are a minor PITA to open on the rack. When they are slid out to the stops, the single screw at the rear is just obstructed by the item above it on the rack. The trick is to unscrew it before you slide it out, then it comes off easily and everything is accessible. The direct front-access drive bays are also incredibly useful, and remove most of the reasons for cracking the case in the first place.

 

[Mercutio]

Here's the datasheet and here's the Newegg..

I'm not sure about that machine's hardware RAID, and it uses 5000-series Xeons rather than the less expensive 3000-series, and it uses FBDIMMs, but the Intel server boards I was looking at were $600 apiece and needed a lot more work to put together.

 

[ddrueding]

Yeah, it just isn't cost effective to move to Xeons or FBDIMMs. The cost per Core/Mhz or GB of RAM goes off the charts once you go there.

 

[P5-133XL]

There are similar Supermicro 5100 servers as an alternative to Tyan.

And unless you have a much higher budget, I would stay away from any Xeons higher than 33xx and FB-RAM. However, dual q3350/q3360 using a 5100 chipset that uses standard DDR2/ECC DDR2 are relatively cost effective. I'd still go for onboard SAS controllers, if they are avail: You can always use SATA drives on them

 

[CougTek]

My problem with your plan of upgrading now is that both the LGA771 and LGA775 are going to be dying sockets soon. Intel is moving to LGA1366. The new Core i7 from Intel is excellent for server loads. I just don't find it in Xeon flavor just yet. If you need to build new servers now and you absolutely want server motherboards and CPU, then you can't opt for the Core i7 platform. However, you can assemble something quite interesting with the non-server components supporting Intel's new platform.

Case : Supermicro CSE-833T 760W triple-redundant PSU, 8 hot-swap 3.5" bays, 3U form factor. 644.99$

Processor : Intel Core i7 920 @ 2.67GHz, LGA1366 299.99$

Motherboard : Asus P6T Deluxe, 6x DDR3 sockets, 2x SAS ports 299.99$

RAM : Patriot 6GB kit (3x 2GB) DDR3 1333MHz 7-7-7-20, PVT36G1333LLK 239.99$

Grand total : 1484.96$ at Newegg

You can sleep knowing that your hardware will be supported for many years. You can climb to 12GB of RAM if you add a second 6GB kit for another 239.99$. I can't see a server for 15 folks needing more than 12GB of RAM anywhere within the next five years.

If you feel you must have ECC RAM, then the obvious choice would be the 5100 chipset from Intel. It is a lot more power-efficient than the 5000 and 5400 series. See this page for a comparison.

 

[Mercutio]

The Tyan setup plus one E5410 (4 x 2.33GHz) + 8GB worth of FBDIMMs comes to around $1500 as well. I could stick in four 500GB drives and still it my budgeted goal of $2000.

P5, the Supermicro machines you listed all seem to indicate compatibility only with 5x00-series Xeons. I'm sure the 3000s will work just fine in them, but I'm not to keen on voiding a system warranty or having support denied because the vendor is a dick about components, nor am I likely to remember a tiny little detail like the fact that I need to lie about which CPU I have installed four years from now when a system board dies.

Still, that first SuperMicro system is certainly a possibility.

Coug, it's absolute truth that you can't wait for the next newest thing to happen. There's always going to be something newer and better right around the corner. At the moment, I can't buy a Core i7 Xeon motherboard with all the spiffy features I want, like ECC RAM and decent hardware RAID.

I really am tempted to build using desktop parts though. I could buy an awful lot of computer with a Q6600 and desktop RAM, in a $150 rackmount case and a standard power supply. I might lose my hardware RAID, but the machine would be $1000 cheaper, easy.

 

[ddrueding]

I had a Compaq Proliant 7U server out here, and it was the most reliable thing I'd ever seen. Shut down twice in 8 years for relocation and never rebooted (windows 2000 and never on the internet). In fact, it was still running brilliantly the day I had it hauled away. We overpaid for reliability on that one (IIRC, it was a $15k server). If I had built something from desktop parts and had a spare ready, it still would have been half the price. That is my plan from here on.

 

[P5-133XL]

The Tyan setup plus one E5410 (4 x 2.33GHz) + 8GB worth of FBDIMMs comes to around $1500 as well. I could stick in four 500GB drives and still it my budgeted goal of $2000.

P5, the Supermicro machines you listed all seem to indicate compatibility only with 5x00-series Xeons. I'm sure the 3000s will work just fine in them, but I'm not to keen on voiding a system warranty or having support denied because the vendor is a dick about components, nor am I likely to remember a tiny little detail like the fact that I need to lie about which CPU I have installed four years from now when a system board dies.

Still, that first SuperMicro system is certainly a possibility.

The Tyan example you linked to was a dual-socket 771. I was just giving an alternative to your Tyan by supplying SuperMicro equivilents. The Intel series 3000 processors are Socket 775 and will only work in single socket applications with tend to max out at 8GB. So if you want 32GB dual processor, I think you will be stuck with series 5000 processors.

 

[P5-133XL]

You can use series 5000 Processors with Series 5100 chipset motherboards and end up using ECC DDR2 instead of FB-RAM. You just need to pick and choose: Some 5100 chipset motherboards use FB-RAM and some use ECC DDR2.

 

[Fushigi]

Just a slight bit of server overkill.

 

[Howell]

Whatever hardware you choose you should build the server in VM. Their next hardware upgrade will be extremely low labor and this one will take about ten minutes longer. XenServer Express is free for up to 2 CPU sockets.

 

[ddrueding]

I'll second Howell's VM recommendation, and point out that ESXi is also free for 2 sockets, though I know you don't like the UI.

 

[Howell]

oo, oo. This is where I get to say the the current version of XenServer has a text console that provides you a subset of the management functions. Otherwise you use the admin console or ssh into a terminal.

 

[time]

Whatever hardware you choose you should build the server in VM. Their next hardware upgrade will be extremely low labor and this one will take about ten minutes longer.

WTF? Their "next hardware upgrade" will certainly include an OS upgrade, as well as just about every piece of installed software. What does the VM buy them?

I'm hoping someone can educate me here - rather than a cost saving, I've only experienced poor performance and unnecessary complexity.

 

[Mercutio]

VMs are great for migrating low-complexity legacy systems on to new hardware (e.g. move the Novell based point of sale system onto a Linux Host). I've done that a bunch of times. I've never worked in an environment with high density clusters when you can seemlessly migrate guest images among your hardware, though that seems to be one of the areas where enterprise computing is head I agree that it's a good idea, since it moves to the idea of hardware independent computing and easy recovery from problems, but I'm not sure I'd want to try it in a single-server environment quite yet. I've not done anything with Xen or ESX, and my experience with Server 2008's Hypervisor didn't exactly give me any warm fuzzies, either.

Maybe I'll experiment with that in the next couple weeks, though.

 

[Mercutio]

VMs are great for migrating low-complexity legacy systems on to new hardware (e.g. move the Novell based point of sale system onto a Linux Host). I've done that a bunch of times. I've never worked in an environment with high density clusters when you can seemlessly migrate guest images among your hardware, though that seems to be one of the areas where enterprise computing is head I agree that it's a good idea, since it moves to the idea of hardware independent computing and easy recovery from problems, but I'm not sure I'd want to try it in a single-server environment quite yet. I've not done anything with Xen or ESX, and my experience with Server 2008's Hypervisor didn't exactly give me any warm fuzzies, either.

Maybe I'll experiment with that in the next couple weeks, though.

 

[ddrueding]

What I really like about VM is that it provides the fastest and most complete backup solution of all time. Just backup the entire VM nightly, and know that you can get their server up and running within minutes. Just install the player on a workstation and load it up; sure it will be slow, but they can deal for a day until the hardware issues are resolved.

 

[Mercutio]

I think I mentioned that bit already.

I don't have access to enough hardware for any one customer's environment to do testing of their applications to see how well they would work in a virtual environment, which makes me rather leery to deploy their servers that way. The LAST thing I want to hear is that someone just spent $4k on a new server system and it's slower than the old setup.

 

[blakerwry]

Have you compared a self made server to one of Dell's 2950 or 2900 servers?

I just spec'd out a 2900 for under $2000 - 2x E5205, 8GB ECC DDR2, 2x80GB SATA on HW SAS RAID, Dual PSU, Remote Access, 3yr warranty. If you call Dell, a sales person can usually start by giving you a 25% discount off of the website price.

I admit the E5205 may be underpowered, depending on your needs, but the rest looks inline with your requirements. But as you know, Xeon's are expensive.

I'm really impressed with Dell's remote access cards as well as their last couple generations of PERC SCSI/SAS cards (LSI megaraid). The remote access card provides remote KVM/Floppy/DVD access to the machine via IP and also provides hardware monitoring and alerts for hard drives, fans, CPUs, PSUs, case intrusion, etc and can be set to automatically reboot or fail the machine upon reaching certain thresholds.

One downside to the Dell is that it can be difficult to obtain spare hot swap hard drive caddy/carriers. So if you plan to purchase your own drives (rather than replacing existing), you'll want to see if you can find these on ebay or used locally. Also, be sure to add the rack rails if this is a rack mount server - they are not included by default.

 

[Handruin]

Just to put another good word for Dell 2950's, our lab and many of the systems I help maintain have been using them for a while now (couple years) and they've been very solid systems (I maintain about 7 to 10 2950's). Aside from a few hard drive failures (though they've been mirrored), we've had no other problems.