question Where'd the X79 motherboards go?

[Stereodude]

So you can get some killer deals on used V1 Xeon (SandyBridge E/EP) processors on eBay, but the X79 / 2011 motherboards are gone from the sales channels which leaves you with an C602, 604, or 606 server chipset, but those are also pretty rare and expensive too. They're generally not well suited for non-server use either. Like all the Supermicro boards don't have USB 3.0. Even used X79 motherboards are going for quite a bit of money on eBay... What's the deal? Are the low Xeon prices driving demand for the X79 / 2011 motherboards?

 

[Mercutio]

Short answer: Yes.
Even LGA1366 Xeon Systems can be objects of lust when you can get 8 or 12 threads for under $100.

 

[Stereodude]

This alternative sort of puts a cap on the price of a used X79 board, but it's more than I was hoping to spend by at least 2x.

 

[Mercutio]

There's something to be said for an unused board and you also have to remember that if you're buying a CPU of a certain age, it's going to cost a quarter what it did new and probably still have 80% of the per-core performance of current top-end stuff.

 

[Stereodude]

There's something to be said for an unused board and you also have to remember that if you're buying a CPU of a certain age, it's going to cost a quarter what it did new and probably still have 80% of the per-core performance of current top-end stuff.

After thinking about it more the EPC602D8A has something else going for it too. You can get 64gB (8x8gB) in registered ECC 1333MHz DIMMs for about $120 on eBay. 32gB (4x8gB) of ECC 1600MHz UDIMMs is over $200. Even skipping ECC and going with regular DDR3 32gB (4x8gB) of 1600MHz DIMMs is $110.

Used X79 boards go for about $200-225 on eBay. The savings on RAM pays basically for the extra cost of the motherboard and it's new.

 

[Stereodude]

There's something to be said for an unused board and you also have to remember that if you're buying a CPU of a certain age, it's going to cost a quarter what it did new and probably still have 80% of the per-core performance of current top-end stuff.

Between these two, which would you get for a workstation / non-server role? Supermicro X9SRA or ASRock Rack EPC602D8A They're $271.99 and $299.99 respectively.

On the Supermicro side:

• 10 SATA connectors and they're all on the Intel controller
• Uses standard 2011 HSF, not narrow (some reviews at NE say it's not a standard HSF mount, but I believe those are in error)
• SPDIF output for high quality stereo audio to receiver

On the ASRock Rack side:

• More PCIe slots
• Less picky about RAM
• Has BMC / IPMI

I'm leaning Supermicro because of the SATA situation and non-narrow cooler since I don't see putting more than a graphics card in it. Exactly how picky are Supermicro boards on RAM? Can you stray from the tested memory list at all?

 

[Mercutio]

I've been punished for not buying tested RAM for a Supermicro system in the past. You don't do that too many times before you just buy something on the HCL. I'll admit that getting the narrow heat sink is a hassle but I'd pay $20 extra for a fuss-free Asrock board, given than I already have my SAS/SATA needs under control.

 

[Stereodude]

I've been punished for not buying tested RAM for a Supermicro system in the past. You don't do that too many times before you just buy something on the HCL. I'll admit that getting the narrow heat sink is a hassle but I'd pay $20 extra for a fuss-free Asrock board, given than I already have my SAS/SATA needs under control.

How long ago was this / what generation of hardware? And how far off the HCL were you? The folks at servethehome don't seem to cite them as particularly picky. They reference other things as being picky.

 

[Mercutio]

Most recently with an LGA1150 board. As I recall, I tried Kingston and Crucial DDR3-1600 ECC UDIMMs and the damned thing wouldn't boot until I stuck in one particular Samsung module.

Edit: This bastard.

 

[Stereodude]

Most recently with an LGA1150 board. As I recall, I tried Kingston and Crucial DDR3-1600 ECC UDIMMs and the damned thing wouldn't boot until I stuck in one particular Samsung module.

Edit: This bastard.

Interesting... The guys at servethehome have assured me that I won't have any problems, and that it only gets dicey when you're using all the slots with max capacity sticks. I ordered the X9SRA and a bunch of other components from Newegg last night, so I guess I'm committed now. I haven't bought any RAM yet though. I'm trying to nail down a deal on the CPU I really want. Otherwise I will probably get a E5-2670 for $65 as a hold me over.

 

[Handruin]

Interesting... The guys at servethehome have assured me that I won't have any problems, and that it only gets dicey when you're using all the slots with max capacity sticks. I ordered the X9SRA and a bunch of other components from Newegg last night, so I guess I'm committed now. I haven't bought any RAM yet though. I'm trying to nail down a deal on the CPU I really want. Otherwise I will probably get a E5-2670 for $65 as a hold me over.

I was careful to only get the RAM listed on the HCL for my supermicro board because I read so many complaints from others about incompatibility. I am in the edge case of using the max size DIMM and max capacity for the board so this could be why it is picky.

Where are you getting a E5-2670 for $65? That's a decent CPU for the price. Which CPU are you trying to get?

 

[Stereodude]

Where are you getting a E5-2670 for $65? That's a decent CPU for the price. Which CPU are you trying to get?

They're all over eBay for under $75 with make offer options. The demand of people going crazy buying them has actually pushed the price of them up a bit on eBay. Natex.us has them for $65 shipped free.

 

[Handruin]

I've been wanting to build a dedicated home esxi server and a pair of those would do very nicely. I'll check it out, thanks for the info.

 

[Stereodude]

I've been wanting to build a dedicated home esxi server and a pair of those would do very nicely. I'll check it out, thanks for the info.

They also have used Intel dual processor motherboards: http://www.natex.us/Dual-Xeon-E5-2600-v2-socket-16-DIMM-Motherboard-LG-p/s2600cp.htm

 

[Mercutio]

Handy, you don't have access to a full-blown Vsphere setup for personal use? I know you switched jobs but for some reason that surprises me.

 

[Handruin]

Handy, you don't have access to a full-blown Vsphere setup for personal use? I know you switched jobs but for some reason that surprises me.

I know it seems strange that I don't have my own VMware environment and I think that's because I had such a large environment to setup and manage in my previous role that I never needed my own at home for anything. Just before I left the environment was upwards of 32 Cisco blades (2x Xeon & 256GB each) and a couple Cisco C260 rack mounts all managed under their UCS. That plus two emc VNX arrays and a bunch of Cisco Nexus SAN/networking components that I was far too spoiled with this environment to ever need anything at home to play/learn with.

Now I have two lame esxi servers (IBM M3 servers) to manage with no vcenter so it's a much different world. I have a couple personal projects I want to work on and need a bit more dedicated resources at home to learn and play. Right now the most I play with vmware at home is through their workstation product. My NAS hosts a single VM using workstation under Linux. I'd like to move that VM out of my NAS.

 

[CougTek]

Now I have two lame esxi servers (IBM M3 servers) to manage with no vcenter so it's a much different world.
Oh wow, what a third world setup! Even I have more at my offices (I manage two buildings, well, more like five, but only two have significant computer gears).

It must be a pretty small company.

Also, of all the servers I've worked with, IBM's are those I hate the most. Retarded interfaces and moronic configuration options. Most picky on RAM of all brands. At least they are stable when you finally figure out how to set them up.

 

[Stereodude]

I seem to have the reverse Midas touch today. All my attempts to buy parts for my LGA 2011 Xeon build via eBay today failed. I made several Best Offers and they straight up got declined, not even a counter offer.

 

[CougTek]

It might not occur to you, but to some people, being offered too low for an object is downright insulting. Some people deliberately ask more than what they think their object is worth, because they expect to be bargained. Others only ask what they believe is a fair price for their item and when someone makes them an offer that is significantly below what they want for their goods, they tend not to take it too well.

 

[Stereodude]

It might not occur to you, but to some people, being offered too low for an object is downright insulting. Some people deliberately ask more than what they think their object is worth, because they expect to be bargained. Others only ask what they believe is a fair price for their item and when someone makes them an offer that is significantly below what they want for their goods, they tend not to take it too well.

I offered the one guy 75% of his asking price. The other was similar.

 

[Handruin]

Now I have two lame esxi servers (IBM M3 servers) to manage with no vcenter so it's a much different world.

Oh wow, what a third world setup! Even I have more at my offices (I manage two buildings, well, more like five, but only two have significant computer gears).

It must be a pretty small company.

Also, of all the servers I've worked with, IBM's are those I hate the most. Retarded interfaces and moronic configuration options. Most picky on RAM of all brands. At least they are stable when you finally figure out how to set them up.

Just our part of the lab is a 3rd world setup. The team I'm on is working on the next-gen product so we have a strange collection of lab equipment. These two IBM servers are leftovers that don't even fit standard PCIe cards so I have two Emulex HBAs free-ballin' it out the back because I can't screw them down to anything. We have three racks of equipment for our team. The company is about 375 people total of which roughly 170 are in engineering. This is our on-site lab and we pay a facility to host important stuff in a dedicated lights-out location in a neighboring town. Our dev-ops/IT team is now two people (formerly 3). Most of us in engineering self-manage our own stuff.

I'm in agreement with your sentiments on IBM servers. We just got in two new Dell R730s that make the IBMs seem like something out of the early 2000s. We even just got (yesterday) two new Lenovo x3650 M5s and I'm not impressed with their onboard management (which is very similar to the IBM M3s). One of them is already complaining about PCIe errors and the only thing I've done is racked and stacked it.

 

[Stereodude]

8 sticks of Registered PC3-12800 ECC DDR3 DIMMs have been purchased on eBay. It's not on the Supermicro HCL, but uses Micron chips. This chip to be exact. Basically all the replies at STH said I should be fine venturing off the tested memory list if I stick to Samsung, SK.Hynix, and Micron based server memory. Avoiding Kingston was advised.

 

[Stereodude]

4 x 4TB Toshiba 7200 RPM drives ordered. I wanted a 5th (spare), but B&H wouldn't sell more than 4 at the discounted $115 price. I will order a 5th from them in a day or two presuming the price doesn't jump.

 

[Handruin]

Newegg does that also which bugs me. I'm waiting for their 6TB drives to go back on sale so I can order some more.

 

[Stereodude]

Newegg does that also which bugs me. I'm waiting for their 6TB drives to go back on sale so I can order some more.

If you want to have your mind blown see how promo code 0301RCKHSE88 applies to Toshiba X300 HDWE150XZSTA 5TB 7200 RPM 128MB Cache SATA 6.0Gb/s 3.5" Desktop Internal Hard Drive Retail Kit as you increase the quantity in your cart from 1 up to the limit of 5. The more drives you buy the less per drive the promo code takes off. :scratch:

1 - $10
2 - $18
3 - $27
4 - $32
5 - $40

I wonder what happens if you place 5 orders of 1.

 

[Stereodude]

Now I'm without a CPU for this build. My attempts to get the processor I wanted at the price I wanted to pay did not pan out. I guess my options are:

• Suck it up and pay for the E5-2689. The best price is currently $225.
• Get an E5-2670 for $65 and get about 91% of the performance I would have gotten from the E5-2689 and enjoy it.
• Get an E5-2670 as a stopgap while continuing to look for a deal on an E5-2689 and probably end up paying about $225, but having two CPUs.

Yes, I know option #2 is the correct choice, but it just isn't what I had my heart set on.

 

[Handruin]

Time is worth a bunch to me so if my heart was set on the E5-2689 I'd just buy it and move on with life. If you're trying to be budget conscious, just get the 2670 and wait for a lot longer for the 2689 or better to crop up down the road and upgrade.

 

[Stereodude]

Well, yesterday I sent a note to the seller of the E5-2689 for $225. They had several listings for them end at $225 without selling a single one. I politely suggested they might get more interest at a lower price like $150 (intentionally shooting low) given the low price of the E5-2670. I didn't get a reply so I decided I would just go for an E5-2670 and get something like a E5-2667 v2 down the road. So this morning I put in an offer for a E5-2670 and the seller has left the offer in limbo, neither accepting or declining. Since then, the E5-2689 was lowered to $200. If I buy a E5-2689 now I risk the other seller still accepting my offer on the E5-2670, which wasn't low ball, and I'll end up with a 2nd CPU I don't need. So I wait... I don't think there's any risk in the E5-2689 selling out at $200. They've got 10. I'd rather it was cheaper but, I'll probably still bite.

Personally, I think eBay should change the Best Offer feature to only give sellers 30 minutes to sit on an offer after they've logged into their account and been forced to see it. Other people say they just retract their offers, but there's no choice of "I found a better deal" and I'm trying to play by the rules.

 

[Stereodude]

So I decided to buy a E5-2670 after all since it's totally unclear how the magic of turbo boost will work in conjunction with the TDP limit of the chip. In short, the E5-2689 could run faster, but may not in my actual use. Both have the same TDP and the TDP rules the maximum clock speeds with an iron first. Only the base clock is guaranteed and they're both 2.6gHz. Someone on STH offered to loan me a E5-2689 to test with if he can find his spare. If it proves to be 10% faster under full load I can sell my E5-2670 at a small loss and buy a E5-2689. If not, I didn't pay 3x as much for the same performance.

 

[Mercutio]

Kind of an interesting use case for these massive systems: ESXi + PCI pass-through for gaming clients. I wasn't aware that ESXi exposed enough 3D functionality to guest OSes to make this feasible. Has anyone tried it?

 

[CougTek]

I've never tried, but I knew it could be done. I've been thinking of setting up a CAD server for our mechanical drawers for a while, at the office.

The Ubuntu+KVM guide is also interesting, albeit a lot more challenging.

 

[Handruin]

Kind of an interesting use case for these massive systems: ESXi + PCI pass-through for gaming clients. I wasn't aware that ESXi exposed enough 3D functionality to guest OSes to make this feasible. Has anyone tried it?

I've never tried it but my limited understanding from other's experience is that there are complications when using nvidia GPUs. I guess nvidia wanted to target customers to be using their more expensive Quadro cards for tasks like this. You may have a much better chance at success using an AMD card with this configuration. There may also be limited functionality using the VMware workstation software if you want to play around with it there.

 

[Mercutio]

I was reading claims that someone had gotten sharing to work well enough in Hyper-V to service multiple desktop clients for video decoding. Makes me wonder if something like a GTX Titan could be made to run multiple games off one box at the same time. I guess it's a decent enough reason to look at ATI hardware if I had an application for it. It might be a decent alternative to streaming systems for those of us with ridiculously high end home desktops.

 

[ddrueding]

I'm putting together an UnRAID system to run 3 clients and my NAS at home. Similar to this:

[video=youtube_share;LuJYMCbIbPk]https://youtu.be/LuJYMCbIbPk[/video]

 

[Stereodude]

I'm putting together an UnRAID system to run 3 clients and my NAS at home. Similar to this:

[video=youtube_share;LuJYMCbIbPk]https://youtu.be/LuJYMCbIbPk[/video]

Okay, that was pretty cool.

 

[ddrueding]

The build I have in mind is an older dual-Xeon with 128GB, 6x8TB, 4x512GB SSD, Intel 10GbE, some low-end Quadros, and maybe put a Titan on one of them. I'll start a thread on the project this afternoon.

 

[Stereodude]

He did say your HW can't be identical, but didn't elaborate exactly why. Or was that only for USB peripherals?

It really can't distinguish between two identical devices?

 

[ddrueding]

I believe it is so that you can easily identify the devices by description and sort them out. I'd bet that some trial and error would also suffice.

 

[ddrueding]

...and it was only for the USB devices. If you didn't see it, here is when he scaled it to 7 gaming machines.

[video=youtube_share;LXOaCkbt4lI]https://youtu.be/LXOaCkbt4lI[/video]

 

[Stereodude]

...and it was only for the USB devices. If you didn't see it, here is when he scaled it to 7 gaming machines.

[video=youtube_share;LXOaCkbt4lI]https://youtu.be/LXOaCkbt4lI[/video]

That's just silly. I like it... A LAN party without the LAN in a single box.