question Where'd the X79 motherboards go?

[Stereodude]

I initiated a return for the other 3 DIMMs of RAM, and the seller accepted. I ordered four of the $16 8gB Hynix PC3-12800R sticks natex.us is selling. I guess I will be fan swapping tonight to see if I can find one that runs a little faster. I bought a small inexpensive DC/DC step-up regulator just in case I can't.

 

[Stereodude]

My other TY-147A fan ran even slower. Tuns out what I thought were 2 more TY-147A fans were actually TY-140 fans. Same basic fan with less PWM adjustable RPM range and a different bearing. So it spins about 2x as fast (~565RPM) at idle which is just fine. I popped it on, closed up the case and ran a Prime95 Small FFT stress test. After over 30 minutes the hottest core was still only 53C. I don't have my Nvidia GTX 650 in the case yet since it's still in another computer. Currently a passive Radeon HD5570 is in there. The 4 7200RPM HDDs in the case aren't powered yet either, so temps will go up some, but so far it runs cool and is nice and quiet.

 

[Handruin]

The eBay seller called me around noon today and we had a good chat. They realized they weren't getting back to me in time so they decided to call. I explained all the troubleshooting I had done and they agreed to send me out a power control module today via fedex. I already have a tracking number so it's possible I'll get it tomorrow. They were nice on the phone and sounded like the genuinely wanted to help solve the issue. I realize I went through quite a few degrees of troubleshooting to figure this out but I felt good during the phone call when he suggested it was the motherboard or the cables or the CPU version or the firmware/BIOS that I had an answer and that I had tested each condition. I'm hopeful this part solves my issue otherwise I'm out of ideas.

 

[Stereodude]

I just hope your mini cluster of dual processor E5-2670 systems don't become self aware and try to kill you. This could be the universe's way of trying to stop you before it's too late. :rofl:

 

[Handruin]

I hope it doesn't also but if it does it'll be news-worthy for sure! This has been an interesting and fun experience and I'm happy to have some extra capacity to mess around with some projects. Now I'm going to work on a vlan setup to isolate these systems from the rest of my home network since a few of them will be exposed to the internet. I've not really done anything like that and want to learn how.

I may explore Noctua fans for these servers. Even at low RPMs I can hear the grindy noise of these fans in my kitchen from the basement. Did you get your replacement RAM yet? I'm assuming not since their shipping is slow.

I still need to get some SSDs and now some UPSs to help protect this stuff. Down the road I will keep an eye out for the rack mounts for these cases and maybe put them all into a rack.

 

[Stereodude]

I may explore Noctua fans for these servers. Even at low RPMs I can hear the grindy noise of these fans in my kitchen from the basement. Did you get your replacement RAM yet? I'm assuming not since their shipping is slow.

I haven't put the dual processor E5-2670 into the basement yet. It has a clean install of Windows 10 on it now. I need to get AVIsynth and other relevant software on it so I can see how it works with what I built it for. I haven't noticed the noise from my new NAS (in the basement) on the first floor yet. Under just the right conditions I could just barely hear the fans in the PoE HP ProCurve switch (also in the basement) on the first floor. They have more of the grindy noise to them. But I'm talking middle of the night everything turned off not running if you listened really closely you could just faintly hear it. I don't really expect the Intel chassis when idling to be audible from the first floor, but I won't know until I try it. I'm a little more concerned about it while under load. I might end up fine turning the thermal clamps in the .sdr file to make it quiet / inaudible from the first floor. It's nice and cold down there though, so it shouldn't have to spin as much to hit the thermal targets as it did on the first floor.

Natex was out yesterday due to their blizzard. I'm not sure if they're in today or not. They posted something about not having power or internet this morning, so I don't expect they'll ship the RAM anytime soon. That RAM isn't for the S2600CP2J system though. The 8th stick for it should arrive on Saturday.

I got an upgrade of Windows 10 running on the single processor system with the Supermicro board last night. I will clean install it after I put the GTX 650 in it. That will probably be tonight. I need to stuff a graphics card in the system in comes out of though, since it's not quite ready for dismantling. I need to go through what's on it and move some data off it first. I've got one or two cards I can use though, like the HD5570, so it's not a big concern.

I also wouldn't mind a real rack myself. I don't know if they pop up on Craigslist or what. I haven't looked. I have a cheapo telecom rack from Monoprice in the basement hanging from the ceiling joist that I have my switches and patch panels in. It can't hold a PC though.

 

[ddrueding]

I've had a 2-post rack in the living room for years, but I think it is time to transition to a 4-poster. Due to cable length limitations it was important for the rack to be close to the desk. Now that USB 3.1 and HDMI 2.0 over fiber are a thing I can put the hardware further away. I may even stop watercooling everything once any noise isn't an issue.

And yes, I've even used thermal epoxy to attach generic waterblocks to the CPU inside my switch and firewall. The modem has a waterblock zip-tied to the side, and that seems to be enough.

 

[Stereodude]

It looks like natex did ship my RAM yesterday. There is a package in the hands of FedEx.

I decided to roll the dice and have one ready for pick-up at the closest BB with one. One is enough since I'm not looking to make another big RAID array with a bunch of drives... Just a large drive to put in my new E5 v1 Xeon desktop.

 

[Handruin]

Nice! What price point are those selling for at BB? Amazon has them for $249...that's an awesome price for 8TB He HGST drive assuming you're lucky to get one with that drive in it! I was spending close to that price on my 6TB drives.

I got my power control board from fedex this morning and it seems to have fixed my issue so I'm psyched. Fedex has really been on the ball with deliveries showing up before 9:00AM every time I get something delivered.

I ALMOST bought a Sandisk Optimus Eco 1.6 TB SSD from ebay last night for $545 brand new. Everywhere else I look that drive is $1400+! Unfortunately it was sold as of 9:00AM this morning. The seller has sold several of these at this price. He only seems to list them one at a time so I'm watching to see if another one shows up.

 

[Stereodude]

Nice! What price point are those selling for at BB? Amazon has them for $249...that's an awesome price for 8TB He HGST drive assuming you're lucky to get one with that drive in it! I was spending close to that price on my 6TB drives.

The same as Amazon. $249.99 + tax. I paid more for some of the 6TB Enterprise Toshibas I bought (they lowered the price partway through my buying). Of course the 6TB Toshibas do have a 5 year warranty. This effectively won't have one. Had I not built my big array already I'd be tempted to buy 8-10 of these and build a big array with them.

 

[Stereodude]

There are other reports of people getting a WD80EZZX in their 8TB My Book units. Which according to the information out there is a HGST He8 running at 5400RPM instead of 7200RPM. I haven't made it to Best Buy to pick up my drive yet. I'm stuck at work waiting for a DHL delivery. Either drive would be okay with me, but the 7200RPM HGST He8 would be a little cooler.

 

[Stereodude]

The WD My Book 8TB I got from BB does this over USB 3.0.



AFAIK, it has to be a 7200 RPM drive to bench like this. I'll open it up later after I "beat" on it a bit.

 

[LunarMist]

The WD My Book 8TB I got from BB does this over USB 3.0.

View attachment 1069 View attachment 1068

AFAIK, it has to be a 7200 RPM drive to bench like this. I'll open it up later after I "beat" on it a bit.

Isn't that shingles drive? Can you run a few hundred GB of sustained writes?

 

[Stereodude]

Isn't that shingles drive? Can you run a few hundred GB of sustained writes?

No, it's not a SMR drive. It should be a relabeled HGST Helium Ultrastar He8 inside the shell.

 

[LunarMist]

No, it's not a SMR drive. It should be a relabeled HGST Helium Ultrastar He8 inside the shell.

Which external drive is it?

 

[Stereodude]

Which external drive is it?

Scroll up a few posts and see if you can figure it out.

 

[Stereodude]

One last benchmark. My IOmeter pattern that mimics ATTO. It reads and writes for 330 second for each test which translates to about 66gB of data at 200MB/sec.

 

[Stereodude]

I opened it today. It has a HUH728080ALE601 in it just like the one PC Perspective bought.

 

[CougTek]

Pat Kennedy gave the Dude his two seconds of glory by mentioning him in his most recent article. Unfortunately for the STH editor, he wasn't lucky enough to get an UltraStar He8 when he opened his My Book 8TB.

 

[Stereodude]

The 8th stick for it should arrive on Saturday.

The 8th DIMM arrived today. But of course it doesn't work. The S2600CP2J flags it as bad and lights up an orange LED next to whatever slot it is in. The X9SRA won't boot with it.

*sigh*

Now I don't know what to do. I don't really want to wait another 2 weeks shipping it back and getting another (presuming the seller even had another). 7 matching DIMMs doesn't help me. I still have to ship the other 3 DIMMs back too.

ARGH!!!

 

[Handruin]

The 8th DIMM arrived today. But of course it doesn't work. The S2600CP2J flags it as bad and lights up an orange LED next to whatever slot it is in. The X9SRA won't boot with it.

*sigh*

Now I don't know what to do. I don't really want to wait another 2 weeks shipping it back and getting another (presuming the seller even had another). 7 matching DIMMs doesn't help me. I still have to ship the other 3 DIMMs back too.

ARGH!!!

That's some bad luck right there. Are these newer ones the ones from Natex?

 

[Stereodude]

That's some bad luck right there. Are these newer ones the ones from Natex?

The Natex ones (4 matching DIMMs) are supposed to arrive today. I guess I could get another 4 DIMMs from Natex and use those 8 matching DIMMs in the S2600CP2J and keep 4 of the 7 good VLP DIMMs for my X9SRA.

 

[Stereodude]

The seller is trying to refuse my refund for four DIMMs (1 bad and 3 good). He only wants to accept the one bad DIMM back. I pushed back, and asked why I should pay for three DIMMs I can't use because he's unable to fulfill my entire purchase. If he had listed only 7 I sure wouldn't have bought 7 (unless they were sold only in a single lot and were the same price as just 4).

 

[timwhit]

The seller is trying to refuse my refund for four DIMMs (1 bad and 3 good). He only wants to accept the one bad DIMM back. I pushed back, and asked why I should pay for three DIMMs I can't use because he's unable to fulfill my entire purchase. If he had listed only 7 I sure wouldn't have bought 7 (unless they were sold only in a single lot and were the same price as just 4).

I hope you saved a ton. I think I'll continue to shop on amazon. Microcenter if they have an amazing price.

 

[Stereodude]

I hope you saved a ton. I think I'll continue to shop on amazon. Microcenter if they have an amazing price.

I've bought a lot of stuff on eBay over the years, and this is the first time I've had this sort of problem. I have had two flaky RAM sellers who sent me one bad stick of RAM and then couldn't replace it with good RAM. The one is taking it all back. I'm still fighting with the other. New sticks cost $50-60. These supposedly working pulls were $15

Unsurprisingly, all four sticks I got from Natex yesterday work fine. They were $16 each. I would have bought all my RAM from them had they been selling PC3-12800R when I was first buying my parts. Initially they only had PC3-10600R.

 

[Stereodude]

I went back and remeasured under Linux. The power draw numbers were basically the same. No idea why it pulls more power in Windows. The CPUs seem to get hotter in Windows too.

So I'm pretty sure the explanation is AVX instructions. The Linux version of P95 was an older version that doesn't use them and the Windows version was a newer version that does. Intel CPU's increase their voltage when executing AVX instructions which increases heat and power consumption.

I've bought a lot of stuff on eBay over the years, and this is the first time I've had this sort of problem. I have had two flaky RAM sellers who sent me one bad stick of RAM and then couldn't replace it with good RAM. The one is taking it all back. I'm still fighting with the other. New sticks cost $50-60. These supposedly working pulls were $15

Unsurprisingly, all four sticks I got from Natex yesterday work fine. They were $16 each. I would have bought all my RAM from them had they been selling PC3-12800R when I was first buying my parts. Initially they only had PC3-10600R.

I got my refund from the one seller for all 4 sticks of RAM. The 2nd seller finally decided to accept a return of 4 of the 8 sticks. Those shipped out this morning. Four additional DIMMs from Natex should be delivered today so *fingers crossed* I will finally have 8 matched working DIMMs for the Intel dual E5 v1 board.

I've got 4 matched working DIMMs for the Supermicro single E5 v1 board left over from the RAM I returned. I tested the crap out of the RAM. I ran a full Normal pass with Goldmemory, which took ~32 hours, did ~24 hours of Memtest86+, and ~24 hours of Memtest86. No errors found. I guess I can finally finish loading software onto it and switch it into use.

 

[Stereodude]

Well, the RAM arrived. All 8 matching DIMMs from Natex are in my Intel S2600CP2J and are getting memory tested now. I'm not expecting any trouble.

 

[Handruin]

Hopefully everything goes well. It's been an adventure so far. I'm spinning up a windows VM with 8 cores to try some bluray conversions to get an idea on how long it takes compared to my desktop. Even if it's not as fast i can probably do several at the same time to get more done in parallel.

 

[Stereodude]

It took Goldmemory about 76hours and 20 minutes to do a "normal" memory test on 64gB of RAM in the dual processor S2600CP2J board. No errors detected.

 

[Stereodude]

I finally got around to testing x264 on both of my v1 e5 Xeon boxes today. I discovered that the Nvidia HW decoding can be invoked via RDP under Windows 10. I don't know if it's newer drivers, or something done in Windows versions after 7... But, it makes things a lot easier for me. So far it looks like the E5-2689 is about 35% faster than my i7-4770k @ 4.2gHz in the 2nd pass. It runs at about 3.05gHz during the 2nd pass courtesy of the Turbo and TDP. I'm not sure where the E5-2670 is going to fall in the 2nd pass. My guess is 3.0gHz since that's the max turbo setting with all cores loaded for the E5-2670.

I'm in the process of experimenting on pinning x264 encodes to certain NUMA nodes, processor affinity, restricting threads (to minimize visual degradation), etc on the dual processor box. I will post more later after I'm able to do more testing.

 

[Stereodude]

After finishing on this particular video The E5-2689 was 36.95% faster than my i7-4770k @ 4.2gHz.

On the other system (dual v1 E5-2670) I had no issues getting x264 to use all of both CPUs on a single encode. It spawns more threads than is advisable per the x264 devs for a 1080p videos, but... Power consumption is about 340W according to the BMC. My P3 Killawatt reads about 343W. CPU speed under load is 3.0gHz.

So the moral to the story is the E5-2689 was a waste of money for the 2nd pass. It was about 10% faster during the first pass staying at 3.3gHz.

 

[Stereodude]

I found my Intel S2600CP2J system in some goofy state tonight sitting at the BMC screen before the BIOS. I had left it running an encoding "benchmark" of sorts. My guess is that it attempted to sleep, but I'm not entirely sure. The logs don't have anything useful other than that the previous shutdown was unexpected. No .dmp file or anything related to a BSOD. I didn't have sleep disabled because when I was testing it before the sleep was greyed out / not an option. I've since changed it to never sleep. Hopefully that's the problem.

 

[Stereodude]

So the S2600CP2J system stayed on and running overnight. My "benchmark" finished running on the one CPU. It's 35.32% faster than my i7-4770k @ 4.2gHz. I'm running a few other permutations of the test now.

 

[Stereodude]

Grr... So I finally got around to setting up the dual E5-2670 system for some real x264 compression jobs, I get two going, one on each CPU and the system reboots somewhere into the jobs due to DRIVER_IRQL_NOT_LESS_OR_EQUAL. I analyzed the .dmp file and traced it back to iaStorA.sys, which is the Intel driver for the SATA controllers. Great...

 

[Stereodude]

So, I couldn't help myself and bought an E5-2687W v2 CPU for the single processor x79 motherboard I have. An eBay seller accepted my quasi-lowball'ish offer. It will replace the E5-2589 v1 CPU in that system. I'm sure they'll get cheaper down the road, but I can get some use out of it now.

 

[Stereodude]

So in my x264 benchmark test the E5-2687v2 CPU is just over 24% faster than the E5-2689(v1). The E5-2689 is only about 1.2% faster than the E5-2670. The E5-2687v2 is 70% faster than my i7-4770k overclocked to 4.2gHz.

I should note that the E5-2670 test was run on a single CPU of a dual CPU system through affinity and node restrictions, so it might be a inflated a very small amount since the OS and other system overhead can run on the other CPU giving the compression job slightly more CPU than it'd normally get in a single processor system. However, the clock speed difference under load only suggests there should be about a 1.67% delta, between the two, so that impact is not very large (if it exists).

 

[Stereodude]

I may have two E5-2690v2 CPUs in route to me. I unexpectedly won an eBay auction for a pair. I thought for sure my snipe was going to be too low to win. Turns out it wasn't. $960 shipped for the pair which is below my target of $500 per CPU. Of course the seller is overseas and I'm not entirely sure that he's not a scammer though the MO doesn't really match with the recent scamming I've seen. I guess I'll find out in the next few weeks.

 

[Stereodude]

I may have two E5-2690v2 CPUs in route to me. I unexpectedly won an eBay auction for a pair. I thought for sure my snipe was going to be too low to win. Turns out it wasn't. $960 shipped for the pair which is below my target of $500 per CPU. Of course the seller is overseas and I'm not entirely sure that he's not a scammer though the MO doesn't really match with the recent scamming I've seen. I guess I'll find out in the next few weeks.

It seems the seller was a scammer. Nothing ever came. The tracking number provided was bogus. The CPUs were listed as being in Montreal. The seller's location is Turkey, he said he was going to ship using "local Italy cargos", and said he shipped it using some Chinese shipping company I've never heard up. Anyhow, I got my money back from eBay/PayPal and bought myself a pair of the same CPUs for $800 on eBay (prices dropped quite a bit in ~6 weeks) from a US based seller. Those arrived today and the system now has two E5-2690v2 CPUs installed in it.

The US seller went a little overboard with the packaging. The CPUs were individually wrapped in anti-static bags from HDDs. The two antistatic bags were taped together, wrapped in bubble wrap, and put into a small cardboard box. That box was taped down into a larger cardboard box which was then placed into a USPS Priority Mail small flat rate box which was then put into a priority mail Tyvek envelope.

 

[Stereodude]

So in my x264 benchmark test the E5-2687v2 CPU is just over 24% faster than the E5-2689(v1). The E5-2689 is only about 1.2% faster than the E5-2670. The E5-2687v2 is 70% faster than my i7-4770k overclocked to 4.2gHz.

So the E5-2690v2 is even faster than the E5-2687Wv2. It's 10.3% faster when running the same x264 encode (when forced to the same number of x264 threads as the 8 core E5-2687Wv2). That makes it 38.6% faster than an E5-2670v1 and 87.6% faster than my i7-4770k @ 4.2gHz.

 

[Handruin]

That's a decent improvement in performance for x264 encoding, especially when comparing the 4770K.