This will be my third attempt at building a system that will be used as a NAS in order to expand on my storage needs. My plan is to have capacity for my needs now and for what I hope will be the next several years. My previous NAS was planned to have a capacity that would last for 3-5 years but I guess once you have tons of space it's easy to fill it up. That system is just a little over two years old and it's at roughly 90% capacity.
First NAS.
8 x 1.5TB
(No longer functioning)
Second NAS.
12 x 4TB
(Still in service)
Hardware:
I decided to reach even further on this next NAS in hopes to make it last for a while.
Supermicro SC846E16-R1200B chassis, dual PWS-1K21P-1R PSU, 24x hot swap 3.5" drive carriers, 3x 80mm mid plane fans, 2x 80mm rear fans, BPN-SAS2-846EL SAS 2 backplane.
Supermicro X9DRi-LN4F+ dual socket LGA2011 motherboard
2x Intel Xeon E5-2670 (SR0H8)
2x Supermicro SNK-P0048P CPU Coolers
1x SC846 Air Shroud
192GB Memory ( 24x 8GB PC3-10600R DIMM)
1x Supermicro SAS2308 (pcie 3.0) SAS2 HBA (flashed to IT mode, fw 20.00.04.00)
1x Supermicro 3.5" Drive Tray and Silverstone 3.5" to 2.5" converter.
20 x 6TB HGST NAS 7200 RPM
4 x 1.5TB Samsung Ecogreen 5400 RPM (from non-functioning NAS build #1)
1 x Samsung SV843 2.5" enterprise 960GB
1 x Mellanox ConnectX 2 - MNPA19-XTR HP 10Gb ethernet (SFP+ DAC cable)
Total raw capacity will be 120TB + 6TB.
Total usable capacity after parity will be 96TB on the primary pool and 4.5TB on the scratch pool.
Configuration:
I'm currently using Ubuntu Server 16.04 headless as the main OS but depending on a few items I've yet to work out I may change this to Xubuntu 16.04 with a GUI.
I'm going to be using ZFS On Linux as the underlying managing filesystem.
I'll have Samba installed for sharing CIFS and NFS across the home network.
The boot drive is the Samsung SV843 that is partitioned roughly in half leaving the other half of the drive available for me to use as either an L2ARC for ZFS or possible as a SLOG. I need to run more performance tests to determine if this will be worth it or not.
I'll be creating two separate ZFS pools.
Main storage pool will consist of the following:
2 x RaidZ2 10-disk vdevs
1 x 16GB SLOG (via Samsgun SV843 partition)
1 x 400GB cache (L2ARC via SV843 partition)
Scratch pool will consist of the following:
1 x RaidZ1 4-disk vdev
I've done full-drive verifications of 18 out of the 20 drives so far and I'm just waiting on the last two drives to complete their 10-hour scan to make sure everything is healthy. The system is up and running with Ubuntu and functioning fine so far. I bought two of the Mellanox 10Gb ethernet cards to put one into this NAS and the other into my existing NAS to direct connect both systems. They haven't arrived yet but hopefully some time next week. I plan to backup the entire pool on the existing NAS temporarily so that I can do some maintenance on that system. When I attempted to rsync the pool, I'm estimating some 60+ hours over my current 1Gb network so I'm hoping I can squeeze a little more speed out of the 10Gb adapter.
I'll post up some pictures of the setup a little later along with some performance numbers. So far the noise of this system isn't bad. It's not quiet by any means but for a system living in my basement it's perfectly fine.
First NAS.
8 x 1.5TB
(No longer functioning)
Second NAS.
12 x 4TB
(Still in service)
Hardware:
I decided to reach even further on this next NAS in hopes to make it last for a while.
Supermicro SC846E16-R1200B chassis, dual PWS-1K21P-1R PSU, 24x hot swap 3.5" drive carriers, 3x 80mm mid plane fans, 2x 80mm rear fans, BPN-SAS2-846EL SAS 2 backplane.
Supermicro X9DRi-LN4F+ dual socket LGA2011 motherboard
2x Intel Xeon E5-2670 (SR0H8)
2x Supermicro SNK-P0048P CPU Coolers
1x SC846 Air Shroud
192GB Memory ( 24x 8GB PC3-10600R DIMM)
1x Supermicro SAS2308 (pcie 3.0) SAS2 HBA (flashed to IT mode, fw 20.00.04.00)
1x Supermicro 3.5" Drive Tray and Silverstone 3.5" to 2.5" converter.
20 x 6TB HGST NAS 7200 RPM
4 x 1.5TB Samsung Ecogreen 5400 RPM (from non-functioning NAS build #1)
1 x Samsung SV843 2.5" enterprise 960GB
1 x Mellanox ConnectX 2 - MNPA19-XTR HP 10Gb ethernet (SFP+ DAC cable)
Total raw capacity will be 120TB + 6TB.
Total usable capacity after parity will be 96TB on the primary pool and 4.5TB on the scratch pool.
Configuration:
I'm currently using Ubuntu Server 16.04 headless as the main OS but depending on a few items I've yet to work out I may change this to Xubuntu 16.04 with a GUI.
I'm going to be using ZFS On Linux as the underlying managing filesystem.
I'll have Samba installed for sharing CIFS and NFS across the home network.
The boot drive is the Samsung SV843 that is partitioned roughly in half leaving the other half of the drive available for me to use as either an L2ARC for ZFS or possible as a SLOG. I need to run more performance tests to determine if this will be worth it or not.
I'll be creating two separate ZFS pools.
Main storage pool will consist of the following:
2 x RaidZ2 10-disk vdevs
1 x 16GB SLOG (via Samsgun SV843 partition)
1 x 400GB cache (L2ARC via SV843 partition)
Scratch pool will consist of the following:
1 x RaidZ1 4-disk vdev
I've done full-drive verifications of 18 out of the 20 drives so far and I'm just waiting on the last two drives to complete their 10-hour scan to make sure everything is healthy. The system is up and running with Ubuntu and functioning fine so far. I bought two of the Mellanox 10Gb ethernet cards to put one into this NAS and the other into my existing NAS to direct connect both systems. They haven't arrived yet but hopefully some time next week. I plan to backup the entire pool on the existing NAS temporarily so that I can do some maintenance on that system. When I attempted to rsync the pool, I'm estimating some 60+ hours over my current 1Gb network so I'm hoping I can squeeze a little more speed out of the 10Gb adapter.
I'll post up some pictures of the setup a little later along with some performance numbers. So far the noise of this system isn't bad. It's not quiet by any means but for a system living in my basement it's perfectly fine.
Last edited:
