Centralizing home storage...thoughts?

[Handruin]

I'm looking to centralize the bulk of my storage for media such as pictures, video, music. I've been looking around a bit and I was wondering what your guys thoughts were about going towards a dedicated NAS device vs. putting together a system that will act as a file server.

I probably need 3-4 TB to hold everything comfortably with some room to grow over the next 2-3 years. I was thinking either way I'd probably want to have 4-6 bays to hold drives. Then I need a way to back it up. I have a Mozy account that I was thinking I could transfer if I built a file server versus buying a dedicated NAS device. My other desktop systems are wired for gigabit for when I need to move larger amounts of files around. For video and music, my laptop and xbox are connected with 802.11n.

What do most of you do to store the bulk of your data? Do you scatter it around multiple machines, or do you have it in a central location and access it over your network from a dedicated system?

 

[ddrueding]

I have my primary storage local on my primary system, just faster that way. I then have it backing up to my other system. With 2TB drives so cheap and easy, "normal" amounts of storage don't need dedicated machines anymore. Here is what I would do in your shoes:

1. 2x2TB drives in local machine in RAID-0
2. 2x2TB drives in some other machine in RAID-0
3. Backup nightly

This is likely more speed and redundancy then you have now or were planning to get. If you don't need to upgrade a chassis or Power Supply, it is half as much as a cheap 4TB NAS (that doesn't have redundancy).

 

[Handruin]

My current system has 4 drives in it now:
2x 250GB
1x 750GB
1x 300GB

I can't really add any more, so that means I'll end up replacing some of the 4 that are in there. I'd want a dedicated boot drive, so I might be able to take out the two 250 GBs and boot off the 750GB.

My other machine I have doesn't have enough SATA ports to add 2x 2TB drives in it like you're suggesting. It also isn't really worth expanding (it's my cheapo dell poweredge). This is why i was thinking a dedicated file server or NAS would make sense.

That NAS you linked to is pricey if it doesn't come with any drives. I was looking at the QNAS TS-639 as a dedicated device if I decided to go that path. It doesn't have any drives in it, but I can put in two and build from there.

 

[ddrueding]

That Buffalo actually does ship with the drives...not a bad deal, really. But not very fast.

Even upgrading your chassis or adding a simple SATA card is still the simpler/cheaper solution.

 

[CougTek]

The best low-to-mid-range NAS I know are made by Synology. Their latest two drives model has been reviewed by X-bit Labs and they liked it a lot. They also sell a 4 drives models (DS409+) that should do everything you want. The network speed is very good too.

 

[CougTek]

Newegg has it.

 

[Mercutio]

I use Centos systems with 6TB drives of drives in RAID5 that are regularly rsync'd to a second array in a different PC.

Works well.

I stopped at 6TB so I can have a hot spare handy.

I'm still using for file servers, though.

 

[Fushigi]

As discussed in another thread, I went the Windows Home Server route. Add drives as you want, no RAID to manage, the OS handles file duplication automatically. You can set up remote access to files on the server & RDP to your Windows clients from the 'net. Mac & Linux aren't directly supported for things like the automated backup but they can see it as a file server & share files/back up that way. Wikipedia has a decent write-up.

One nice thing about WHS are the add-ins that extend the functionality.

Pre-WHS, I did what dd said in post 2 - had the PCs back up to each other. It always felt like a kludge, but that could have been just the way I was trying to script it.

 

[Mercutio]

I don't look at "no RAID to manage" as anything like an advantage.

 

[Fushigi]

Why? You get data redundancy without having to manage anything. Drives with duplicated data can be pulled & taken off-site for backup or connected to other machines for file access. You don't have to buy identical drive types, speeds, capacities, etc. Doug could, for instance, buy a single 1.5 or 2TB drive for his PC and toss all of the other drives in a WHS and WHS would make sure files were duplicated across the array.

 

[Stereodude]

You guys must have a lot more "data" than I do.

I have a server in the basement with 2 RAID arrays. One is a 2TB RAID-5 array consisting of 5 500GB IDE drives on a 3ware 7506-8. The other is a 1TB RAID-1 array on the Intel ICH10R.

I have the really important stuff backed up onto two different external HDs (so I have two different backups). Less important stuff backed up to only one external HD, and stuff I don't care much about isn't backed up at all.

I have a 400GB, 500GB and 1.5TB external HD. The 400 and 500 are in USB to IDE enclosures and the 1.5TB is in a eSATA/USB enclosure.

Oh yeah, the server runs XPx86 and is manually backed up by me.

 

[Mercutio]

Why? You get data redundancy without having to manage anything.

I strongly prefer having CRC checks above and beyond the file system for data integrity. I like having reasonably expansive amounts of data (more than the size of a single disk) and more than anything else I don't mind doing the data management you're talking about it. If I'm shepherding terabytes of irreplaceable data, I would much rather store it to tolerate multiple faults rather than one or two.

 

[Handruin]

Something about the windows home server just doesn't appeal to me or make me feel safe with my data. If I'm going to pay for the WHS software, I'd rather just run a dedicated server with server 2003 so that I can actually install anything I want on the system. I should install it and give it a try before I judge it too much. Even with the WHS, I'd need another system to run it (or buy one of the pre-configured systems). It seems like there is no clear winner for managing data. Fushigi, is there a max to the size or number of drives WHS will allow? How is the performance?

I don't have lots of data like Mercutio, I just want to make it easier to manage in a single location. I'd be ok with the central location having 2-3TB and then backing it up to various external drives.

 

[MaxBurn]

Something about the windows home server just doesn't appeal to me or make me feel safe with my data. If I'm going to pay for the WHS software, I'd rather just run a dedicated server with server 2003 so that I can actually install anything I want on the system. I should install it and give it a try before I judge it too much. Even with the WHS, I'd need another system to run it (or buy one of the pre-configured systems). It seems like there is no clear winner for managing data.

That's pretty much my thoughts when I was considering a NAS device in years past. Now MSFT has a little fuller featured NAS device that requires full computer hardware to run on. Its the right idea but till it deploys on an embedded platform with a lower overall package price I don't see it gaining at lot of the market.

 

[Mercutio]

There's always FreeNAS.

 

[CougTek]

There's always FreeNAS

Tried it, works well. Simple to setup and to use. Unbeatable price.

 

[Handruin]

I could try something like that. I just need to piece together a cheap system. Do you think a dual core AMD AM2 chip would suffice? I have a working gigabyte GA-MA69GM-S2H motherboard kicking around. Judging by the CPU support list, I have a lot of options. It has on-board video and 4 SATA ports with 1x pcie 4x, and 1x pcie16x.

Motherboard gigabyte GA-MA69GM-S2H (currently have)

PSU Antec 480 Watt (not sure if this is enough)

CPU AMD X2 5600+ for $61

Memory 2x Corsair 2GB 240-Pin DDR2 SDRAM DDR2 800 $48

Case Antec 300 $59 (shipped)

I'm underwhelmed by the drives that are out right now. Seagate seems like the drive to avoid right now (yes/no?)? Samsung only offers a 1TB drive as its largest, so my only other option is WD with either their 1.5TB or 2TB drive, but they are costly. The Samsung price is also the best at $75 for their 1 TB.

Hard drives 2x 1TB Samsung. $150

Total: $318 + $8 S/H

For the sake of argument, I have access to a legit copy of server 2003, would you still pick freeNAS over server 2003 to manage your data?

 

[ddrueding]

I wouldn't, because I don't know the insides of FreeNAS or Linux. If I knew Linux half as well as I knew windows, I would use it.

 

[Fushigi]

I like having reasonably expansive amounts of data (more than the size of a single disk) and more than anything else I don't mind doing the data management you're talking about it.

WHS presents the storage space to the client machines as one aggregate; you are not limited to the capacity of a single drive for your share. I'd suggest that most folks wouldn't want to do the data management themselves. Even as an IT professional, at the end of the day I've had enough dealing with tech and just want things to work. A few years ago I would have had your attitude about it but now, after 20+ years doing IT for a living, I'm content to take the easy way out sometimes.

Doug - This screen shot from 2007 shows a WHS with 14+ drives. I only have 3 on mine - an internal 1.5TB, internal 500GB, and USB 1.5TB. This blog entry notes someone has a WHS with 27 drives. Someone also has a WHS with 22TB of storage. And I'm guessing they're the one with 19.3TB used. As to performance; I've never timed it but I'd guess the network connection would be the limiting factor.


Calling WHS a NAS is akin to calling a configured PC a browser. While it certainly is providing that service, it does plenty more. And, like a good browser, WHS is extendible. I wouldn't argue it fits all needs, but if you have a handful of PCs and you want an easy backup solution, it fits the bill with a minimal additional cost v. other solutions. The other features are the icing.

MaxBurn - Consider WHS running on an Atom-based platform. That should lower the power draw a fair bit. That said, there are power management extenders that could be used w/out running on a netbook CPU.

 

[Mercutio]

WHS presents the storage space to the client machines as one aggregate; you are not limited to the capacity of a single drive for your share. I'd suggest that most folks wouldn't want to do the data management themselves. Even as an IT professional, at the end of the day I've had enough dealing with tech and just want things to work.

Any way I look at it, I don't see a compelling reason to use WHS. Either I can buy some kind of NAS device (e.g. a Buffalo Terastation) that does the same thing for about what WHS + a PC + a bunch of hard disks cost, and simplify my administrative duties to maybe a half-dozen mouse clicks, or if I want full, extensible control over my data storage, I can choose one of any varieties of *nix flavors (e.g. FreeNAS) that offer full service storage options, not just the crippled options presented by Microsoft.

You're effectively arguing that the limited OS support provided in the WHS backup client is somehow enough of an added value to make it worth my while.

I just don't see it that way.

 

[CougTek]

I could try something like that. I just need to piece together a cheap system. Do you think a dual core AMD AM2 chip would suffice?

Are you serious? I've configured FreeNAS on a 1.7GHz Celeron and it was fine. You don't need processing power for a NAS. Most commercial solutions use embedded chips like AMD Geode and such (way slower than even an Atom).

 

[Santilli]

Splash had a removable drive setup for SATA he recommended. Since SATA storage drives are so flaky, I figure a removable drive setup is the best. Fill it up, pull it, until you need it.

I've been using Granite Digitals removable SATA drive setup, and, since the drives aren't up, aren't using power, it's a nice solution. With a multiple drive box, you could make two copies of what you want to store at intervals. Given the price tags on some of these solutions, I think removable drives in a system is the way to go.

 

[Handruin]

Are you serious? I've configured FreeNAS on a 1.7GHz Celeron and it was fine. You don't need processing power for a NAS. Most commercial solutions use embedded chips like AMD Geode and such (way slower than even an Atom).

I have an AMD (AM2) motherboard in my closet so I figured I'd use that. The cheapest AM2 CPU I can find is $30 USD. I get what you're saying about running on a celeron. I don't mind paying a few more dollars for an extra core to give it a little longer life span.

 

[ddrueding]

I've run a hundred times more IDE/SATA drives than I have SCSI drives, and had about the same percentage fail. Calling SATA "so flaky" would be an overstatement.

 

[Handruin]

I agree with that also, that I don't find SATA to be at all "flaky". I've been really happy with SATA since it has been out. I've moved completely over to it and haven't looked back.

 

[LunarMist]

SATA is fine. I think Greg has some issues with old computers or drivers. Compatibility problems are nothing new, but not a drive's fault. For example, the 1.5 and 2TB drives don't work in some systems or with some apps, yet work fine in others. There is much stupid, short-sighted planning.

 

[Fushigi]

Any way I look at it, I don't see a compelling reason to use WHS. Either I can buy some kind of NAS device (e.g. a Buffalo Terastation) that does the same thing for about what WHS + a PC + a bunch of hard disks cost, and simplify my administrative duties to maybe a half-dozen mouse clicks, or if I want full, extensible control over my data storage, I can choose one of any varieties of *nix flavors (e.g. FreeNAS) that offer full service storage options, not just the crippled options presented by Microsoft.

You're effectively arguing that the limited OS support provided in the WHS backup client is somehow enough of an added value to make it worth my while.

I just don't see it that way.

As I've been saying, WHS does far more than a traditional NAS. If those features have no value to you, fine, but you shouldn't dismiss their potential for others. Sure, some NAS units do come with add-ons that provide some of the WHS additional features, but those units aren't cheap. I'd also say those units are more niche products, so you can expect limited support & upgrades from the vendors. Being based on Windows Server technology, WHS gets updates regularly. MS has not only provided updates but has also added features over time.

The NAS units (I looked at Buffalo since you mentioned them) also seem limited in capacity, offering only a couple of USB ports for drives on top of the two or four internals; I've already linked to WHS installations with many more drives than that. So far WHS seems limited by your ability to physically attach drives. In fact, at 27 drives, the concept of the drive letter is officially busted.

And I've never made the statement that WHS is good in a mixed OS environment. I've said that for non-Windows clients it can act as a file server but you lose some of the other benefits. In fact I've specifically said it might not be a good fit for you or for others in some environments.

 

[Handruin]

Fushigi, have you run into any situations where the 10 user restriction has limited you with WHS?

I think I'm going to build the machine I listed above and I can try WHS on it to see if I like it before putting the machine into real use.

 

[Santilli]

Sorry my comment come out like that. I was talking about the current high failure rates in desktop drives, in particular Seagate, and in SATA. In a couple other threads, the discussions have focused on which of the current offering of large drives is likely to last, and, the Newegg customer reviews also point to fairly high failure rates. When I pick a current SATA drive, I look back a generation or two, and see which drives are still getting good comments, and, there are not many.

That said, I do have a Velociraptor in my HTPC, thanks to David, and a good price, and it's performance while not SCSI, is really good. Highest SDTR I've ever seen in any single drive, starting at a peak of 130 and averaging 105 mb/sec. 8.2 ms RAT, which, while not scsi, makes the system real snappy. I also use Seagates 7200 rpm SATA drives for backup.

SCSI is NOT in the running dollar wise, unless you are filthy rich, for a home back up, mb/dollar wise. Also, I imagine you are using an ethernet connection to the backup, and, that limits the speed considerably anyway.

 

[Handruin]

Also, I imagine you are using an ethernet connection to the backup, and, that limits the speed considerably anyway.

I'm hoping to build a system that can sustain 60-80MB/sec over my network. Having drives that perform at a decent level isn't a bad thing.

 

[Santilli]

I don't get that kind of speed copying from one SATA drive to another, in the same machine.

One of the other considerations might be your energy bill. Anyone know what each drive runs, power wise, or, adding another computer on line?

The HTPC software shows a 185 Watt draw, with the old monitor hooked up to it.

 

[Stereodude]

I don't get that kind of speed copying from one SATA drive to another, in the same machine.

I can get 100MB/sec copying between my two 1.5TB 7200.11 drives.

 

[ddrueding]

I get 197MB/s over SATA, from one drive to another.

 

[Santilli]

Check my math:
4.25 gigs in 55 seconds:
4250 divided by 55=77 mb/sec?

 

[ddrueding]

Check my math:
4.25 gigs in 55 seconds:
4250 divided by 55=77 mb/sec?

That's how I did mine, but I used a larger sample (to minimize human error).

40.1GB in 203 seconds = 197MB/s

 

[Mercutio]

It's going to depend on the size of the files being and how much the drive heads have to move around, too, at least in the case of Greg's data.

 

[ddrueding]

I used a single file, so it wouldn't have mattered too much, even if they hadn't been SSDs.

 

[Mercutio]

Well, right, but if Greg took a collection of jpegs from a folder somewhere, that would make a big difference in his calculation.

 

[ddrueding]

Well, right, but if Greg took a collection of jpegs from a folder somewhere, that would make a big difference in his calculation.

No doubt.

 

[Fushigi]

Fushigi, have you run into any situations where the 10 user restriction has limited you with WHS?

No, I don't have anywhere near 10 machines: 2 laptops, 4 desktops, and the WHS box. So I've plenty of room for the netbook I'll eventually pick up. And really, 2 of the desktops are in my Folding farm (which is idle at the moment) so they could easily be deleted. And one of the notebooks is my main work notebook; I haven't added it to the WHS environment as I won't store work-related info on my home machine.