Shingled drives and backup hardware

[Tea]

I've been looking at Seagate's new(ish) shingled drives for archival storage. Has anyone used them? Any thoughts?

My primary application is for off-line backup of large, seldom-changing datasets. Performance is almost immaterial for that purpose as the backup task can run in the background for days if need be, and if a restore is ever required, speed for that matters even less. As I understand it, SMR drives have good write speeds for small amounts of data (<20GB or so) and slow right down with larger writes because of the need to re-write the other tracks overlying the refreshed data track. (They must have very interesting firmware.)

Another application in the back of my mind is archival on-line storage - currently I use four standard 4TB Seagate PMR drives for this. This is mostly write once with occassional part-refreshes, and read infrequently, so SMR might work pretty well for that too. Swapping out the 4 x 4TB PMR for (say) 3 x 8TB SMR plus (maybe) a 4TB PMR for more frequently written stuff is attractive.

Tape backup, by the way, is insanely expensive per GB. It costs a lot more than hard drives do. Cloud backup, the same applies: not worth it for large data sets. In any case,
I don't like handing my data over to outsiders who can and do snoop - as we know in these post-Snowdon days. There is nothing sensitive on there, but screw them, they can't have it.

What is everyone else doing for backup?

 

[Buck]

Seagate, HGST, and WD have Archive drives for this very scenario. They're good for this purpose and the price. Recently I built a RAID 10 backup for a customer with 4 6TB WD Red drives, and they are working well for him.

 

[Tea]

I didn't know HGST and WD had similar products. Interesting. Thanks Buck, I'll look them up.

By the way, why does HGST still exist? I thought they were sold to WD (notebook product) and Toshiba (desktop drives).

 

[Handruin]

I've read a little about SMR drives but I've never owned or used one. I'm not a fan of the performance characteristics but I understand your use-case doesn't need a performing drive. I do care a bit about performance however because when you're talking about 8TB of of data that could live in a drive that size, the amount of time in the most ideal situations means you could read/restore from that drive and still not be done reading the data after 13 hours. Writing would take even longer with SMR in the equation.

I'm a fan of ZFS these days when it comes to managing large amounts of backup data. When you introduce SMR technology to ZFS there is the ever-growing performance problem of write amplification from these types of drives. One of the many features that may interest you is the 256-bit checksums for every block that will aid in you not having issues with bit-rot over time as you leave your drives archived somewhere on a shelf or spinning in a system. No matter how good your backup is, if you don't realize you have bit rot, you'll be backing that up also.

My approach for backups is a local NAS which runs the CrashPlan client. My local desktops also run CrashPlan and backup to my NAS. This allows for a rapid local backup when 60-70GB of photos come in from an event. The cloud-based storage can take some additional time to sync so I rely on the local backup to get me a second copy and then the third off-site will be ready in a day or two. I've mainly invested in 4TB HGST NAS drives right now. I'm running a 12 x 4TB with raidz2 (ZFS version of RAID 6). I no longer mess with fancy raid controllers all I want is numerous port counts. If/when 8TB HGST helium drives come down in price I'll consider migrating/investing in those.

 

[Mercutio]

The write performance on Seagate archive drives is horrific. They write just fine in bursts, up to maybe 10 minutes at a sustained high speed. But in single drive or RAID operation, with or without an SSD serving as a cache drive, they slow down or stop writing entirely for periods of up to a minute before resuming whatever sluggardly minimum rate they can manage. Usually my disk queue length while writing to one of them is pegged at 100. Microsoft's long-time advice for hunting for disk performance issues says a value of 2 is unacceptable on a server.

They're archive drives. I understand that we aren't supposed to care about performance, but in my experience it takes in excess of 30 hours to fill a 5TB Seagate drive regardless of the disk interface in question and only something less than eight hours to fill a 4TB 7200rpm Deskstar.

On the other hand, unlike some of the other slow drives, they do seem to behave in RAID arrays. And disk reads are fine. So other than patience on initial writing, they seem to be OK.

 

[Tea]

Interesting, Merc. For my primary application (archival backup) it already takes around 24 hours to fill a 4TB drive. (I haven't timed it as it really doesn't matter, I just start the job and leave it running, then unplug the drive when I next happen to think about it and don't mind a reboot (perhaps a day or two later) and take it away off-site for safety. For this purpose, it wouldn't matter if it took longer. A month would be pushing it but a week would be fine. I do brute-force incremental backups whenever I think of it (once few months is usually enough, or anytime after I make major changes to the dataset such as on return from a field trip but even with these there is seldom any urgency as the new data is already backed up onto the multiple external drives I take away on field trips and these don't get erased until I need the space for another trip. But the bulk of the data is static and changes from one year to another only insofar as I gradually organise it better. In other words, a year-old backup of (say) my 2009 work differs from the latest copy only insofar as I've done more sorting and grading.

No doubt there are much faster backup solutions around on the software side. As taught to me on Uncle Tannin's knee, I like very simple brute-force methods. The less there is to go wrong, the less goes wrong. I use Fastcopy (google it: a brilliant little free utility from Japan) and simply copy the entire folder structure as-is. That's faster than Xcopy or Windows drag and drop, but not vastly so. Doubtless there are much cleverer and faster dedicated backup packages which compress and de-dupe and so on, but I don't care for any form of backup where you need special tools of any kind to examine and/or restore the data: I just want another copy of the files. Simple is good!

Similarly the hardware: it's a generic cheap mainboard with a cheap little Pentium-G and 8GB of RAM. The only unusual thing is the 4 x 4TB data drives (and a 2TB boot drive - last of the wonderful old Samsungs). Currently I disconnect the DVD on the sixth and last SATA port and plug in a suitable backup drive, usually a 4TB Seagate but whatever I have handy.

I was interested to see your lack of response to my complaint about tape. A few years ago you would have rushed in to defend tape backup with examples and costing. I was sort-of hoping that you'd do the same this time and persuade me that tape would be better and/or cheaper. From my cursory glance at what's around and from your unexpected silence, I gather that tape just can't compete on a cost-per-GB basis anymore and hard drives are thus the only game in town.

BTW: pending further information, my present intention is to buy an 8GB Seagate SMR drive sometime over the next few weeks and fill it up with data. I'll post again with my impressions of it. Depending on how that goes, I'll consider swapping two of my 4TB data drives over to SMR as well. I'm pretty confident that SMR will be just fine for 3rd-level storage (i.e., periodic backups stored off-line), not so sure about the 2nd-level (primary archive, always on-line).

BTW (2) HGST and Seagate are following completely different strategies with SMR. Exactly as they have done with hybrid drives, Seagate are backing implementations which are invisible to the OS and just go faster or slower as the underlying technology determines, while HGST/Western Digital are producing units which are explicitly not standard drives and need different hardware and/or drivers. I can see good reasons behind both strategies. It will be interesting to see how it plays out in the market.

BTW (3) Doug, thanks for interesting remarks. ZFS (which I'd never heard of) sounds like an excellent idea, though possibly not for me as I'd need extra hardware to set it up and probably some skills I don't have. Something to keep in mind though, it certainly seems to have some unique advantages.

 

[Mercutio]

With regard to tape, I don't think it makes much sense unless you're at the point where your backup volume would require multiple hard disk drives under any circumstance. An LTO drive will cost probably $1000. Tapes are cheap and each one will hold 800GB or 1.5 or 2.5TB apiece, but you have to need enough tapes for the up-front cost on the drive to make sense. I suppose a decent rule of thumb is that if you need enough hard disks to hold your backup that $1000 sounds like a sensible amount of money to spend, you're better off with tape than with disks.

With regard to ZFS as implemented in FreeNAS: You can handle it. You do everything through a web interface. Installation is just imaging a file to a USB, CF or, hell, running off a DVD. The biggest reason I stepped away from FreeNAS for my own purposes was that my stuff was sitting on powerful hardware (to get the I/O support that I needed) that I couldn't really use for anything else. You can run ZFS on FreeBSD, OpenSolaris or (sort of) on Linux, but that's probably too much hassle for your needs.

 

[mubs]

There'd be other ramifications in the tape vs drive debate: handling requirements, space requirements, environment requirements, etc. Looks like for Tea's purposes, disk drives match her needs better.

 

[Mercutio]

Those also factor in to my decision not to advocate in this case.

 

[Buck]

By the way, why does HGST still exist? I thought they were sold to WD (notebook product) and Toshiba (desktop drives).

HGST manufacturers all of their storage devices separate from WD. They did not sell their desktop drives to Toshiba. WD did sell one manufacturing plant to Toshiba as part of the deal in order to satisfy regulators. HGST also continues to do their own R&D for products, keeping completely separate from WD (HGST has R&D everywhere). Even though HGST is owned by Western Digital (not WD), as part of the purchase deal with regulators, HGST remains a separate company (that's why we see HGST with 8TB drives, but not with WD). So far, what has happened is that Western Digital Corporation has become the parent company to HGST and to Western Digital Technologies (WD). Although they are separate companies, they obviously are a part of the same parent. So, we're beginning to see a path where the two siblings are manufacturing fewer products that compete. I do like the serious amount of R&D and products that HGST has in the Enterprise SSD market.

 

[Mercutio]

That's actually really good news.

 

[Buck]

That's actually really good news.

You can hold off with your fatwah on HGST a bit longer.

 

[DrunkenBastard]

Tea, Storagereview did a bit of analysis on one of the shingled drives: Summary was it can be used for backup but seems like you need to do a fair bit of workarounds to avoid their issue where extended sustained writes cause performance to go to complete shit:

http://www.storagereview.com/seagate_archive_hdd_review_8tb

 

[Mercutio]

You can hold off with your fatwah on HGST a bit longer.

Those who lie in congress with swine must be made to repent and suffer glorious and just wrath. Blessed be the platters and the magnets and even the tiniest bits, for it is not fault of data that it must reside in the inequity and despair of Irvine's Gehenna.

re: DB's helpful link. Those numbers are definitely in line with my observations. For the moment, I'm using 5TB archive drives in a tiered RAID6 configuration as second-line storage. I might add ~50GB of data a day, which is actually synchronized onto that array from smaller array of 4TB drives. Most of the time, I'd never notice the poor write performance, but was it SLOW the first time I copied data over to it.

 

[Tea]

Cheers Merc, that's a useful summary. Clearly I'm better off with drives. I only use 10-12TB per backup set at present, plus or minus a bit depending on how much of the unimportant stuff I do.

Just out of curiosity, are those 800GB - 2.5TB tape capacity numbers real TB or the phoney "with compression" numbers tape drive makers used to annoy everyone with last century?

Mr Grumpy used to have tape drives, BTW. He seems to remember things with a 400 in the capacity (must have been 400MB) which ran off floppy controllers, and one or two different things which fitted in 3.5 inch floppy drive slots with a single cable (was it IDE? Or proprietary? He can't recall) and most vividly some very fast (for the time) physically large (5.25 inch slot) SCSI-connected units which backed up the enormous $2000 Micropolis 1.7GB SCSI drive in the big OS/2 file server, back when he used to spend real money on computers instead of lenses.

Maybe I should find a spare box - there are bound to be a few lying around the shop, and I imagine that FreeNAS would run on just about any old thing - and do the ZFS thing with it. Hell, believe it or not, in the bottom cupboard right next to me here at home is Tannin's old Smoothwall box which, for some reason has been sitting there untouched since maybe 2004. Not that it would do, it's probably a 5x86 with a ....

 

[Tannin]

... sorry to interrupt dear. It was an interesting machine: Baby AT desktop case and PSU (haven't seen one of them for a while), IBM 6x86L-200, 72-pin RAM, and best of all, a hard drive I didn't have in my collection! Quantum Pioneer SG 2.1GB. Very cute looking thing with a deeply sculpted body picked out in black and silver. The Pioneers, by the way, were seldom seen. So far as I know they were a lower performance alternative to the Fireballs but by no means a cheap and nasty thing like the Bigfoot or the Fireball TM.

Anyway Tea, you can have the spot in the cupboard where it's been sitting all these years to put your ZFS machine, if you want it.

 

[Mercutio]

The 400MB tapes were Travan. They ran on a ftape (floppy tape) interface, Parallel or SCSI interfaces. In ancient days, I actually had a mounted /tmp filesystem on an Yggsdrasil Linux machine. It was just as slow as you would imagine. The other option was DC2120, which were 60, 80 or 120MB tapes IIRC. The WORM drives of the time were Syquest SQXYZ and Iomega Bernoulli disks with capacities of maybe 25MB at the time, but those things were science fiction exotic devices compared to tape. You could get tape drives in any big-box electronics store.

And yes, I'm talking about uncompressed capacities. LTO4 is 800GB, 5 is 1.5TB and 6 is 2.5TB. I've had an LTO4 jukebox for a while and I've not really had problems with them. I have a tape fail every once in a while but that's an $15 tape and not a $50 drive.

 

[Stereodude]

HGST manufacturers all of their storage devices separate from WD. They did not sell their desktop drives to Toshiba. WD did sell one manufacturing plant to Toshiba as part of the deal in order to satisfy regulators. HGST also continues to do their own R&D for products, keeping completely separate from WD (HGST has R&D everywhere). Even though HGST is owned by Western Digital (not WD), as part of the purchase deal with regulators, HGST remains a separate company (that's why we see HGST with 8TB drives, but not with WD). So far, what has happened is that Western Digital Corporation has become the parent company to HGST and to Western Digital Technologies (WD). Although they are separate companies, they obviously are a part of the same parent. So, we're beginning to see a path where the two siblings are manufacturing fewer products that compete. I do like the serious amount of R&D and products that HGST has in the Enterprise SSD market.

That is not entirely correct. The pre-merger HGST drives (designs and know how) and at least one manufacturing location went to Toshiba. What the new HGST drives are is anyone's guess. WD isn't going to run HGST separately too long (if they really even are now).

link 1 link 2

 

[LunarMist]

Shingles is terrible.

 

[Chewy509]

The 400MB tapes were Travan. They ran on a ftape (floppy tape) interface, Parallel or SCSI interfaces.

My favourites were the Seagate TR5's where the drive belt (literally a rubber band) would fall off during operation... (For those that haven't seen them, the drive belt was on the underside, and would slip off once they became slightly worn, which was approximately about 800hrs of operation). Cost of replacements from Seagate (at least here in Oz) was about AU$90 ea. We found replacements at a local electronics store (JayCar from memory) for about 50c ea...

The others I used to get to play with at the time included DDS (2/3/4), VXA1/2 and lot's of DLT (mainly SDLT drives from HP/Dell). Oh the memories...

 

[Mercutio]

That is not entirely correct. The pre-merger HGST drives (designs and know how) and at least one manufacturing location went to Toshiba. What the new HGST drives are is anyone's guess. WD isn't going to run HGST separately too long (if they really even are now).

Buck might have some additional insight to the current management structure because of reasons.

And it sure is nice to see some of you folks around. Tea, DB, Bartender, Clocker, mubs. You guys are missed when you're not about.

 

[mubs]

That is not entirely correct. The pre-merger HGST drives (designs and know how) and at least one manufacturing location went to Toshiba.

But Toshiba drives don't seem to fare well. In those drive surveys, or user reviews on various web sites. That is surprising o me; unless Toshiba mucked with the HGST's design / materials / process.

 

[Tea]

It's good to be back, Merc. Only wish that P5 was here too.

On a more cheerful note, we have used quite a few Toshiba drives over the last year or two. Well, quite a few by our current standards, which is to say only a handful. (Mr Grumpy has been semi-retired for a few years now and only does two days a week on average. He's generally quite mellow these days. I help out if I feel like it, or if he remembers to ask nicely.) No problems with them at all. The only downside is that they don't work with the very handy Seagate utilities we mostly use for drive cloning, though we try to avoid cloning drives as a rule and prefer to reinstall if it's possible. If worst comes to worst, you can always clone (say) a WD onto a Seagate first and then clone the Seagate onto the Toshiba you want to use. I have also used an open source cloner (details forgotten) but it was clunky. Probably it has improved since then, it was a while ago. Tannin mostly just orders drive without saying which brand. His favoured wholesale chap knows we take after Merc and don't like Western Digital much, so he just sends whichever of Seagate or Toshiba is more convenient to him, sometimes a mix of the two.

 

[Adcadet]

Sorry to jump in late. Just moved and will be tweaking, but my basic system involves backing up via CrashPlan to the cloud, backing up via CrashPlan to an NFS shared FreeNAS-hosted ZFS 8-drived mirrored stripe (~RAID 10). I also use Deja-dup to copy everything to another internal drive. I've kept a box of hard drives with complete backups at my office, and during the move I've kept one of these boxes at both sets of in-laws and another in my car. And during the move I also kept a full backup set via CrashPlan stored on a second FreeNAS machine running ZFS (stripped across two RAIDZ-1 pools with two hot spares) - this machine is now being reconfigured.

My new home includes a detached, second garage located about 100 feet away from the house. I'm debating if I should just sneakernet hard drives over to the garage for off-site-ish backup or run ethernet over to it and place one of my backup servers there and store bare hard drives in my work office located about 7 miles away.

 

[Tea]

Got an 8TB Seagate Archive drive today. It's at home running a backup job now. So far, it's performing exactly the same as any other drive. It will be interesting to see what it's doing when I get home - I'm sending about 5TB to it in the current backup set; I'll fill it up with other stuff (which also needs backing up) when that finishes.

 

[Tea]

It's been happily writing all day at a perfectly acceptable pace. I use Fastcopy for my backups and with 1.4TB written so far, the transfer rate has been sitting at about 26MB a second. This compares comfortably with my Thinkpad laptop drives (two physical drives, both Seagate hybrids) where a short test run gave 30MB/second. On the desktop system the SMR drive is backing up, I'm getting 85MB a second between two standard 4TB Seagate drives. So the trade-off seems to be, in exchange for the much lower cost per MB and convenience of having a larger capacity in a single unit, a write speed about one third of a standard drive.

I haven't bothered measuring reads, they are fine. It reads just like anything else. Seems a touch faster actually, which it may well be seeing as I'm comparing it to fairly old commodity drives with (typically) around 80-90% of capacity used.

At this stage, I'm thinking I'll get a second one (also for backup) and start selling off my 4TB drives or using them for some other purpose. Currently the system - an elderly Pentium G 630 with 8MB or RAM - has a 2TB boot drive and 4 x 4TB Seagate data drives. I've more-or-less decided to replace the four 4TB PMR drives with a pair of 8TB SMR drives and maybe use one of the leftover 4TB units to upgrade the boot drive - last of the wonderful old Samsungs but old enough now to think about replacing.

Summary: for backup use, clearly a good answer. For quasi-archival on-line storage - most writes are small, and the large ones are so large that they take hours anyway so a few more hours don't matter - it looks promising.

 

[Tannin]

Tea, you are repeating yourself. Just 'coz you wrote most of the thread doesn't mean you shouldn't read it before replying.

 

[Buck]

That is not entirely correct. The pre-merger HGST drives (designs and know how) and at least one manufacturing location went to Toshiba. What the new HGST drives are is anyone's guess. WD isn't going to run HGST separately too long (if they really even are now).

It was the Viviti manufacturing facility in Shenzhen, the Viviti Mars product lines, and a supply of Western Digital Malaysia heads that went to Toshiba. HGST and WD still do not manufacture together or do R&D together.

 

[Tea]

I checked out the WD archive drives. they have a 6TB unit available but it's hellishly expensive - about double the price per GB of the Seagate one. It's an enterprise-only unit, not a SATA drive. No doubt it's a lot faster and the "archive" part has to do with running cool and quiet and using less power than a standard drive. Anyway, not relevant to my needs, but interesting.

More interesting is the behaviour of my Seagate unit. It finished its backup set earlier today while I was at work. The 3.5TB set transferred at between 25 and 30MB a second. This was a pretty consistent rate right through the set of 14 individual queued backup jobs. (Fastcopy is smart: by default if you start multiple copies to a single physical destination drive, the second and subsequent instances of Fastcopy wait until the first instance has finished because that's faster than having them all run at the same time.) It was doubtless faster to begin with but I had other stuff to do and didn't get to monitor that.

The drive then sat powered on but idle for a while - at a guess, for some hours. Just now I started another backup set, and it zipped along at 135MB/sec for the first 400GB. It then slowed to 130MB/sec and I expected the rate to drop a long way at that point. Note that these transfer rates are total rates for the backup task averaged over the entire run so far, not instantaneous rates. So (for example) if the drive managed 100MB/sec for the first 100GB and then slowed to 10MB/sec for the second 100GB once the buffer was full, you'd expect to see the reported average rate start to drop at that 100GB mark and keep dropping down to about 55MB/sec at the end. For a 500MB copy under the same circumstances you'd wind up with 28MB/sec reported by the finish.

But now we get to the interesting bit. It's now been running this job for an hour and 40 minutes, it's copied almost 800MB and it is still averaging 130MB/sec! Figure that out if you can! 800MB and still going at pretty much full speed.

How can we explain this? It's going to the same drive. There are some background tasks in both cases but these are trivially light duty. The first set comprised 14 individual queued jobs ranging in size between 1GB and maybe 600GB, mostly in the 200-400ish range, with the source data coming from one or other of two bog-ordinary Seagate 4TB drives.(The data set is about 6TB but I filter the backup job to ignore the very low priority folders.) The individual files are all JPGs or Canon CR2 raws, so of moderate size, organised into folders nested only three or four levels deep as a rule. Fastcopy was using an ignore filter but I can't see that as being relevant.

This present backup task is coming from a third bog-ordinary Seagate 4TB unit and it consists of video files. It's now been running for 1:52 and has now dropped slightly to 129.6MB/sec - i.e., it's still scooting along at an excellent page - and had copied 885GB. What gives?

Could it be that the SMR drive likes large files? Perhaps there is less overhead involved with directory entries? Perhaps the firmware is optimised for big sequential writes?

Update: 2 hours 28, 1.15TB, still averaging 129.1MB/sec

 

[Tea]

More interesting observations. That previous copy ended up averaging 106MB/sec for 1.75TB. Fastcopy auto-started the next one and after a few tens of GB (when I happened to look at it) was back to a sluggish 35MB/sec. So I stopped the copy and let the drive idle for half an hour. On restarting, it sat around 110MB/sec. An hour or so later, it's done 340GB and it's still sitting on a respectable 104MB/sec. (Once again, these are video files, which are mostly quite large.)

Now the drive has 128MB of DRAM cache and, according to rumour, a 20GB magnetic buffer. Seagate don't disclose the size or the type, but it's either a dedicated PMR section for temporary storage or a dedicated SMR section where, through some magic, the drive can just write rather than do the complicated SMR read, write, write dance. The obvious way to do this would be to use SMR and simply skip some of the tracks, sacrificing a little capacity to gain a useful write buffer area to smooth things out until the firmware gets time to move it over to its permanent location.

You would imagine from that that you would get normal PMR-type performance for the first 20GB of a sustained write operation, plus a bit more (because the buffer would still be writing itself out to the SMR area while you add data to it). Assume that buffer flushing takes twice as long per GB as buffer filling. (It's probably more, but make that assumption to start with.) In that case, you should get ~130MB/sec for the first 30GB, then something like 30MB/sec from there on until the end of the write. But you don't!

Even more interesting is that the short half hour break more than pays for itself. With a normal drive, you simply fill the queue up with everything you want to send, and leave it to Windows (or your copy software) to keep the queue full for however long it takes the drive to write the data. That's the fastest way to do it. You can't improve the total write time in any way. You can only make it slower or (best case) make no difference. With this drive, a half hour idle after a big write more than tripled the write speed on resuming, and that speed difference remains in effect an hour later! So, if I had just continued to force-fed the drive with all the data it could take, it would have spent the last 90 minutes writing at 30MB/sec and have completed 160GB by now. Instead, I let it rest for 30 minutes, and despite only having had 60 minutes worth of write time since then, it's done 375MB!

It seems obvious that the drive is quite good at clearing its buffer so long as it isn't having to do other stuff at the same time but it can't cope very well with writing out the buffered data if it is also being bombarded with new write requests. Like Mr Grumpy, it's really only good for one job at a time, and it runs much better if it's allowed to have an afternoon nap or two.

The next experiment will be a different backup set, about 1.5TB in size, stored on the same source drive as the first video set this evening. This is mostly small files in moderately deep-nested folders. Will that make a difference? Tune in next week to find out. Same bat time, same bat channel.

Another thing to test is the optimum rest time. My half hour was just a guess. Would it have gone faster after a longer rest? Or would it be just as refreshed with a 20 minute break? I've searched the web (not very diligently) for information on this without result. Seagate don't tell you nuffin. I'll just have to experiment. (Not that it actually matters much for my application, I'm just curious.)

There is a useful article, by the way, here http://www.tomsitpro.com/articles/seagate-8tb-archive-hdd-review,2-822-2.html (Yes. Tom's of all places! I haven't been there for years.)

(1:24 now, 530GB, still sitting on 140MB/sec.)

 

[ddrueding]

I was thinking as I was reading your analysis that writing a proper test suite for drives like this would be a nightmare. I wonder if ZFS or other drive aggregation tools would be able to take this kind of thing into account in an automated fashion?

 

[Mercutio]

The "falls off a cliff" sustained write speeds are definitely in line with my experiences, but I don't see anyone writing large copy managers with built-in break times so a drive can catch its metaphorical breath, either.

 

[Tea]

Indeed a bugger of a thing to write benchmarks for, Dave. And as we see more SMR drives coming along, with different sized buffers, different firmware optimisations, and different mixes of on-disc and RAM and flash buffering, it will get trickier still.

Meanwhile, I now have solid evidence that the key factor is the average file size. It makes a massive difference. After several hours rest, I started that third backup, the one with smaller files in a somewhat more complex directory structure. After a mere 56GB, this is crawling along at 23.6MB/sec. Average file size is about 600kB. Compare with the previous job (mostly video files, average size 16MB with many much larger ones in the 1GB range) which finished up after 3 hours 25 with a total of 890GB at 72MB/sec average. I was keeping an eye on it until I went to bed at around 80% complete and it hadn't budged from a little better than 100MB/sec until then. I speculate that the few assorted folders of little files in that data set were responsible for the late slowdown. (Fastcopy operates in normal alphabetical order, and as it happens those small-file folders come late in that, so it fits.) (It also provides good evidence that Tannin is a lousy housekeeper. What is all that random stuff doing filed under the video folder anyway?)

How much difference to transfer rate does average file size make with standard PMR drives, if any? (I probably should know this, though I can't see how it would make any practical difference insofar as until now you were pretty much stuck with either PMR or flash, so it has been one of those questions with no practical import Tannin always tells me to ignore 'coz they don't pay any bills.) A quick web search yields nothing useful. Let's find out for ourselves then.

Going PMR to PMR, 30GB of that small-file folder gives me 57MB/sec. The first 30GB of video folder (the one that copied at 130B/sec to the SMR drive yesterday) gives me 62MB/sec. (Not a bad pair of tests, these two, as they happen to be coming off the same Seagate 4TB drive and going to the same Toshiba 2TB unit. Both drives are about 90% full, BTW.)

So we are seeing a difference according to file size, but it's small on PMR. There is a huge difference with the SMR drive, however. (It did occur to me that it might have to do with cluster sizes, but no, all drives are using the same NTFS default 4k clusters.) Earlier, I stopped the third backup at the 56GB mark. Now, on restarting it, with the SMR drive having been idle for a half hour or so, I immediately get the same low transfer rate. It's bouncing around a bit but in the 30-35MB/sec range.

Short answer: the Seagate Archive drives hates writing small files: it's only half as fast as a bog-ordinary standard PMR drive. On the other hand, it writes large files almost twice as fast as a standard drive does. Given the very high areal density, this makes sense. We should also note that this is a brand new drive, so all of these writes are first-time writes. The drive is now 80% full. What happens when we start doing overwrites? Will it slow down hugely? Or does the firmware move data around during idle time so that, after a while, your free space becomes contiguous, ready for faster writing?

 

[Tea]

I don't see anyone writing large copy managers with built-in break times so a drive can catch its metaphorical breath, either.

I do see that, actually Merc. Well, in a sense, anyway.

At the primitive level, I could easily write a batch file to do just this. In practice I won't because I don't especially care whether a backup job takes two days or six days just so long as it gets done.

But at a more sophisticated level, it occurs to me that this is exactly the sort of thing you'd be doing if you were a cloud storage provider looking to optimise your performance at the lowest possible cost per GB. SMR drives are the cheapest per GB, so you want to use them for your seldom-accessed bulk storage. On the other hand, you want to get the best write performance possible out of them because that lets you save money on expensive faster flash and PMR drives. I shouldn't think you'd pause the write process, you'd just distribute it between different drives. In the simplest example, you'd have two drives and write to each one alternately while the other one rests. Similar logic applies to manufacturers of multi-drive storage boxes (NAS and the like), if they thought the expense of the new firmware was worth the saving on drive space.

Of course, this unit is an OS-transparent SMR drive suitable for plug-and-go use, where you would think that the big users (server farms, NAS providers) would do better with one which exposes its inner workings to the OS and which, instead of having on-drive buffers and smart firmware, expects the host system to understand it via suitable drivers and (probably) modified write optimisations. At present it seems that Seagate are making the former type and WD the latter. You'd imagine that both firms will cross over and do both types as more models reach production, so as not to miss out on half the market.

 

[LunarMist]

How can one determine if the drive has the shingles? I need many long reads and writes (hundreds of GB to TBs), so any funky slowdowns are not acceptable.

 

[Tea]

All mainstream drives use PMR, Lunar. SMR is new and only a handful of special-purpose drives use it. WD (and possibly HGST and/or Toshiba too) make SMR drives for the enterprise. These can't be plugged into normal systems, certainly not without proprietary drivers to manage the complex write requirements. Seagate make a line of SMR drives which are sold and badged as "Archive Drives". See http://www.seagate.com/au/en/products/enterprise-servers-storage/nearline-storage/archive-hdd/ The Seagate philosophy is to put all of the necessary smarts into the drive itself. So far as your operating system is concerned, it's just a very large standard hard drive.

Actually, these would be perfect for at least some of your needs. You store a vast amount of data, and in particular you have to back it up. An SMR drive is perfect for that task as it is much cheaper than equivalent storage in PMR drives would be. Read performance is excellent - faster than my PMR drives. Write performance is perfectly sufficient for backup purposes - or this has certainly been my experience so far. Early days yet but indications are good.

For small writes (say 10-20GB) write performance is similar to any other drive.

For large writes (500GB or several TB) write performance varies with the average file size.

For large files (such as video) write performance is as good as or better than a standard drive. For example, last night I copied 800GB of video at an average speed of about 100MB/sec. This is faster than copying that same data to a standard 4TB PMR drive. (Probably an expensive high-speed PMR drive would do better, matching or perhaps beating the SMR drive - but who can afford to buy terabytes of high-end hard drives for bulk storage?)

For small average file sizes, the speed drops to about one-third of that provided by a PMR drive.

For small to moderate file sizes, the write speed is better, about half of a PMR drive. My test case for this was my original 14-part backup set, which should be of special relevance to you as it is my photo archive. The older years are all smallish out-of-camera JPG files; the last 8 years or so out of the 14 consists of 50% Canon RAWs and 50% out-of-camera JPGs. (I habitually shoot raw + JPG and keep both files.). So in your case - I presume that you shoot all raw - the average file size will be larger again and the data transfer rate correspondingly higher. At a guess, perhaps 50MB/sec or about 2/3rds of the speed of a PMR drive, maybe higher. I'll test that for you if you like.

Hmmm ... turns out to be even better than that. using my folder of 2012 pictures and copying only the raws ('coz you won't have any JPGs, I imagine) I'm getting 70MB/sec, which is faster than any of my conventional PMR drives can do. The SMR drive is currently 82% full, the same as them, so it's a pretty fair test.

On the evidence so far, a shingled drive sounds like exactly what you need, Lunar. The only caveat at this stage is performance after we have filled the drive up with data, then erased something, then start putting some other thing on instead. Will that be lots slower? I'll find out over the next few days.

 

[ddrueding]

Short answer: the Seagate Archive drives hates writing small files: it's only half as fast as a bog-ordinary standard PMR drive. On the other hand, it writes large files almost twice as fast as a standard drive does. Given the very high areal density, this makes sense. We should also note that this is a brand new drive, so all of these writes are first-time writes. The drive is now 80% full. What happens when we start doing overwrites? Will it slow down hugely? Or does the firmware move data around during idle time so that, after a while, your free space becomes contiguous, ready for faster writing?

So backing up via a file consolidation tool (Winzip or Windows Backup) might offer a significant performance improvement? Interesting.

But at a more sophisticated level, it occurs to me that this is exactly the sort of thing you'd be doing if you were a cloud storage provider looking to optimize your performance at the lowest possible cost per GB. SMR drives are the cheapest per GB, so you want to use them for your seldom-accessed bulk storage. On the other hand, you want to get the best write performance possible out of them because that lets you save money on expensive faster flash and PMR drives. I shouldn't think you'd pause the write process, you'd just distribute it between different drives. In the simplest example, you'd have two drives and write to each one alternately while the other one rests. Similar logic applies to manufacturers of multi-drive storage boxes (NAS and the like), if they thought the expense of the new firmware was worth the saving on drive space.

This is what I was thinking.

 

[Tea]

Now we get to the bad bit. As a test, this morning I wrote a few tens of GB to a temp folder, then deleted it. Went out to the office for three hours or so, then came back and started another copy. It is mega slow. Although the data is erased and the recycle bin emptied, The drive is laboriously doing the read, write, write thing, and I do mean laboriously. It's been running at about 200kB/sec for five hours now. Yes, kB. Obviously, the firmware isn't smart enough to understand the file system.

For backup purposes, this isn't so important, but it's starting to look as though this unit isn't going to cut the mustard as a data drive.

Required here is a way of telling the drive to ignore the erased data. Presumably, this would mean a driver, or at least a utility. (Possibly there is already such a thing, but if so it isn't available from Seagate.) I'd be very surprised indeed to discover it wasn't both possible and practical. The drive is obviously perfectly capable of going a lot faster in this instance as it doesn't need to preserve the bits its overwriting and, doing fresh writes on an empty drive, it runs between 10 and 60 times faster than this. I'd be interested to discover what happens if I format the drive (short format? long format?) or delete the partition and recreate it. Does that reset the firmware's counters? Seagate really need to document this stuff. I was going to say "document this stuff much better", but actually I mean "document it at all".

 

[Tea]

Oh, and curiously enough, there is still about 1TB of blank, never-used space on the drive. It is presumably writing over the deleted test data instead though.

 

[Buck]

I checked out the WD archive drives. they have a 6TB unit available but it's hellishly expensive - about double the price per GB of the Seagate one. It's an enterprise-only unit, not a SATA drive. No doubt it's a lot faster and the "archive" part has to do with running cool and quiet and using less power than a standard drive. Anyway, not relevant to my needs, but interesting.

The WD Archive (Ae) drive is a PMR datacenter drive, but is SATA III. It'll work in your machine like any other SATA III drive. What is the price you are seeing for this product in your area? In the US it's ~415AUD.