Excruciatingly slow writing to external disks

[mubs]

I thought I started a smilar thread a few years back for the same problem with the same HDDs, but can't seem to find it in Advanced Search.

Anyway, I've been on a backp up spree. I have 2 x 500 GB USB3 Buffalo external HDDs and 2 x 1 TB USB2/FW800 WD external HDDs. My system has USB2 and USB3 ports on the motherboard, and I have an add-in PCI-E FW800 card from Unibrain. Speeds have been pitiful.

With the Buffalos: I mainly sync, so the slowness isn't a killer. I was using the front ports on the case; maybe there's a problem there. Will retest with the ports built into the motherboard that are at the back.

With the WDs: I reformatted them and copied from internal to them, one internal partition at a time. I tried USB2 first (because I was lazy to go to the back of the case for the FW ports), and got about 2 MB/s transfer speed. Then I tried FW800. Poor again.

The externals are set to write faster (cache) in Device Manager; they are on a UPS.

I searched and found two suggestions on the web: turn off Remote Differential Compression (Control Panel / Programs & Features / Turn Windows Features On and Off), and turn off Background Intelligent Transfer Service. Both are on by default in W7.

Here are the stats. The start time is when I look at the system clock (I have to click on it to see the seconds) as soon as I let go of the mouse button on the drag-n-drop. There are two end times: when the Windows "Copying" windows goes away, and when the light stops blinking on the WD external. This is a crude sample, and in these tests, there are few files, but each is individually large.

Remote Diff Compression on
BITS Off
18,891 MB
10 files
12:53:39
13:04:05 = 30.1 MB/s
13:06:20 = 25.15 MB/s


Remote Diff Compression off
BITS On
3,703 MB
9 items
12:40:05
12:41:07 = 59.7 MB/S
12:43:20 = 29.0 MB/s


Remote Diff Compression off
BITS off
3,987 MB
3 files
12:46:35
12:47:17 = 95.9 MB/s
12:49:27 = 23.18

There seems to be a buffer worth about 2 mins 10 secs inside the external drive. Oddly, the time taken till the LED on the external stops blinking seems to be about the same regardless of whether Remote Differential Compression and Background Intelligent Transfer Service are running or not.

After copying these (tests were actually part of the backup {copy} process), I copied the rest of the partition in one go to the same external HDD: ~ 83,400 files, ~ 250GB. Now the copy window says "1 Day Remaining". What horseshit is this? The LED on the external blinks continuously at a steady pace.

The drives in the externals are probably 5400 RPM WDs. Woe unto me.

I will later test the Buffalos again for xfr speed. If anybody has any advice I'm all ears. Thanks.

 

[mubs]

I tried to kill the copy; it hung on the Canceling window forever till I cut the power to my system.

There's enough anguish on the web about WD My Book desktop external drives. At least WD doesn't discriminate between A) Windows and Mac users - they're both screaming and B) Interface: USB & FW both suffer these slowdowns.

Oddly, one chap who had these issues says it flies when he uses an Ubuntu LiveCD and strongly recommends taking this approach. This points the finger at Windows (and MacOS) that seem to choke on large file transfers.

Such major flaws in Windows even after 15 frigging years?

 

[Bozo]

I also had slow copying to an internal drive that is on a removable tray. I stated using Robocopy and running it from a command prompt. Huge improvement in speed.

I also wrote a small batch file that runs from the command prompt. Again, a huge improvement in speed.

Seems odd the 25+ year old technology is faster than the latest and greatest. Might explain why Linux is so much faster.

 

[mubs]

All is good. I did a Coug on the WD enclosure; ripped it apart, and sure enough there was a WD Green drive in it. Put that in a Transcend external enclosure I have that's USB3. Had initial hiccup because the drive is so intricately linked to the controller in the WD enclosure that it went back to raw state and I didn't realize that. After initializing and formatting, I'm getting the kind of speeds in the Transcend enclosure that I should eb getting. Still too afraid to copy large chunks of data; will do piecemeal to keep it good for now till I get it all backed up.

I had ordered the Transcend enclosure more than a year ago, but never opened it. It's a beaut; impeccable fit and finish, everything needed supplied, easy to use, and really fast USB3. I had ordered it on Amz US and had my bro bring it over. What a contrast between it and the WD POS.

I now have a second WD external to gut, but don't have an enclosure for it. Need to find an equally reliable and well made one.

 

[Mercutio]

I've found that I'm better off copying tens of thousands of tiny files using Teracopy or even xcopy rather than Windows Explorer. Big files usually average to be about the same regardless of copy method, but if I'm copying 30,000,000 jpgs someplace, the differences can be enormous.

 

[mubs]

Thanks Merc, I'll look into Teracopy. I guess xcopy still works in W7, in a CMD window of course.

What took me 1.75 days with the WD external and still didn't get 1/4 of the job done has taken me .25 days with the same disk drive in the Transcend enclosure. Go figure. I'm sure you're going to tell me "I told you it's WD". But I bought those some 3.5 years ago in a moment of weakness when I thought all the drives in my then system were dying and I needed large backup space in a hurry and those were the only 1 TB externals I could find in a hurry.

 

[Stereodude]

Be sure to look at FastCopy.

 

[Buck]

XCOPY is still a speedy solution for copying files in Windows 7 (with the appropriate switches), while Drag n Drop with Explorer is not. Another great backup tool is Cobian Backup 11 - Gravity.

These WD drives and their speed are not necessarily the issue, as 5,400-rpm drives are plenty fast for contiguous writes (incremental backups would be different). I use both backup methods with USB 2.0 2 TB Passport drives. Switching to USB 3.0 is a good idea, but speed is also dependent on the USB bridge/hub chip. My biggest issues have been the speed difference encountered with implemented USB solutions pertaining to motherboards and enclosures - so inconsistent, and USB 2.0 itself. I recently had a customer that brought over 5 TB of FLAC files from two external drives, a Seagate USB 2.0 and a 3.0, to his new computer system. He copied the data with Drag n Drop in Windows 7. The speed delta between the two drives was astounding, with the USB 2.0 drive being super slow.

At least you have the benefit of being able to remove the drive from the enclosure and using another enclosure. Newer 2.5" USB drives have the USB chip/port integrated to the drives PCBA. This seems to have not migrated yet to 3.5" drives.

 

[Mercutio]

Be sure to look at FastCopy.

That's actually really interesting. I spent a lot of time a few years ago ensuring that my file paths didn't exceed MAX_PATH after having a ton of problems because of the way I organize music. I've never heard of that one. Thanks.

With regard to performance differences for USB enclosures, I believe it's all about the SATA bridge chip that's in use. You really have no way of knowing what you're getting before you buy and for some enclosures (ioGear, in my experience, is definitely in that category) you don't always get the same thing even if the box is visually identical.

 

[Mercutio]

I should also say that file copies in Windows 8 are different beasts from Windows 7. They're prioritized differently and they seem to be sort of universally faster on new Windows than older ones. It's probably not a reason to upgrade for most people but I do know this is a problem that will fix itself when the whole world moves to Windows 10 in a couple months.

 

[mubs]

Thanks guys. The person who who designed the enclosure electronics for the WD external should be hung upside down by their private parts. That was the culprit. Apart from the crappy implementation of the USB2 / FW800 interface, it forces a virtual CD drive on you (this can be turned off, but not deleted, and requires addition sw download and installation).

How's this now? This is the same bare WD drive in the Transcend enclosure:



Can't seem to get the pic to show inline.

 

[mubs]

Merc & SD, thanks for the tips on Teracopy & Fastcopy.

I tried them and will pass. IMHO, they are not robust and barf on large copy jobs. FYI, my CPU (inconsequential, but nevertheless) is an i5-3570k @ 3.40 GHz. I have 16GB of physical RAM. All drives (even the old ones) had enough free space. My system is lean and mean; no games installed, no questionable software, and the system is rock solid (till the one drive started to fail). All copies were between two spindles, not inside the same single spindle.

Teracopy died several times, with Windows popping up a message that Teracopy stopped unexpectedly and it would check for solutions. This never happened if I gave it small chunks - < 30GB, for instance. Also, many times it stuttered and was doing nothing; no disk i/o, nothing happening that I could make out. These pauses lasted for 3-10 seconds, and it would suddenly resume. It was the fastest at copying, but the frequent barfing got to me, and I was wondering how reliable the copying was.

Fastcopy also hung a couple of times on large copies - > 100GB folders. Again, the reliability concerns me.

I decided that I would stick with Windows Explorer. At least I can trust it. And with the two new 3 TB drives, it is very, very fast, so I'm happy with it.

 

[Stereodude]

I've used fastcopy to copy and verify multi TB chunks of files and have never had an issue.

 

[CougTek]

Be sure to look at FastCopy.

I'm trying to download the 64-bit version and I can't even access the website. I've tried via Majorgeeks too without success.

Anyone here has the 64-bit 2.11 version?

 

[Stereodude]

http://stereodude.net/FastCopy211_x64.zip

 

[CougTek]

Thanks.

 

[mubs]

After my experience with WD externals with crappy software that can't be uninstalled, and horrible speeds, I was nervous buying another WD external, this time 2.5" 2TB. It did come with trialware in folders, needed to be installed, so I just formatted the disk and got the "whole" space back. That is, 1.81 TB out of 2TB. That sucks when you lose 200GB because of marketing gimmicks. Strangely the disclaimer that it is 2TB decimal and not binary isn't on the box. Made in Thailand. 3 year warranty. Their warranty support is supposedly good here (I can see Merc smirking).

I did a full scan yesterday, returned no bad sectors; took 6.5 hours. Incidentally, in W10, drive --> properties --> tools --> check drive has a single option that apparently does a chkdsk only. There is no option to scan sectors like in W7. I had to use Minitool Partition Wizard to check the sectors.

 

[snowhiker]

Moderately O/T.

I wanted to copy the entire Dell monitor CD driver disk to a folder on my HD. Create folder on my HD and Drag-n-Drop using Windows Explorer. Copy proceeded at 1.3 MB/s. Took MINUTES to copy 354 MB. WTF. Googled, then went to Control Panel --> Admin Tools -->Services --> Turned off "Windows Search" service.

Tried copy again and only took a few seconds. <face-palm>

Any issues if I simply disable this service? Is this service different than the "search" box on the start menu?

 

[ddrueding]

IIRC, that service is required for the simple start->search of local programs.

 

[time]

That service may be just the indexer, because after stopping it, all my searches still work under Windows 7.

 

[mubs]

Well, well. My Toshiba Canvio Basics 2 TB arrived today, and completed surface testing in five and a half hours, a whole hour faster than the "similar" WD Elements on the same USB 3 port, same testing sw.

Now copying stuff to it, and it is going much faster than the WD. Was cheaper to buy, and is running cooler as well.

Only unknown is warranty support; it's supposed to suck here, that's why I bought one of these and one WD.

 

[Mercutio]

Toshiba warranty service is a PITA to deal with in the US. It's probably not going out on a limb to guess it's not going to be better in India. Warranty service is the one and only thing WD has consistently done well.