My fun project

[Mercutio]

Odd that Pradeep mentioned how hot Hitachi drives get in his other post...

Over the weekend I got my replacement batch of 20 7k400 SATA drives, and I decided to use some of them to move around some existing data.

So Saturday I installed four of them on a cheap 4-port controller controller, a Supermicro 5-bay hot-swap enclosure, and set them up as a soft-RAID5 on a 2000 Server. Let it build for half a day and spent the rest of the weekend copying things to that array.

I do stuff like that all the time, so I can add disks to arrays or swap old disks for new ones or whatever.

Except Monday night, when I got home from work, the machine was hung. That's not normal at all. I powered it off, pulled the drives out of their racks, and they were REALLY hot. Too hot to touch, really. Hotter than a Cheetah gets. And this was despite active cooling (the room was around 82 degrees, even with an AC running. It's been $%^#ing hot here).

I opened the machine, splayed the drives out so they weren't touching, and rebooted, after giving the drives time to cool, I rebooted.

chkdsk came up to say "invalid file attributes, truncating at position 5", which I take to be some kind of data corruption. Mind you, this message only flashed on the screen - I don't know if chkdsk did anything or not...

Except that Disk management shows my array to be 100% free space (I had been using a little over 1TB of it, I think), post array rebuild. chkdsk only displays that short message and then exits.

Hm.

I don't really have any data recovery software made to work on dynamic disks, but I've been kind of wondering what I can do about something like this, anyway.

So I think I'm going to try to recover that data, just to see if I can. I think that probably something in the MFT is messed up or conflicting or something, and given that chkdsk basically doesn't do anything, I'm guessing the data is still there. Since Windows CAN see the whole volume, I think I'll be able to find some way to work with it.

My first tool will be Ontrack Easy Recovery Pro, since I own a copy. After that I'll probably test demo products to see if I can find anything that works.

Should be fun.

 

[Buck]

Looks like a good project. Let me know about Easy Recovery, because I use that too.

 

[Handruin]

I've had positive results in the past with Get Data Back. If you're experimenting, give it a try.

 

[ddrueding]

I'm quite interested in your results as well. I don't have much experience with dynamic disks, but have always wondered how hard it would be to move them around and do maintenece/recovery stuff on them.

 

[ddrueding]

Hoping you hadn't deleted the original of the data yet?

 

[Mercutio]

Hoping you hadn't deleted the original of the data yet?

No. I just want to see what I can get off these drives.
For the record, Easy Recovery Pro got nothing. Nada.
Active Partition Undelete didn't see anything, either.

Get Data Back for NTFS did something interesting: It managed to recover some data - about 20% of the array, actually - well enough that I could read files and re-save them if I chose (it also took about 2 hours to analyze my volume to do even that much).

So I visited RunTime software, and found that they have a RAID Recovery tool, and it's a 30-day free trial!

So I tried it. I've tried bunches of different parameters for determining the precise interleave of my array, but no luck so far. For me to proceed I'm going to have to build an entire second 1.2TB array to save as a single image file so that Get Data Back can analyze it... and then I'd need a third to actually SAVE any of that data. Good thing I have all these 7k400s sitting around.

I guess what I'm going to do is install a second SATA controller in that machine (it only has two native ports of its own, and I'm using them), install four MORE drives, probably on an AT power supply, since at this point I'm a little worried about the PSU holding up, and dump whatever it wants onto the new drives. And then I can analyze from there.

I'm guessing this will be a 2 or 3 day process. But since I had positive results just from Get Data Back, I'd say there's a good chance this is recoverable.

 

[ddrueding]

I have had good experiences with Get Data Back, but it does take FOREVER to do anything. Thanks for the update, and glad to hear you didn't lose anything.

 

[CougTek]

Over the weekend I got my replacement batch of 20 7k400 SATA drives,...

Hitachi didn't mind to exchange them? I mean, they really swallowed that the drives were DOA and that it wasn't the customer's fault?

 

[Mercutio]

I managed to get my distributor to take them back, not Hitachi.

Anyway, regarding my RAID recovery: I let Runtime software's RAID recovery make an image of the ruined RAID. This is a process that has been running for, uh...

15 days. Approximately.

Fortunately the machine that's going to run Get Data Back NTFS is a slightly more muscular machine. Supposedly, I only have about 10 hours of analysis to do after this.

 

[Mercutio]

... and there's my data. All of it. Right where I left it.
I used RunTime Software's Get Data Back NTFS and RAID Recovery tool.
Processing took around 16 days, all told. I recovered 937GB of data. Recovery was largely automatic.

I'd say that's pretty goddamned impressive.

 

[Handruin]

Man, that's a long time for recovery, but it worked.

 

[Mercutio]

Actually, as I'm looking at the data...

It did very well with small files - pictures - from a spot check of a couple hundred out of 460,000 - I only found two that didn't open. Which were probably bad images to begin with. That's a massive segment of my recovery, but it's not everything that I copied over.

It did less well with MP3s - mostly files between 10 and 20MB: About 33% open and play back.
AVI and MPEG files (mostly 500MB - 2GB) are at about 33% as well.

My precious .CBR files (collections of JPEGs that have been RAR'd or ZIPped and basically contain scanned comic books) fared terribly - I've only been able to open ONE out of the fifty or so I've tried.

Comparing what has been recovered with the existing data that I have on one of my file servers, I can see file sizes that're mostly right. For the files themselves, though it actually looks like GBDNTFS has just written a lot of file full of zeroes. At least, that's what a hex editor is telling me.

Now I'm working through GDBNTFS with a different set of parameters. It'll run overnight, but I'm curious to see if any more of my files become readable in the process.

 

[tazwegion]

Considering the amount of time you've spent on this project... the thread title implies a little masochism doesn't it? ROTFL :lol:

G/L all the same

 

[Mercutio]

I've spent hardly any time at all. Every couple days I VNC into the PC that's doing the work and check the status of a blue bar. When that gets done I check the integrity of whatever it recovered.

So far, the stuff from Runtime Software is the ONLY program that's been able to deal with an NTFS error across a softRAID, but my working theory at this point is that I'm working out of a software limit of some kind, due to the size of the array I'm trying to recover. I've tried a couple different sets of parameters now, and none have been much more than 50% successful.

 

[tazwegion]

My point was, recovering lost data is NEVER fun

As far as that software limit idea/theory... could it be because the fact that it's a 30 day trial version? :eekers:

 

[Bookmage]

I've used Get Data Back with much success on normal partitions and single hard drives. GDBNT worked great on dynamic drives as well as basic drives. Thankfully I haven't had to use it on raid partitions, but I do remember it being slow. It took a good 20+hours for a 400GB drive. I'll have to remember this for future reference.

 

[Mercutio]

Funny but true story. I call it "Just how awesome IS a GA-7S748S?"

I've only sold maybe six nforce2 systems in my life. Last weekend one of them died. Turned out to be a bad Asus board (there's a shock!), but as it happens, I have one sitting in the machine on my tech bench... the one that's been running iterations of my RAID recovery.

It's been taking something like 10 or 12 days(!) to write the 1.2TB image file to the soft-RAID0 I've been using, from a hardware RAID5.

Anyway, today I tore apart my little bench machine, fixed my customer back up, and, not having another nforce2 board handy (I have several at my office...), I went ahead and dropped in a GA-7S748S that I got back from an RMA last week. Didn't change the CPU (XP2500) or RAM (1GB PC3200). Just put all my disks n' stuff on the new board and fired it up.

Also, for the record, the SiS board booted fine from a Windows Server install that'd been done on an nforce2.

I restarted RAID reconstructor about 3:00PM today, figuring I'd check on it, oh, around the beginning of September.

Right now I'm sitting here eating my dinner and I see that it's 76% done writing an image.

I cannot begin to imagine how it could be THAT MUCH faster. I'm gonna be completely shocked if the image can actually be opened and run against GDBNTFS.

 

[ddrueding]

A google for that gigabyte model gives nothing, did you type it right?

 

[time]

I'd guess it's a GA-7S748 with SATA added?

Pretty academic for me seeing AFAIK they didn't make it to Oz.

 

[Mercutio]

The vanilla GA-7S748 is actually that rarest of creatures, a board with neither sound nor network. There's an "S" variant that has sound and an "-SL" variant that has sound and network.

Anyway...

With the substantial upgrade that is the SiS disk controller, my trials became dramatically faster, and making the massive image file I need to run my recovery became a process of hours (could it be that the recovery software doesn't interact well with the nvidia board? Seems reasonable), rather than weeks of time. I've been managing two runs a day since I made the switch.

My 9th image (of a possible 24) happened to be the one that is a 100% match to preexisting files. Since I know better what to look at now, I spot checked primarily 1GB or larger files - mostly mpegs and DVD images. This time, unlike my first run, everything I checked opened perfectly.

Total time to full data recovery from scrambled array 36 days, 12 hours. 6 days over the limit for the trial copy of the RAID Reconstructor I used.
But to me it's worth $100 to find out that the I can do this again if I ever have to.

Sock Puppets and gentlemen, it can be done.

 

[mubs]

Thanks for the valuable info and for being the guinea pig, Merc. You're a swell guy.

 

[Splash]

...I cannot begin to imagine how it could be THAT MUCH faster...

The crap performance issue is likely an issue with the way nVidia has implemented buffering, not a GetDataBackNTFS problem.

The GetDataBackNTFS test is pretty convincing testimonial as to their product's effectiveness. You should E-mail them your observations.