Newbie question on RAID

[burchis]

I have a RAID-0 (2-80gb) on my system. I am concerned about file fragmentation on this RAID configuration. Does WindowXP's defrag utility correctly read and display data for RAID's that are setup with 3rd party IDE controller cards?

ie- Starting with a freshly formated RAID drive, I created two video (avi) files from a program that captures video via firewire input. This program is Dodcap but regardless of the program, WindowXP's file management should correctly store the files in sequential order. However, after two files have been placed on the RAID drive, Microsoft's defrag program recommends that I defrag the drive. The files appear to be fragmented according to the analyzes report. I would expect that these two files should of been written to the RAID in sequential order.

Is it possible that Microsoft's defrag utility is not correctly interpreting the files on the RAID via the 3rd party controller, or am I experiencing some other setup problem?

Looking for an explanation for this.

 

[time]

Fragmentation is a function of the file system and has nothing to do with the arrangement of disk blocks.

The Windows defragmentation analyzer is hopeless and should usually be ignored. Nevertheless, I'd defrag if it recommends it.

Video files tend to be huge - you should expect some fragmentation unless you have a clean disk. Did you defragment before capturing the video? In any case, you may have system files scattered around that the defragmenter won't move; these guarantee your file won't be contiguous.

As a RAID 0 user, you do appreciate how easily you could lose your data?

 

[Mercutio]

I can attest that the builtin Windows 2000 defragger can defrag hardware RAID arrays. Also, never defrag a large RAID5 that you need to use anytime in the next month.

I'll let someone else do the RAID0-is-usually-a-bad-idea lecture, but I will say that unless you're doing something REALLY spectacular, the combined STR of your two drives will be MORE than adequate, fragmentation or no, and a drive with even a 2MB cache isn't going to be bothered by a ~15ms full seek to the next file fragment; you should be able to manage a constant data stream regardless. Unless you're doing realtime editing on multiple video streams, you're almost certainly fine.

 

[burchis]

The Windows defragmentation analyzer is hopeless and should usually be ignored. Nevertheless, I'd defrag if it recommends it.

What do you mean hopeless and to ignore, and then in the same breath say I should respect it and defrag nevertheless? This makes no sense to me!

Video files tend to be huge - you should expect some fragmentation unless you have a clean disk. Did you defragment before capturing the video? In any case, you may have system files scattered around that the defragmenter won't move; these guarantee your file won't be contiguous.

Taken from my second paragraph of my original posting- "ie-starting with a freshly formated RAID drive...." Wouldn't a freshly formated drive be better then defraging the drive? And what system files would I have on a RAID drive that is freshly formated? btw- this is not my system drive but dedicated drives used for video capture.

Two files that become fragmented doesn't cause me any trouble on a 160gb RAID setup. However, if I continue to add more video files to this drive and eventually approach the capacity of the drive, then file fragmentation will become a factor. I am simply wanting to know if RAID's via IDE controller cards and disk management from WindowsXP is truely compatible if every respect, especially in writing files to disk.

 

[Mercutio]

Yes, it is.

The thing you should be most worried about, from a performance standpoint, is fragmentation of your MFT. When an NTFS volume gets to be about 85% full, I've noticed that performance seems to drop like a stone.

 

[burchis]

As a newbie to this fourm, what is MFT?

 

[Mercutio]

Master File Table. The NTFS version of your File Allocation Table.
It can be fragmented. Fragmented MFTs are bad because they create lots of extra seeks to get to the files or parts of files that you actually want.

 

[burchis]

Don't these MFT's have mirror backups to prevent fragmentation?

 

[burchis]

What actually happens and why would you "Mark partition as active"?

 

[Tannin]

The mirrored MFT or FAT is simply for data integrity purposes: i.e., the second copy is there just in case the first copy gets hosed. At least that is certainly the case with FAT file systems, and I can't see why it would be any different with NTFS.

More generally, defragmentation is grossly overrated as a performance tool. It used to be really important, back in the days when a typical average seek time was 85ms. Yes, that's 85, not 8.5. On-drive buffer in those days was ... er ... I can't even remember if there was any at all, and operating system cache was still a gleam in the designer's eye.

Back then, fragmentation mattered. These days, and indeed any time since about ... oh ... about 1990 or so, fragmentation has become very much a non-issue. Just ignore it. Unless your drive is way too full (Mercutio's 85% figure is the sort of figure I have in mind, maybe even more) it is not noticable. You can't detect it with the naked eye, so it doesn't matter.

Finally, yes, a freshly formatted drive cannot be fragmented. Formatting is the most effective defragmentation tool of all. Indeed, it is the only one I ever use; typically, oncve every couple of years or so. That's plenty often enough.

 

[time]

What do you mean hopeless and to ignore, and then in the same breath say I should respect it and defrag nevertheless? This makes no sense to me!

I was trying to explain that the analyzer usually understates the effects of fragmentation because AFAIK, it estimates it as a fraction of disk size rather than calculating the effect on individual files. Consequently, it will rarely suggest defragging, so you should ignore it and defrag anyway.

On the other hand, if things are bad enough for it to recommend you defrag, your disk is quite likely to be in a million pieces - figuratively speaking.

It's also possible that it's just plain wrong, which is why I tried to caution you about accepting its findings as gospel.

Taken from my second paragraph of my original posting- "ie-starting with a freshly formated RAID drive...." Wouldn't a freshly formated drive be better then defraging the drive? And what system files would I have on a RAID drive that is freshly formated? btw- this is not my system drive but dedicated drives used for video capture.

I considered all these possibilities before responding, but you declared yourself to be a newbie playing with RAID 0, so I decided to assume you were. "Freshly formatted" does not exclude the installation of an operating system between the format and the video capture. In fact, without your additional information, the presence of an OS was a reasonable assumption.

 

[Tea]

Really stupid question time. You have an NTFS drive. The MFT is located in the middle of the drive - e.g., if it's an 80GB drive in a single partition, at about the 40GB point. This is where NTFS always puts the MFT, in the middle (to reduce seek times). Assume the drive is freshly formatted and empty.

Obviously, if you then copy over a 50GB file, it will be fragmented (some of it will have to be on each side of the MFT).

But what about a 10GB file? Will NTFS put one half before the MFT and the other half after the MFT?

 

[time]

What actually happens and why would you "Mark partition as active"?

It means you want it to be bootable.

 

[time]

Tannin didn't mention that fragmentation also occurs when you have a lot of churn. Windows is really good at that.

 

[burchis]

Thanks to all that have contributed, the light is becoming brighter and brighter.

 

[LiamC]

This is where NTFS always puts the MFT, in the middle (to reduce seek times). Assume the drive is freshly formatted and empty.

Learn something new everyday. I knew HPFS did this, but everything I've read about NTFS was ambiguous. Thanks Tea.

 

[mubs]

In defragmenter, the color green is used to represent "system files". On non-system / non-swap NTFS partitions (even those that are empty), I presume this chunk is actually the MFT. All of my non-system / non-swap NTFS partitions have a big green chunk right at the beginning. They also have a much smaller green chunk about 35% into the partition, counting from the beginning. This smaller chunk is about 10% or smaller of the size of the big chunk at the beginning.

Somebody care to explain?

 

[Tea]

This is where NTFS always puts the MFT, in the middle (to reduce seek times). Assume the drive is freshly formatted and empty.

Learn something new everyday. I knew HPFS did this, but everything I've read about NTFS was ambiguous. Thanks Tea.

I'm only assuming that NTFS is the same in this regard. I learned about HPFS, never bothered reading up on NTFS Bill, just assumed that, because NTFS is only a renamed version of HPFS which was itself largely written by Microsoft, NTFS would be the same. After all, why would you change something like that - the centre is obviously the best place to put a file table, so you wouldn't change that part. At least I wouldn't have thought so.

 

[LiamC]

Could have been something like licensing. Dunno.

I like how MS said that NT was new from the ground up. No OS/2 code. I still have NT 3.5 on disc somewhere, and IIRC, you can cause it to generate OS/2SYS! errors! :eekers:
:mrgrn:

 

[Mercutio]

If you take an NT3.5 machine, you can format its drive with HPFS and run some OS/2 programs on it (command line ones). You can then upgrade to NT4, where those capabilities still work, and then to Windows 2000, where they will continue to work (although, if you START with Windows 2000, neither HPFS nor the OS/2 subsystem are available, IIRC). Windows XP and 2003 are the first versions to actually complain about HPFS.

 

[Tannin]

But even with them, you can apply patches and stuff to maintain HPFS compatibility. Not that it matters, as most OS/2 users these days are presumably running JFS instead. Very nice file system.

 

[Mercutio]

Like getting OSX to run on a PC, having OS/2 stuff running on a Win2000 machine is great just for the novelty value.

 

[Bozo]

I usually run O&O defrag and 99% of the time it shows the MFT in the center of the partition. When it doesn't, the partition has been manipulated in some way. IE: partition Magic.

Bozo :mrgrn: