HDD performance bottleknecks
- Thread starter Jimshady
- Start date
Mercutio
Fatwah on Western Digital
Data set is a big one. Most drives these days are optimized for high transfer rates and rather unappealing access times. If your data set consists of lots of little files (e.g. thumbnails of graphics files), your performance will generally be poor.
Another might be bus contention, particularly over the 32bit/33 MHz PCI bus. While I'm aware that plenty of devices now have specialized non-PCI busses to call their own, that doesn't mean everything is using them. Gigabit NICs, RAID cards, Video-Input cards and even modern sound cards can deliver substantial amounts of data to a PCI bus. It's very possible to oversaturate PCI now, even in a standard desktop PC.
Core frequency is another issue. If your motherboard is still only servicing its IDE ports on a 133MHz bus vs. 200MHz+ on a newer board, that's a performance bottleneck. A P4 3GHz on a 533MHz bus has an 18x multiplier. That's a lot of idle time!
Another possibility is using old or outdated drivers for your storage devices. Every once in awhile I find new-ish PCs that don't have the proper IDE driver installed, or worse, don't have DMA enabled. That is, in a couple words, not cool.
Need more? How about improper use of RAID? An application that needs to write a lot of non-critical data probably doesn't need to be on a RAID5 array. If your application is "gaming", where the load on your disk drive usually consists of loading just a couple large texture and map files, RAID0 is a liability.
What about a poorly placed swap file? Normally you want your swap file on or near the "beginning" of a drive's space. If your swap space is located far from the data you're actually using, that can be a liability.
Er... is that a good enough start?
Another might be bus contention, particularly over the 32bit/33 MHz PCI bus. While I'm aware that plenty of devices now have specialized non-PCI busses to call their own, that doesn't mean everything is using them. Gigabit NICs, RAID cards, Video-Input cards and even modern sound cards can deliver substantial amounts of data to a PCI bus. It's very possible to oversaturate PCI now, even in a standard desktop PC.
Core frequency is another issue. If your motherboard is still only servicing its IDE ports on a 133MHz bus vs. 200MHz+ on a newer board, that's a performance bottleneck. A P4 3GHz on a 533MHz bus has an 18x multiplier. That's a lot of idle time!
Another possibility is using old or outdated drivers for your storage devices. Every once in awhile I find new-ish PCs that don't have the proper IDE driver installed, or worse, don't have DMA enabled. That is, in a couple words, not cool.
Need more? How about improper use of RAID? An application that needs to write a lot of non-critical data probably doesn't need to be on a RAID5 array. If your application is "gaming", where the load on your disk drive usually consists of loading just a couple large texture and map files, RAID0 is a liability.
What about a poorly placed swap file? Normally you want your swap file on or near the "beginning" of a drive's space. If your swap space is located far from the data you're actually using, that can be a liability.
Er... is that a good enough start?
Buck
Storage? I am Storage!
Improper placement of drives on the ATA cable. For example, using a single hard drive for your system and placing it as Master to your CD-RW, and then wondering why burning CDs takes longer then expected.
Improper use of a hard drive. Such as using a desktop ATA drive for a high I/O environment. Yes, I know they're cheaper then SCSI drives and 10,000-rpm ATA drives, but if you really want the appropriate performance for the task, spend the right amount of money for the correct hardware.
Excess heat. Build a mini-heater-computer system in a mini-tower case, greatly inhibit the airflow, and watch hard drive performance drop like a lead balloon. Track calibration becomes exponentially difficult as temperatures increase.
As a contrast to your question Jimmy: Using the fastest drive around to stream audio or video data is a huge waste of hardware. Using RAID 0 or demanding for CQ capabilities for surfing the web is a great waste of hardware. Granted, I don’t’ mind selling computer components, but I’m not going to talk a customer into something they don’t need.
Improper use of a hard drive. Such as using a desktop ATA drive for a high I/O environment. Yes, I know they're cheaper then SCSI drives and 10,000-rpm ATA drives, but if you really want the appropriate performance for the task, spend the right amount of money for the correct hardware.
Excess heat. Build a mini-heater-computer system in a mini-tower case, greatly inhibit the airflow, and watch hard drive performance drop like a lead balloon. Track calibration becomes exponentially difficult as temperatures increase.
As a contrast to your question Jimmy: Using the fastest drive around to stream audio or video data is a huge waste of hardware. Using RAID 0 or demanding for CQ capabilities for surfing the web is a great waste of hardware. Granted, I don’t’ mind selling computer components, but I’m not going to talk a customer into something they don’t need.
Mercutio
Fatwah on Western Digital
Poor choice of pairings:
PIO-mode devices on the same IDE channel as UDMA devices lead to having two PIO mode devices (lesson: don't put an LS-120 on the same cable with anything you actually want to use)
Single-ended devices do the same thing to LVD SCSI.
And while I'm at it improper termination on a SCSI bus is another possible factor.
PIO-mode devices on the same IDE channel as UDMA devices lead to having two PIO mode devices (lesson: don't put an LS-120 on the same cable with anything you actually want to use)
Single-ended devices do the same thing to LVD SCSI.
And while I'm at it improper termination on a SCSI bus is another possible factor.
PIO mode vs UDMA mode
cache disabling; or mis-configuring a cache (external to OS)
not enough RAM for caching purposes;
Insufficient paging file
wrong or non-existant drivers; improperly configured drivers
raid-rebuilding
degraded raid
Fragmentation
Excessive number of files in a single directory
file system corruption (forcing the OS to recover the data)
Encryption or compression
Bad sectors/tracks; missaligned tracks; bad timing tracks; excessive temperature; vibration - Anything that might cause ECC errors that require the HD to correct data or reset itself on an ongoing basis.
cache disabling; or mis-configuring a cache (external to OS)
not enough RAM for caching purposes;
Insufficient paging file
wrong or non-existant drivers; improperly configured drivers
raid-rebuilding
degraded raid
Fragmentation
Excessive number of files in a single directory
file system corruption (forcing the OS to recover the data)
Encryption or compression
Bad sectors/tracks; missaligned tracks; bad timing tracks; excessive temperature; vibration - Anything that might cause ECC errors that require the HD to correct data or reset itself on an ongoing basis.
sechs
Storage? I am Storage!
Mercutio said:
I don't have a problem with this. Windows XP sometimes does, but the hardware certainly doesn't.
Buck
Storage? I am Storage!
P5-133XL said:
Two excellent points that I often see, P5.
Customer: "There is something wrong with my harddrive, it is always working, and makes a lot of noise."
Me: [Check system configuration; notices 128MB of RAM for the usual overweight Windows XP system. Everything else is A-okay.]
Me: "Good news, your drive is fine, we could easily fix the problem with a memory upgrade."
Customer: "Memory upgrade? You mean I need a new hard drive."
Me: [Stupid customer, memory is RAM not the hard drive] "No-no, the hard drive is fine we just need to increase the amount of RAM you have in the system."
Customer: "But I already have a 40-gigabyte hard drive."
Me: [Wondering where I last left my Louisville Slugger.]
---------------------
Customer: "My hard drive light comes on a lot, and when I scan at higher resolutions, I get a memory error message."
Me: "What type of work do you do with your computer?"
Customer: "I do a lot stuff with photographs from my (kids, grand kids, birds, etc.)."
Me: [Check hardware and system. One 5,400-rpm drive, 128MB of RAM, equally sized page file, no scratch file for Photoshop defined.] "We can fix that. It'll take some minor software adjustments, and some hardware improvements."
Yes, it happens all the time especially with rounded cables (They put all the data lines in close proxismity of each other). That is why the newer 80 line udma cables with a ground between every data line was created. Those ground lines are there to absorb the crosstalk.
Note, that normally you won't see any effect of crosstalk other than performance. Starting with the ATA-66 standard there is an ECC mechanism between the HD and the controller that constantly monitors the validity of the data and if there is crosstalk corruption the controller will request a resend of the data with no error codes being sent to the computer.
Note, that normally you won't see any effect of crosstalk other than performance. Starting with the ATA-66 standard there is an ECC mechanism between the HD and the controller that constantly monitors the validity of the data and if there is crosstalk corruption the controller will request a resend of the data with no error codes being sent to the computer.
The location of an onboard controller and its specific BW limitations will be specific to the design of the chipset. To find out the BW limitations you really need to study your chipsets flow diagrams with their multitude of buses and their respective bandwidths. To give you a few example diagrams to emphisize this point.
For example the AMD 760 chipset:
You can see the Southbride connects to the PCI bus which connects to the Northbridge. So here everything shares the PCI bus. This structure was the norm till just very recently.
However with the Intel 875 chipset everything is different.
Here, different devices connect to the southbridge at different speeds and everything shares a single 266MBps bus between the Northbridge and the Southbridge. The individual buses like the ATA-133 all can operate at their maximum bus speed with no contention but at some point they will all have to share the internal NB to SB connection that operates at 266MBps and at that point the SB will throttle back the throughput of the individual buses.
Then here is how an Opteron 8131/8111 chipset is connected.
Here you will see a totally different structure. Here the onboard IDE controller is connected (along with a bunch of other I/O stuff) to the Southbridge which connects to the Northbirdge at 2x400MBps and the dual PCI-x bus independantly connects to the Northbridge. at .5/1Gbps.
For example the AMD 760 chipset:
You can see the Southbride connects to the PCI bus which connects to the Northbridge. So here everything shares the PCI bus. This structure was the norm till just very recently.
However with the Intel 875 chipset everything is different.
Here, different devices connect to the southbridge at different speeds and everything shares a single 266MBps bus between the Northbridge and the Southbridge. The individual buses like the ATA-133 all can operate at their maximum bus speed with no contention but at some point they will all have to share the internal NB to SB connection that operates at 266MBps and at that point the SB will throttle back the throughput of the individual buses.
Then here is how an Opteron 8131/8111 chipset is connected.
Here you will see a totally different structure. Here the onboard IDE controller is connected (along with a bunch of other I/O stuff) to the Southbridge which connects to the Northbirdge at 2x400MBps and the dual PCI-x bus independantly connects to the Northbridge. at .5/1Gbps.
nice. Thats an awesome response, thanks.