CougTek
Hairy Aussie
The small fan on the Perc 6i looks like a noisy one. That case is well ventilated, but not well noise insulated. You must not put this system in your bedroom or living room, aren't you?
I don't know Openfiler, but it seems like an interesting piece of software. I use FreeNAS at work, how does it compare?
I don't know Openfiler, but it seems like an interesting piece of software. I use FreeNAS at work, how does it compare?
Last edited:
ddrueding
Fixture
LOL. Wrong government.
I can't hear the 40mm fan over the 92mm fan that comes with the Supermicro drive bay. That fan is louder than anything else. You're 100% right that this case is well ventilated, but also not well insulated for noise. A few pieces of noise dampening material would be useful. The 40mm fan is rated at 3500 RPM, 4.11 CFM and 14 dBA if that helps at all.
I have a fan that I want to buy on auto-notification from newegg that will replace the noisy fan in the supermicro, but I have to live with it until then. The NAS is in my office room right now and I can't hear it anywhere else in the house. I also haven't been leaving it on 24x7, but will eventually put it in the basement or somewhere out of sight.
In my short comparison between freeNAS and openfiler, I like the UI usability of OpenFiler better than freeNAS, but the features seemed comparable. I also had hardware compatibility issues with freeNAS which made me move back to OpenFiler. My hard drives were disconnecting with the onboard nvidia SATA controller. Now that I have a Perc in here, I should try and go back to freeNAS to see if it even recognizes the controller.
With OpenFiler, I was also able to easily update all the rPath packages using the built-in update mechanism which I find very handy. When they come out with improvements for compatibility and performance (as was the case with their iSCSI software), I can easily get the conary package from their rbuilder project.
I have a fan that I want to buy on auto-notification from newegg that will replace the noisy fan in the supermicro, but I have to live with it until then. The NAS is in my office room right now and I can't hear it anywhere else in the house. I also haven't been leaving it on 24x7, but will eventually put it in the basement or somewhere out of sight.
In my short comparison between freeNAS and openfiler, I like the UI usability of OpenFiler better than freeNAS, but the features seemed comparable. I also had hardware compatibility issues with freeNAS which made me move back to OpenFiler. My hard drives were disconnecting with the onboard nvidia SATA controller. Now that I have a Perc in here, I should try and go back to freeNAS to see if it even recognizes the controller.
With OpenFiler, I was also able to easily update all the rPath packages using the built-in update mechanism which I find very handy. When they come out with improvements for compatibility and performance (as was the case with their iSCSI software), I can easily get the conary package from their rbuilder project.
I will do some shortly. I've been playing around with this NAS a lot and at the moment the RAID 5 is taken apart to do individual drive tests. It takes about 5 hours to rebuild the array, so that will probably start sometime later tonight. I'll have some numbers for freeNAS and OpenFile with basic file transfer tests for large files and for numerous small files. Are there specific things you are looking for?
Here is some basic numbers to start.
I did a file copy from my desktop computer (core i7 860) from a source drive (Samsung F3 1TB 7200 RPM) to my new NAS array.
My NAS is running OpenFiler 64-bit with a RAID 5 configuration. Keep in mind the RAID 5 is still initializing while I ran this test (because I'm impatient).
I copied a large file 3389.63MB in size and it took 31.7 (rounded up to 32) seconds via a hand help stop watch. That works out to almost 106MB/sec transfer over my GigE link. I'm also using Jumbo Frames at 9K. See attached image as reference.
I did a file copy from my desktop computer (core i7 860) from a source drive (Samsung F3 1TB 7200 RPM) to my new NAS array.
My NAS is running OpenFiler 64-bit with a RAID 5 configuration. Keep in mind the RAID 5 is still initializing while I ran this test (because I'm impatient).
I copied a large file 3389.63MB in size and it took 31.7 (rounded up to 32) seconds via a hand help stop watch. That works out to almost 106MB/sec transfer over my GigE link. I'm also using Jumbo Frames at 9K. See attached image as reference.
Another quick test with the same environment as above. This time I copied a file that is 7861.72 MB in size to hopefully avoid caching on either system. The NAS has 6GB of RAM and my desktop has 8GB of RAM.
The file transferred in 74.4 seconds. That works out to about 106MB/sec just like my last test. I've attached another reference screen capture to show network utilization is very high (which I feel is a good thing).
The file transferred in 74.4 seconds. That works out to about 106MB/sec just like my last test. I've attached another reference screen capture to show network utilization is very high (which I feel is a good thing).
I decided to try a different test with smaller files. I copied over a directory of RAW camera images with range from 10-15MB each with an guess at an average around 11MB. There are 569 files in 7 folders that were copied. The total file size is 6637.43 MB and it took 81.9 seconds to copy to my NAS. That works out to about 81MB/sec transfer rate.
My last test for tonight was to copy over smaller files and see how well it performed. For this test I copied over 1,716 mp3 files in 176 folders to the NAS. The files range from 3MB to 5MB in size. The total size transferred is 9535.02 MB and it took 144.4 seconds to transfer. That works out to about 66MB/sec.
I decided to try copying the same files back from the NAS to my desktop and the performance was a little bit slower. The time to transfer the data back was 185.1 seconds for the exact same data. That works out to about 51.5MB/sec.
I decided to try a copy back of a single large file that is 3389.63MB in size and it copied back to my desktop in 30.5 seconds. That works out to 111.14 MB/sec transfer. The network utilization was damn near pegged at 99% the entire time.
I decided to try copying the same files back from the NAS to my desktop and the performance was a little bit slower. The time to transfer the data back was 185.1 seconds for the exact same data. That works out to about 51.5MB/sec.
I decided to try a copy back of a single large file that is 3389.63MB in size and it copied back to my desktop in 30.5 seconds. That works out to 111.14 MB/sec transfer. The network utilization was damn near pegged at 99% the entire time.
BingBangBop
Storage is cool
- Joined
- Nov 15, 2009
- Messages
- 667
Rather than network utilization which is a good measure of network efficiency. Would it not have also been useful to have shown bytes sent per interval and bytes received per interval?
I sure that it may be beyond the scope of your NAS project but I'd like to know what happens if you have two network cards bound together. Will that increase the throughput?
I sure that it may be beyond the scope of your NAS project but I'd like to know what happens if you have two network cards bound together. Will that increase the throughput?
I'll add those measure for my next testing, thanks for the suggestion. I've found that the network utilization roughly translates to megabits (ie 90% = 900mb) which is why I've been using that number.
Your other suggestion isn't beyond my scope with this project. In other threads I was inquiring about the HP ProCurve switches with VLAN and trunking because I wanted to test exactly what you were asking about. I won't be buying that equipment for some time, so it'll be a while before I can actually test that. The OpenFiler software does allow for NIC teaming, so the possibility is there.
Your other suggestion isn't beyond my scope with this project. In other threads I was inquiring about the HP ProCurve switches with VLAN and trunking because I wanted to test exactly what you were asking about. I won't be buying that equipment for some time, so it'll be a while before I can actually test that. The OpenFiler software does allow for NIC teaming, so the possibility is there.
I tried your suggestion and attached an example. I tried copying a file from my NAS to my desktop.
For one sample of time (one second updates), there was a value of 114,252,631 bytes received with a measure of 90.63% utilization. The bytes converts to 108.96 MB/sec. 90.63% of 1000mb is 906.3 megabits/sec. That translates to 113.2875 MB/sec. I guess there is a discrepancy there between what I thought it worked out to and what it actually is. The part I'm likely missing is that the utilization is both network traffic up and down. If I take the 632,548 bytes that was sent up during that period, that works out to 0.60 MB. That still doesn't match the bytes per interval.
For one sample of time (one second updates), there was a value of 114,252,631 bytes received with a measure of 90.63% utilization. The bytes converts to 108.96 MB/sec. 90.63% of 1000mb is 906.3 megabits/sec. That translates to 113.2875 MB/sec. I guess there is a discrepancy there between what I thought it worked out to and what it actually is. The part I'm likely missing is that the utilization is both network traffic up and down. If I take the 632,548 bytes that was sent up during that period, that works out to 0.60 MB. That still doesn't match the bytes per interval.
Chewy509
Wotty wot wot.
IIRC, that figure includes the entire packet size, not just the payload... Take off 5-10% for headers and you should be close to actual throughput at the application level.
Also don't forget about other traffic at the same time, eg ARP, DNS, NTP, NETBIOS, etc..
I'll be buying a new network switch sooner than expected. My netgear ProSafe 8 port GS108 switch has begun to give me this flash of death if more than two ports are connected. Googling the issue shows that many other people have had the same issue...ggrrrr. I've had it for several years so it's out of warranty. Online people said it was a capacitor issue but I opened mine up and none of them are bulged.
I did check the power supply with my volt meter and its output is 12V but my volt meter was only showing 7.6V output. That could be the problem, but I don't know...
I did check the power supply with my volt meter and its output is 12V but my volt meter was only showing 7.6V output. That could be the problem, but I don't know...
ddrueding
Fixture
I have noticed more and more that cheap DC bricks are the weak link in small electronic devices. 50% of my Linksys router woes are because of this as well. I wish I knew of a decent manufacturer of quality replacements.
I read online that netgear will replace the power brick for $26...sure and if that doesn't fix this I just wasted $26 on their shitty product. I looked up when I bought this and it was 11/14/2006 from newegg for $72. I was hoping for more than 3.5 years of service, but that didn't pan out.
I'll just suck it up and buy a better switch. I originally thought my network card in my NAS had died, but I have everything connected into my D-Link (which is also a GigE switch) and everything is working fine.
I'll just suck it up and buy a better switch. I originally thought my network card in my NAS had died, but I have everything connected into my D-Link (which is also a GigE switch) and everything is working fine.
ddrueding
Fixture
That is the frustrating part; the brick isn't even their product! Sure, they chose to include it and (one hopes) are aware of the QC issues, but I've found that the QC of the product itself is much higher than the brick.
Sol
Storage is cool
I had the same problem with that switch (although I did have slightly bulging capacitors but only slightly) mine was just under 3 years old so I called up Netgear, sent them the switch (just by itself) and they sent me a new one still in the box (Total cost was about 5 pound for shipping). It's definitely worth calling them since it's entirely possible that the switch does have a problem in which case they'll probably replace the power supply along with the switch. (The warranty on the current pro-safe switches is lifetime but I thought even before that it was more than 3 years... 5 maybe...)
If you were in the UK you'd be welcome to the spare power supply I got when they replaced my switch, but I'm guessing it'd cost more than $26 to send it to you...
If you were in the UK you'd be welcome to the spare power supply I got when they replaced my switch, but I'm guessing it'd cost more than $26 to send it to you...
Thanks Sol, that's good info. I'll try giving them a call and see what my options are. I tried registering it online and there wasn't a lot of information except an option for $130 US to buy support on the phone which is laughable considering the switch is about $50 now. I'll call them next week and see what they can do (if anything).
Stereodude
Not really a
This is why you buy HP ProCurve products. 
Lifetime Warrantyoke:
Lifetime Warranty
oke:Stereodude
Not really a
Well, I didn't know about them 3.5 years ago either. I found out about them ~2.5 years ago when I was researching a 24 port partially managed gigabit switch for my house. At the time HP Procurve 1800-24G was the only fanless option plus it had the lifetime warranty (not sure if any others exist now).I didn't even know about ProCurve 3.5 years ago.
blakerwry
Storage? I am Storage!
The specs state it's only good up to 10,000 ft altitude... guess people on mountain peaks must forward their packets with non HP gear ;- )
blakerwry
Storage? I am Storage!
Seriously though, I appreciate the low wattage, fanless designs.
We currently have a stack of Cisco 48 port switches that we need to do something about. Unfortunately, Cicso switches that provide more than 48 ports don't really use any less power.
We currently have a stack of Cisco 48 port switches that we need to do something about. Unfortunately, Cicso switches that provide more than 48 ports don't really use any less power.
blakerwry
Storage? I am Storage!
Just complaining about the high power usage of the Cisco gear. At work we have three 48 port 10/100 switches providing the bulk of our desktop connectivity. Each switch is using an average of ~ 100 watts. Between the three switches and some misc equipment that's all the puny 450VA UPS they are attached to can handle reasonably.
After looking at it, the newer Cisco gear doesn't really use any more power than the HP stuff, we just have a couple aging 2948g switches which apparently draw twice as much power as something new (2950/2960).
I do still appreciate any fully solid state PC equipment. Kudos is in order to HP and any other vendor that can remove failure prone devices like fans.
After looking at it, the newer Cisco gear doesn't really use any more power than the HP stuff, we just have a couple aging 2948g switches which apparently draw twice as much power as something new (2950/2960).
I do still appreciate any fully solid state PC equipment. Kudos is in order to HP and any other vendor that can remove failure prone devices like fans.
Do your Cisco switches charge you licenses per port usage? ;-) Seems most of the fibre channel Cisco stuff we have charge per port which is crazy. The HP 1820G-24 looks to pull 30W which seems reasonable for my needs. I plan to keep it below 10,000 ft.
blakerwry
Storage? I am Storage!
per port? No, I agree that is crazy. I try not to buy into that type of licensing scheme (for any product).
We buy most of our network equipment second hand from a reliable reseller. The switches in question have been in place longer than I have. Other than a buggy version of CatOS on one of them (easily upgraded) they've been rock solid for several years with no downtime. Probably worth the 100 watt price tag - which I would estimate works out to be ~$100/year to operate.
We buy most of our network equipment second hand from a reliable reseller. The switches in question have been in place longer than I have. Other than a buggy version of CatOS on one of them (easily upgraded) they've been rock solid for several years with no downtime. Probably worth the 100 watt price tag - which I would estimate works out to be ~$100/year to operate.
blakerwry
Storage? I am Storage!
btw, I think pretty highly of HP switches. I've worked on some older HP stuff that had a (very) limited implementation of 802.1Q/VLANs. I'm sure their new stuff is better in that regard.
They've always had a good web interface and their reliability has been as good as any other.
I'm sure you'll be more than pleased.
They've always had a good web interface and their reliability has been as good as any other.
I'm sure you'll be more than pleased.
BingBangBop
Storage is cool
- Joined
- Nov 15, 2009
- Messages
- 667
They still all run on smoke: When the smoke leaks out, they die. I'll be even more impressed when they make electronics that doesn't depend on smoke running through everything.:rotfl:
No problem. It's actually good I did run them and have them recorded. I'm actually now having performance issues with my NAS and I can't figure out why. The array seems to be very slow and I'm noticing some of the LED activity lights on one of the drives stays on for long periods during writes which doesn't seem to reflect the nature of a striped array.
I pulled apart the array and disconnected all 5 drives to connect them to the normal SATA ports on the motherboard so I could review the SMART data for problems. All 5 of the drives show a warning by the C7 hex which is titled "UltraDMA CRC Error Count". There is a value between 1-3 on all the drives. My Google searches suggest that when this value has a warning there is a problem with the cabling. I'm concerned that maybe there is a problem with my Supermicro 5 drive rack. I'll need to mount the drives independently and test them for a while to see if the count increases during high amounts of IO.
I also received new cables in the mail today (from China) so I can test with those to see if the cables I bought are bad.
I pulled apart the array and disconnected all 5 drives to connect them to the normal SATA ports on the motherboard so I could review the SMART data for problems. All 5 of the drives show a warning by the C7 hex which is titled "UltraDMA CRC Error Count". There is a value between 1-3 on all the drives. My Google searches suggest that when this value has a warning there is a problem with the cabling. I'm concerned that maybe there is a problem with my Supermicro 5 drive rack. I'll need to mount the drives independently and test them for a while to see if the count increases during high amounts of IO.
I also received new cables in the mail today (from China) so I can test with those to see if the cables I bought are bad.
I did some testing with my array outside of the Supermicro drive rack. I mounted the drives in the case like normal and ran the cables directly to them. So far I'm seeing better performance and no weird slow down issues when writing to the array. I think maybe one of the ports or maybe the backplane is bad in the drive cage.
I went through amazon and they're already shipping me out an advanced replacement and gave me a prepaid return through UPS within 30 days. I'm really impressed that amazon pays for the returns without charging my credit card!? I'm so used to newegg and having to pay for my own return. I hope this fixes my issue.
sechs
Storage? I am Storage!
Get Prime. They'll overnight replacements to you.
I got my replacement Supermicro CSE-M35T 5 drive bay and I have the same performance problems with this new one.
During large writes to the array, the activity LED for one or more drives stays on longer and brighter than the rest of the drives. The write performance becomes terrible and file transfers crawl and are inconsistent. I did some basic large monolithic copy tests from my desktop to my NAS. After the file completes the LED activity lights continue to blink for a good 30 seconds after the transfer completed. I'm assuming this is due to the cache flushing to disk, but since the write access is terrible it's slow to destage.
To test if the drive bay is bad, I next turned off the NAS and removed all 5 drives from the drive bay. I then individually cabled the drives directly to the array and run power to them separately. I booted the NAS back up and copied over the same test files once again and the performance was back to normal. The write speeds are much improved but I have no activity LEDs to compare with.
There is something odd with these Supermicro drive bays that either do not like my drives or maybe the RAID controller. The last thought is that I got a second bad unit with the same problem.
I'm going to have to send this one back also since it doesn't work for my needs. I'll have to find another alternative because I was banking on using three of these. I read the instruction manual and there is no configuration for the drive bay that relates to the SATA bus. I'm out of ideas for this device.
During large writes to the array, the activity LED for one or more drives stays on longer and brighter than the rest of the drives. The write performance becomes terrible and file transfers crawl and are inconsistent. I did some basic large monolithic copy tests from my desktop to my NAS. After the file completes the LED activity lights continue to blink for a good 30 seconds after the transfer completed. I'm assuming this is due to the cache flushing to disk, but since the write access is terrible it's slow to destage.
To test if the drive bay is bad, I next turned off the NAS and removed all 5 drives from the drive bay. I then individually cabled the drives directly to the array and run power to them separately. I booted the NAS back up and copied over the same test files once again and the performance was back to normal. The write speeds are much improved but I have no activity LEDs to compare with.
There is something odd with these Supermicro drive bays that either do not like my drives or maybe the RAID controller. The last thought is that I got a second bad unit with the same problem.
I'm going to have to send this one back also since it doesn't work for my needs. I'll have to find another alternative because I was banking on using three of these. I read the instruction manual and there is no configuration for the drive bay that relates to the SATA bus. I'm out of ideas for this device.