Building a Storage Server Thread

[ddrueding]

Don't worry Merc. I missed the "a" after reading it three times... I couldn't figure out what was grammatically incorrect.

 

[Handruin]

:bstd:

 

[Bozo]

Merctio:
Good write up and you are right on. The only thing that I will add is the $$$$$.
We went with the Intel 865 chipset for one main reason. It could be had with built in video and was reasonably priced. Again, the money thing. That's another reason we are using 3Ware SATA controllers. Cost. Our largest server has 4-120Gb SATA hard drives in it (RAID 5). To get that kind of storage space with SCSI the price would have doubled at least.
As far as ECC memory goes, I used that in the SCSI servers as I had a little more money in the budget. Again, the $$$$$ thing.

BOZO :mrgrn:

 

[CityK]

In the realm of the wireless user, very little can be done aid performance, unfortunately.

Personally I would go with:

Unfortunately, in the realm of the wireless user, very little can be done to aid performance.

 

[ddrueding]

In the realm of the wireless user, very little can be done aid performance, unfortunately.

Personally I would go with:

Unfortunately, in the realm of the wireless user, very little can be done to aid performance.

I prefer:

If you are a wireless user, and are looking for performance, you're fu*cked.

 

[blakerwry]

wow, I decided to do my updating on the server pre core 2.... I found that transfers between the linksys NC100 and the SiS 900 NICs are absolutely fantastic.. I'm getting nearly 9MB/sec sustained traffic over the LAN. I'm sure that works out to be at least 8.5MB/sec for file transfers over SMB.

I guess the SiS 900 and the linksys were limited by the speed of the prev. NIC in my server.

I'm pretty impressed with these cheaper NICs. It would, however, be nice to have a gigabit card with a switch that allowed gigabit uplink... or better yet a straight gigabit switch.

Price is a big consideration however.

 

[blakerwry]

just clocked it, the 1GB file now transfers at 8,982,020 bytes/sec.

I also notice I'm not showing any errors. (before I was getting maybe 1 Tx Error per 100MB with the netgear)



I think this definitely shows the performance of NICs isn't necessarily related to their cost. (The 3com NICs being my most expensive, but nearly the slowest performing.)


Are there any good (and recent) speed comparisons of NICs so that we can choose the right one for the server (whether dual 100 or gigabit)

 

[Bozo]

Are there DOS drivers available for either the SIS or Linksys NICs?


Bozo :mrgrn:

 

[Mercutio]

There are for the Linksys.

 

[blakerwry]

The SiS has drivers that came with the motherboard. There is a DOS packet driver, lanman.dos, a win3.1 driver, netware 4, 5, and 6, win 9x/Me, and NT/2k/XP drivers.

The linksys NIC comes with drivers for SCO unix, instructions for getting the freeBSD 3.3 drivers along with liux, a DOS packet driver, lanman drivers for DOS and OS/2, win3.1, netware (3, 4, and 5) drivers, along with the standard win9x/Me and NT/2k/XP drivers.


Both have a diagnostic utility.

 

[blakerwry]

wow, I found a bug in the kernel... I experienced it with Fedora core 1... looks like it's been there since RedHat 7.2

ifconfig wraps the Tx/Rx count at 4GB because of the use of 32bit interger variables..

It is documented in bugzilla.redhat.com... aparently they chose to leave it in for compatibility/performance reasons. That's too bad.

Well, i just transfered in the range of 20 GB to my server without a single collision or error. Very cool.

 

[blakerwry]

wow, testing some other transfers, a transfer from winXP AthlonXP 2GHz w/ SiS900 NIC-> winXP Celeron 2.0GHz w/ realtek 8139 gets as high as 10MB/sec.

 

[blakerwry]

Oh, I should have added, that is through two switches over about 30 feet of CAT5e.

 

[Bozo]

Just finished assembling a file server with these parts

1-Intel BOXD865GBFL mb $149
1-Intel BX80532PG2600D cpu $249
2-Crucial CT6464Z40B $238
1-Antec SX1040BII case $135
4-WD1200JD hd $569
1-floppy drv $13
1-3Ware 8500-4 kit $379
1-Sony CD-ROM $39
Total $1392
I upgraded the power supply to a Antec True480 which added $109.
Add in Windows Server 2003 $485.60

Grand total $1996.60

If you shop at Newegg, you could knock 20% off the hardware prices. We must use an "authorized vendor".

360Gb of storage for under $2000. Hard to beat.

Bozo :mrgrn:

 

[Fushigi]

I don't know Intel & Crucial part #s. Can you detail what the CPU is and how much RAM you added?

How will the disks be managed? You mentioned 360 GB so I'm assuming RAID5. What are your partition plans?

 

[Bozo]

It's an Intel D865 chipset motherboard with a P4 at 2.6Ghz.
The 3Ware is set up in RAID 5
I have it partitioned with 10Gb for the operating system and programs, the rest is data storage.
There is 1Gb of RAM. (2-512meg sticks) DDR PC3200

Bozo :mrgrn:

 

[Mercutio]

... and $500 cheaper if you use *nix instead of 2003 Server. $1050 (drives + OS) buys 6 250GB Maxtor drives. Of course, that's less impressive on a 4-drive 3ware card...

 

[blakerwry]

It's an Intel D865 chipset motherboard with a P4 at 2.6Ghz.
The 3Ware is set up in RAID 5
I have it partitioned with 10Gb for the operating system and programs, the rest is data storage.
There is 1Gb of RAM. (2-512meg sticks) DDR PC3200

Bozo :mrgrn:

That looks less like a storage server and more like something built to handle multiple tasks. That CPU and RAM would be such a waste on a storage server.

One thing I notice about this board is the lack of real estate used. Looks like they could have just made it an M-ATX.


What will this server be used for? For a storage server I would have chopped $100 off the CPU and the RAM and put it towards larger hard drives or a controller with more channels(future upgradability, hot spare, etc). The price of the OS is a biggie as well, is that small business 2k3?

 

[Bozo]

The version of Windows server 2003 is 'Standard Edition'. MS SQL Server 2000 will also be installed to process data. That's the reason for Server 2003.
One thing I've learned in our plant, do it oversized as someone will want to add more task, programs, and data to it.
The hard drives can be replaced in 3 years (at the end of their warrenty) with larger ones which should cost less than these did. That's why I choose SATA. In three years what will the supply of PATA be like? I also wanted RAID 5.
As for memory.....it only cost an extra $119 to have a Gb. Not much in the whole scheme of things. And probably worth it down the road.
The 2.6 Ghz CPU might be overkill now, but a year from now it will pay off I'm sure.

Bozo :mrgrn:

 

[Handruin]

1GB ram is worth it for SQL server 2000. I would have sprung the extra $ for it, but for a storage server I may not have.

 

[Fushigi]

Hopefully the server will be rebooted on occasion. I've seen multiple servers where SQL2000 was eating 1.7GB RAM according to Task Mangler. This was after the server was left up for several weeks/months. I'm fairly sure it's got some memory leaks as rebooting brought it back down to something reasonable, like less than 10 times the size of the db it was managing. :lol:

 

[Handruin]

At work, we've seen the exact same thing in our small SQL server 2000 (sp3) setup. The machine hosting it is a dual PIII 1GHz, with 2 GB ram, but it crapped out the other day.

It was eating up roughly 1.7GB ram and the database is about 300 MB in size.

 

[blakerwry]

1GB ram is worth it for SQL server 2000. I would have sprung the extra $ for it, but for a storage server I may not have.

I completely agree.

 

[CityK]

I don't know if the document from JoJo's post covers it, but there was an interesting Exchange related thread on SR yesterday.

 

[Bozo]

The install date for this server has been pushed back, so I decided to install XP Pro on it just to see how it runs.

If I have time I might try Red Hat too.


Bozo :mrgrn:

 

[time]

At work, we've seen the exact same thing in our small SQL server 2000 (sp3) setup. The machine hosting it is a dual PIII 1GHz, with 2 GB ram, but it crapped out the other day.

It was eating up roughly 1.7GB ram and the database is about 300 MB in size.

I don't understand why people use the poxy thing. We've got Oracle here managing a 4GB database and using about 200MB of RAM. And it's fast. :-?

 

[Mercutio]

It's pretty easy to set up and manage. Per e_dawg, there are some idiots (not that e_dawg is an idiot) for whom the insta-upgrade between utterly-inadequate-for-anything-but-prototyping MS Access and SQL Server makes it the ONLY solution.

 

[Fushigi]

Application requirements. Otherwise, of course, I'd use DB2/400. :lol:

 

[blakerwry]

hmm, i noticed some odd behavior. I was testing a product called e-smith SMEserver 6.0 when I realized that my bandwidth between my computer<->Dlink switch<-> netgear switch<-> 3com 905 was a wopping 9MB/sec.. plug the server into the D-link switch so that there's one less hop and I get even better performance.


looks like linux is able to get alot more out of this NIC than windows (with either the MS or the 3com drivers).. linux calls this card "boomerang" if it makes any difference.

Maybe I will end up using these cards in my file server for transmit load balancing.

 

[CougTek]

It was eating up roughly 1.7GB ram and the database is about 300 MB in size.

I wonder how MySQL 5.0 would perform for the same task. Apparently, the very recently released 5.0 version is quite impressive and it's closing the gap with Oracle (I wrote "closing the gap", not "reached").

 

[Handruin]

Oh man... I try to promote MySQL all the time at work, but people have Oracle and SQL server stuck in their heads and consider MySQL to be a kids toy. It frustrates me, as I'm the hugest MySQL fan. (and I don't really know that much about SQL) Kinda ridiculous, I agree.

Not only is MySQL 5 susposed to be good, but MaxDB has many of the same features as Oracle and SQL server. (ie procedures, triggers, views...)

I run MySQL 4.0, apache, and php on my desktop and I'm kinda a fanatic about PHP at work. (which is sad at times) Everyone at work is so big into this C# with ASP.net kick that it makes me ill. I hate it, and I hate having to install a stupid framework to run applications...bah I'm not saying it's that bad, I just don't like how it gets priority over a potential alternative such as PHP...everyone looks at me funny when I recommend it...I always here..."WHat is PHP?"

Anyway, the application we run requires SQL server, so we're stuck with it. :roll:

To answer your question, I would suspect MySQL 5 could handle the job...if the application supported it. I'm hoping to create a small web based tool at work to prove to people that php and mysql can handle the task...

 

[CougTek]

I always here..."What is PHP?"

Tut, tut, tut,

Doug, Doug, Doug. Installed a spell checking feature but misspelled a four letter word. Incorrigible.

I always HEAR...

 

[ddrueding]

Yopu know, this ASP.net + SQL server + web services this is pretty cool...

I'm currently in the middle of a half-dozen projects using this combination.

 

[Handruin]

:sqnt:
I was on a rampage to get that message out before bed... I have no excuse. I suck at spelling. I do know the difference, but that time it slipped out.

 

[time]

MySQL is only competitive with single table queries - which rarely happen in real life.

There's an awful lot of BS floating around about MySQL, much of it initiated by MySQL AB. Basically, it's ideal as a back end for a web server ... and that's about it. It is slow at any sort of realistic query, collapses under load, and is poor at handling updates.

More importantly, it just lacks the ability to perform any kind of operation that goes much beyond a simple join. I've seen its SQL support described as a much less capable version of that available in Microsoft Access - hardly a ringing endorsement.

To be fair, the trend is to move business rules into the middleware, which takes a lot of pressure off the back-end. But I haven't heard of many people trying to build 'real' n-tier products with MySQL; it's just too primitive.

MaxDB is not MySQL. It is a completely separate product that MySQL AB has gained rights to. It is also charged per seat, albeit at a rate about one quarter that of Oracle. And the support costs would make Oracle Corp blush.

I'm cynical enough to suggest that its performance is likely to be proportionate to its cost, when compared to Oracle or DB2. And I've no doubt that development costs will be higher.

A great deal of public support for MySQL is badly misguided. There are other open source solutions that are far, far more competent and offer competitive performance. PostgreSQL and Firebird spring to mind immediately. The MySQL bandwagon is sucking interest away from these worthy products, and in the event of a MySQL AB trainwreck, will probably tarnish the whole idea of seeking alternatives to Oracle and MS SQL Server.

 

[blakerwry]

anymore updates?

I was surprised to see the following out of my 100baseT network today

 

[ddrueding]

Holy cow...what did you do to manage that, blake?

 

[Buck]

So, what type of system would you build to run MS SQL Server? I have a customer that is looking for a dual P4 system with 1GB of RAM and SCSI disks for this type of operation. I don't know anything about the needs or requirements for anything SQL, but he does - he works for a company that does database development. However, he could still be off with the hardware side of things.

 

[Fushigi]

So, what type of system would you build to run MS SQL Server? I have a customer that is looking for a dual P4 system with 1GB of RAM and SCSI disks for this type of operation. I don't know anything about the needs or requirements for anything SQL, but he does - he works for a company that does database development. However, he could still be off with the hardware side of things.

Our standard server is a Dell PE2650 with dual Xeons, 2GB RAM, and 4 36GB SCSI disks. We see SQL Server occasionally developing a memory leak or something. When it happens, RAM allocation jumps to 1.7GB so if the budget allows, some additional system RAM might be good.

Otherwise, to refine the spec further we'd need to know the transaction load, db size, dedicated vs. shared, etc.

 

[Buck]

I just went through Dell's online configurator, speced out the following:

Dual Xeon 3.06GHz
2GB Memory
4 SCSI Channels
5 HDD ((2) 73GB (3) 146GB)
Dual Gigabit ports

. . . and the price came out to $9,000.00. Well, this makes me feel much better about my initial estimates, which came out lower, and with 4 times as much memory.