Why is the site always slow?

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
We can start discussing ways to make it easier for Doug and faster for all of us.

Require registration to start with an e-mail to a mod?
Colo a dedicated server?

I'm assuming the bulk of the admin duties are dealing with SPAMers?
This place is important to me. I would happily send $10-25/mo to make it better, and could donate a rather beastly machine for the purpose.

I looked into colocation many times but the cost of the service was the issue, not the hardware, though I do appreciate the offer to donate a beastly machine. If we can find a reasonable colocation area, maybe we can build a dedicated machine through donations or something?

Spammers are an issue and I've been trying to find better ways to reduce them. There are improvements that can be made. I spent a few hours last Saturday troubleshooting some code, but I made no changes or progress yet. I have a list of things I need to change/implement to make it a little better.

This topic should move to its own thread for better visibility.

If there is anything we can do to help, please let us know.

What are the hardware requirements for a web site server?

It really isn't practical or affordable for us to use our own hardware. Rack space at an ISP ain't cheap and they may not allow anyone onsite to maintain the hardware.


David also offered hardware, but like time mentioned, it's not the hardware or cost of hardware, it's absolutely the hosting which is expensive. I did look up some colocation hosting in Mass last night and it looked to cost around $150/month for bare minimal space (1U) and very limited bandwidth (less than 5Mb/sec).
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
What about hosting at home on your FIOS?

I explored that at one time, but my concern would be reliability. I recently had an outage when the cable fell off the poll some how and it took FIOS about a week to get it fixed. This isn't a normal thing, but an example of how long the site could be down. I can investigate this option again. I expect it would cost a bit more than paying for hosting, but if I applied what I pay now into my internet bill, it might only be slightly more.
 

Stereodude

Not really a
Joined
Jan 22, 2002
Messages
10,865
Location
Michigan
I'd go for the don't ask don't tell policy. Just start hosting with your current plan and let them complain to you if they actually notice. Then you can play dumb, apologize, and pay the extra.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
I'd go for the don't ask don't tell policy. Just start hosting with your current plan and let them complain to you if they actually notice. Then you can play dumb, apologize, and pay the extra.

Since my current plan is DHCP enabled, what's the best way to manage the domain name updates in a case like this? I'm currently doing this with a dynDNS service so I know it works fine, but it's a made up DNS name. I can give it a try, I don't think we consume that much bandwidth if there is a way for DNS to be updated when DHCP changes.
 

Howell

Storage? I am Storage!
Joined
Feb 24, 2003
Messages
4,740
Location
Chattanooga, TN
Since my current plan is DHCP enabled, what's the best way to manage the domain name updates in a case like this? I'm currently doing this with a dynDNS service so I know it works fine, but it's a made up DNS name. I can give it a try, I don't think we consume that much bandwidth if there is a way for DNS to be updated when DHCP changes.

Custom DYNDNS supports updating from an agent.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
$30/year seems reasonable for this service. Maybe I'll give it a try with a temporary domain name that you guys can try accessing and see if the performance will be reasonable. I have a dedicated machine I can use for this (at least a little while until I figure out a better setup).

I'll need to start a new post regarding that once I have something setup. I'll see if I can get this going Sunday.
 

Howell

Storage? I am Storage!
Joined
Feb 24, 2003
Messages
4,740
Location
Chattanooga, TN
$30/year seems reasonable for this service. Maybe I'll give it a try with a temporary domain name that you guys can try accessing and see if the performance will be reasonable. I have a dedicated machine I can use for this (at least a little while until I figure out a better setup).

I'll need to start a new post regarding that once I have something setup. I'll see if I can get this going Sunday.

You can go ahead and set everything up at dyn and just not change dns at your registrar yet.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
You can go ahead and set everything up at dyn and just not change dns at your registrar yet.

Sure, but then how do we test this? Just use one of their dynamic names for now or give out an IP?
 

Stereodude

Not really a
Joined
Jan 22, 2002
Messages
10,865
Location
Michigan
That sound like too much trouble...

FWIW, the one domain I have is registered with NameCheap. They allow updating the IP dynamically as part of the cost of the domain (~$10 year). My router supports their update "protocol" so it updates the IP (when it rarely changes). So, depending on who you have storageforum.net registered with you may not have to pay anything extra.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
That sound like too much trouble...

FWIW, the one domain I have is registered with NameCheap. They allow updating the IP dynamically as part of the cost of the domain. My router supports their update "protocol" so it updates the IP (when it rarely changes). So, depending on who you have storageforum.net registered with you may not have to pay anything extra.

I'll take a look into that also, thanks for the idea. I can always move my domain name to NameCheap if that option will work for me. I'll have to see if it's possible to update using my router (the one you hate).
 

Howell

Storage? I am Storage!
Joined
Feb 24, 2003
Messages
4,740
Location
Chattanooga, TN
I'll take a look into that also, thanks for the idea. I can always move my domain name to NameCheap if that option will work for me. I'll have to see if it's possible to update using my router (the one you hate).

Many of the dynamic dns services have agents that run on a machine behind the router.
 

Stereodude

Not really a
Joined
Jan 22, 2002
Messages
10,865
Location
Michigan
I'll take a look into that also, thanks for the idea. I can always move my domain name to NameCheap if that option will work for me. I'll have to see if it's possible to update using my router (the one you hate).
Heh... I know namecheap isn't the only domain registrar that supports dynamic DNS either so there should be other options. I don't think the D-Link supports namecheap in their firmware though. At least in the case of namecheap you can also have the server run an agent program that updates the IP also. But using a program on a PC is a little more complicated since it doesn't know when the IP changes (whereas the router does) so you end up scheduling it to update on some periodic interval.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
Many of the dynamic dns services have agents that run on a machine behind the router.

Heh... I know namecheap isn't the only domain registrar that supports dynamic DNS either so there should be other options. I don't think the D-Link supports namecheap in their firmware though. At least in the case of namecheap you can also have the server run a program that updates the IP also.

Hopefully they have a Linux agent/software (I'll look into this more). :) I would opt for a Linux machine since it'll be exposed to the internet.
 

timwhit

Hairy Aussie
Joined
Jan 23, 2002
Messages
5,278
Location
Chicago, IL
Hopefully they have a Linux agent/software (I'll look into this more). :) I would opt for a Linux machine since it'll be exposed to the internet.

DynDns has a Linux agent, but it's a pain in the ass to get working. I just run the Windows agent on a Windows machine that's on the same network.
 

Mercutio

Fatwah on Western Digital
Joined
Jan 17, 2002
Messages
22,275
Location
I am omnipresent
Wow. I missed a lot of thread.

It goes without saying that SF means a lot to me and I'd be willing to bear out the costs for keeping it going. I do have internet service that explicitly allows hosting, though at the moment I'm not paying for a static IP and I have a tendency to redline the upsteam bandwidth. Still, it's an option.

At one time, phpBB had a very poor reputation for security. Have thing really gotten better in the last few years?
 

ddrueding

Fixture
Joined
Feb 4, 2002
Messages
19,728
Location
Horsens, Denmark
For less than the price of colo we could manage another business-class internet connection at your place, and even manage some redundancy.

If we were able to take the hosting cost and responsibility off Doug, and some of the others can help him with the admin stuff, and make the site stupid fast at the same time, I think this would be better for all.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
Wow. I missed a lot of thread.

It goes without saying that SF means a lot to me and I'd be willing to bear out the costs for keeping it going. I do have internet service that explicitly allows hosting, though at the moment I'm not paying for a static IP and I have a tendency to redline the upsteam bandwidth. Still, it's an option.

At one time, phpBB had a very poor reputation for security. Have thing really gotten better in the last few years?

For less than the price of colo we could manage another business-class internet connection at your place, and even manage some redundancy.

If we were able to take the hosting cost and responsibility off Doug, and some of the others can help him with the admin stuff, and make the site stupid fast at the same time, I think this would be better for all.

I have a thought...and it's something you (David) mentioned some time ago. What if we coordinate an environment that is mirrored in two places. I can try the plan to host it here via FIOS. I might (but not sure) get more upstream than mercutio with my 25/25 Mb internet and I also don't torrent even a fraction of what he does.

Mercutio can continue as normal, but perhaps have (if willing) a system (maybe even a VM) that allows mysql to use as a remote and we coordinate synchronizing the files between the two. The static files don't change very often and at most it would be missing the uploads until it finished synchronizing. If my ISP goes down and blocks me, Merc's takes over in the mean time.

This is what my ISP is at right now from MA to NH:


From MA to CA:


From MA to Melbourne:


From MA to Toronto:
 

ddrueding

Fixture
Joined
Feb 4, 2002
Messages
19,728
Location
Horsens, Denmark
Sounds good to me, of course it doesn't involve much effort on my part ;)

I also lack the skills to do much of anything in this scenario. My contribution may just be financial/hardware.
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
I'll see if I can get two VMs up and running and if it works, the remote VM could in theory be given to anyone to host. I was was going to try using a bare centOS 5.5 x_64 configuration to start with.
 

timwhit

Hairy Aussie
Joined
Jan 23, 2002
Messages
5,278
Location
Chicago, IL
The static files don't change very often and at most it would be missing the uploads until it finished synchronizing. If my ISP goes down and blocks me, Merc's takes over in the mean time.

You could also synchronize the static files fairly regularly in a couple ways.

1. Use rsync to copy deltas on a schedule.
2. Setup a version control system and commit any new/changed files to this on a regular basis (crontab). The standby system could then poll the VCS (also crontab) on a regular basis to pull down the new files. I like this option because everything is backed up and versioned. git, svn, or cvs would work well for this.
 

timwhit

Hairy Aussie
Joined
Jan 23, 2002
Messages
5,278
Location
Chicago, IL
Mercutio can continue as normal, but perhaps have (if willing) a system (maybe even a VM) that allows mysql to use as a remote and we coordinate synchronizing the files between the two.

Would you plan to use MySQL replication for this?
 

Handruin

Administrator
Joined
Jan 13, 2002
Messages
13,926
Location
USA
There was a note from the hosting service regarding an issue. Sorry I wasn't able to post before now.
 

Santilli

Hairy Aussie
Joined
Jan 27, 2002
Messages
5,278
Why would going from 20mbps to 80 mbps make a huge difference in perception about how fast this site loads?
 

LunarMist

I can't believe I'm a Fixture
Joined
Feb 1, 2003
Messages
17,497
Location
USA
Why would going from 20mbps to 80 mbps make a huge difference in perception about how fast this site loads?

Because it is your perception, not reality. As well the site is running decently at the moment, but we know that it varies quite a lot.
 

Santilli

Hairy Aussie
Joined
Jan 27, 2002
Messages
5,278
Might also be because the path is very clean, with double digit only ms.
tracertstorageforumnet-1.jpg
 

LunarMist

I can't believe I'm a Fixture
Joined
Feb 1, 2003
Messages
17,497
Location
USA
None of mine are near 100 either, and I probably have about the slowest sustained internet speed of anyone at SF. None of that matters when the website is slow. It doesn't exactly take huge bandwidth to download some paragraphs of text and a few small graphics quickly when the site is running well as it is now.
 
Top