Forum backups

[Tea]

Within the next week or two, I expect to have a cable system up and running and connected to my office network (see this thread for details - and also if you want to help me set it up right). That will give me:

1: A cable link in the 512k class. (They no longer cap the cable link speeds on these new deals - you are limited only by the speed of the network and your ability to pay the download charges - but expecting more than 512k out of it would be foolish.)

2: Dual 2.5GB per month bandwidth allowances before I have to pay extra cents per MB - dual 2.5GB because I'm getting it on both at home and at the office, so 5GB in total.

3: A static IP (which costs extra, so I'm only doing that at the office).

4: Unlimited storage space. (Well, limited by my ability to buy enough hard drives, but that should be space enough.)

5: Unlimited free downloads off-peak - that's between 1:00AM and 7:00AM my time. I can download as much as the cable will carry between those hours and it doesn't count towards my 2.5GB limits.

So: what say I set up a machine in the DMZ (which will probably also function as my mail server and possibly as a www or FTP server) with a nice big hard drive, hand Doug the keys to my firewall, and we get a proper off-site backup happening? Kristi and I can make an emergency CD backup of Doug's backups from time to time - once a week, let's say, as there two or three burners attached to the network.

Worth doing?

 

[Buck]

What is our present backup solution?

 

[Clocker]

Tannin-

Most cable ISPs prohibit running servers in the AUP. You mihgt check that before you set things up....

C

 

[Tea]

AUP?

I figure that I won't be generating that much traffic and that what they don't know won't hurt them. My idea is not to switch my main site over, just to provide myself with lots of space for things like pictures of hard drives for Buck to drool over. And for Doug to push a copy of the forum database onto, of course.

 

[Mercutio]

Clocker, if Tannin does his thing from his business cable connection, he can do whatever the hell he wants, up to and including re-selling his bandwidth.

And usually no one care unless either 1.) His ISP is doing portscans looking for AUP violations (I'd think hitting the kiddies using Kazaa would be a higher priority) or 2.) Bored admin deciding to be a jerk or 3.) Tannin actually does something to cause a problem

 

[Buck]

My ISP did close down port 80, so my old websites went down for a bit until I switched them over to something else, like 8080 or 8085.

 

[The JoJo]

With a small amount of traffic, running a webserver or other services, should be no problem and not draw any attention to you.

Can't think of anyone here not running some services from their home computers...
And as far as I have heard atleast here in Finland this has not been a problem.

Just log on to www.dyndns.org or www.no-ip.com so that you always know how to connect to your home computer.

 

[Handruin]

What is our present backup solution?

Present backup solution is nightly around 12:00 AM ESD. Both the phpBB2 and xoops databases are backed up and time-stamped via script that Mercutio help me with. (thank you again)

I then download the gzip'ed files onto one of my hard drives (atlas 10KIII) where I have SF site files organized. I then zip the entire directory and copy it to my other hard drive (barracuda 18XL). This zip is also copied to my CD-RW giving me 3 locations with the same data.

The HTML files for the current working forum are also backedup, but since these files rarely change, I don't download them nightly. Xoops is the same way, since it is a "package" system, I don't download them, however I have a copy on my hard drive.

 

[Buck]

So, would it be worth setting up a new backup location in Ballarat? If the entire process could be automated (zips and multiple copies), why not? Perhaps one of the copies made by Tony gets sent to Handruin at 12:00pm?

 

[timwhit]

It would be worth it if the unthinkable happened and something bad happened to Doug, or in case of nuclear war (I doubt Ballarat would be a prime target).

I don't see how it could hurt anything.

 

[Jake the Dog]

tannin,

obviously you've either gone with tel$tra or optus but your package sounds more like an optus one. is this correct? if so, make sure your AUP stipulates that you are allowed to run the type of web services you want to or otherwise make sure you run everything on non-standard ports. optus can be pretty ruthless when one breaches their AUP but other then that, they are generally way better then tel$tra!

 

[Tea]

1: Nope. It's cable, not ADSL. Neighborhood cable.

2: Doug already has an excellent backup system (as he outlined just now). I see no reason at all to stop that system, which is practical and convenient. Besides, I'd rather trust Doug to remember to burn CDRs than me! (Really. I forget things.) (Often.) But I'm thinking about the "Doug falls under a bus" scenario. (Sorry Doug - hey, we all know that if you wer to meet with a nasty traffic accident, it wouldn't be a bus. At least a limo, probably a Ferrari.) Or the "Doug's flat gets burned down and his computer and his backup CDs melted" scenario. Stuff like that. I'd imagine that Doug and Mercutio would make a modification to his script so that it sends an extra copy of the DB over to my box at, say, 5:00AM Australian time.

3: WTF is an AUL. (OK, I can guess it's some kind of relative of an EULA, but what does it stand for?)

4: How do you spell "neighbourhood"?

 

[Tea]

5: Atilla the Hun is generally better than Telstra.

 

[Buck]

4: How do you spell "neighbourhood"?

Where is the Grammar Police?

 

[timwhit]

4: How do you spell "neighbourhood"?

Where is the Grammar Police?

Where are the Grammar Police.

 

[The Grammar Police]

"Is"? Or "are"? I'm going to have to think about that. Either way, we don't do spelling. Which is just as well.

PS: You can always tell Tony's sock puppets: perfect grammar and abominable spelling. Why doesn't the fool do something useful, like dream up an alias that is the opposite way round? I could soon sort the alias' grammar out - do that in my sleep -and he could fix the spelling for me, Tea, Ed, Dave and (of course) Tannin himself.

 

[Buck]

It should read:

"Where is The Grammar Police?"

The name of this forum contributor is: The Grammar Police. Just like we would say, "Where is Tea", or, "Where is Tony Wilson", we would say, "Where is The Grammar Police".

 

[James]

An AUP is an Acceptable Usage Policy. Make sure you read yours carefully!

 

[Handruin]

1: Nope. It's cable, not ADSL. Neighborhood cable.

2: Doug already has an excellent backup system (as he outlined just now). I see no reason at all to stop that system, which is practical and convenient. Besides, I'd rather trust Doug to remember to burn CDRs than me! (Really. I forget things.) (Often.) But I'm thinking about the "Doug falls under a bus" scenario. (Sorry Doug - hey, we all know that if you wer to meet with a nasty traffic accident, it wouldn't be a bus. At least a limo, probably a Ferrari.) Or the "Doug's flat gets burned down and his computer and his backup CDs melted" scenario. Stuff like that. I'd imagine that Doug and Mercutio would make a modification to his script so that it sends an extra copy of the DB over to my box at, say, 5:00AM Australian time.

3: WTF is an AUL. (OK, I can guess it's some kind of relative of an EULA, but what does it stand for?)

4: How do you spell "neighbourhood"?

There is always the possibility of something bad happening, so if you can setup an FTP server, and no one objects, I can ship over backups on a regular basis if you want. I still need Mercutio’s help with the automation of the script. I run it manually every night, but the original idea was to set it up as a cron job and have the 2-3 latest copies available for download. I was never able to get that part of the script working.

This is what I have so far:

#!/bin/bash

#----------------------------------------------------------------------------
# Your basic variable assignments
#----------------------------------------------------------------------------


dougsbackupdir=../sqldump
dougsemail=webmaster@storageforum.net
todaysdate1=`date -I`
todaysdate2=`date +-%H%M`
todaysdate=$todaysdate1$todaysdate2
todaysfname1=phpbb2$todaysdate.sql
todaysfname2=xoops$todaysdate.sql

cd $dougsbackupdir

# Datebase backup to a date-named file

mysqldump –umyusername -pmypassword handruin_phpBB > $todaysfname1

# gzipping, had to add nice because server kills the script

nice -n 19 gzip -9 $todaysfname1

# Datebase backup to a date-named file

mysqldump –uxoopsusername -pxoopspassword handruin_xoops > $todaysfname2

# gzipping

nice -n 19 gzip -9 $todaysfname2

# todaysfname1=$todaysfname1.gz
# todaysfname2=$todaysfname2.gz

# cp $todaysfname MONTHLY_BACKUP_$todaysfname


# fi

#----------------------------------------------------------------------------
# Here\'s the dramatically easier backup/deletion routine.
# We pop the name of the day\'s backup on the end of a file, read the file name
# at the top of the file, delete that file, copy the remaining lines to a
# temp file, and overwrite the previous day\'s list.
#----------------------------------------------------------------------------

# echo $todaysfname >> backuplist

# numberdaysbackup=`wc -l backuplist`

# if [ \"$numberdaysbackup\" -ge 6 ]
# then

# rmdate=`head -n 1 backuplist`

# rm $rmdate

# tail -n 5 backuplist > bul

# mv bul backuplist
# fi

 

[timwhit]

But the Grammar Polices' location says "We are everywhere!" Which implies that there are more than one of them. I still think it should be 'are'.

It would be like saying where is the police? Or Where is the Chicago Police? Neither is grammatically correct. Where is Police officer Poncho when you need him? The last one is corrent because Police officer Poncho is one person.

 

[Cliptin]

But the Grammar Polices' location says "We are everywhere!" Which implies that there are more than one of them. I still think it should be 'are'.

It would be like saying where is the police? Or Where is the Chicago Police? Neither is grammatically correct. Where is Police officer Poncho when you need him? The last one is corrent because Police officer Poncho is one person.

Leave it to The Grammer Police to be grammaticly inconsistent. OTOH, If The Giver can consistently speak in the third person I see no reason why The Grammer Police can't consistently speak in plural.

 

[Tea]

Ahh, thankyou James. Well, I had a good careful look at the fine print, and so far as I can tell, there is no barrier at all to using my bandwidth as I please. Cool!

 

[Handruin]

Tea, if you are still willing to save a backup of SF, I can begin shipment to your FTP server once you give the OK, provided you are still interested. I have 353 backups, totaling 503 MB of space. Would you like to arrange something? I will also make a reminder to create a more detailed disaster recovery plan for the bus scenario. To whom should something like this go?

My upload speed sucks (about 100 Kb/ps, yes little b), so I can send these files in smaller packages over a few days.

 

[Tea]

My office cable hook-up is not done yet, Doug. I hope to get it next week sometime. Also, I have to build a Smoothwall box (easy), run some new network cables around the place (tedious but easy) and set up a box in the orange zone running an FTP server (or whatever system works out to be most convenient to you and to me). (Will no doubt require some skull-sweat.) Should be done by the end of next week (earliest) or the end of the following week (latest). Once I've got the setup setup (so to speak), we can start slipping forum backups onto it.

Now, as you say, having a backup is one thing, having the knowlege of what to do with it should a restore ever prove necessary is another thing entirely. I am inclined to think that my role in this should be simply to provide the backup hardware and keep the data safe. Hopefully another member - maybe Mercutio or James or P5 - can step forward and make sure that they have sufficient knowledge of the setup requirements such that in the case of dire emergency of the bus flavour they can take my backups and recreate the SF site from scratch.