Difference between revisions of "Backup"
(pedia backups) |
|||
| Line 5: | Line 5: | ||
The organic design backups are created daily by [[User:Bender]], compressed to .t7z and distributed over SSH to various other domains. After the upgrade to MediaWiki1.9.3, we are changing the way that [[wikid.pl]] works to integrate with MediaWiki as a proper extension on a per-wiki basis rather than a per-server basis. The backup system of [[OD2]] is designed to stay much more up-to-date and to allow simple restoration of backups into functioning wiki's again. This same mechanism will be used to create template-wiki's. | The organic design backups are created daily by [[User:Bender]], compressed to .t7z and distributed over SSH to various other domains. After the upgrade to MediaWiki1.9.3, we are changing the way that [[wikid.pl]] works to integrate with MediaWiki as a proper extension on a per-wiki basis rather than a per-server basis. The backup system of [[OD2]] is designed to stay much more up-to-date and to allow simple restoration of backups into functioning wiki's again. This same mechanism will be used to create template-wiki's. | ||
| − | == Simple backup == | + | == Simple workstation backup |
| − | The following [[backup.pl]] Perl script is a simple backup solution to backup all databases in a local MySQL server, and save them to a '' | + | The following [[backup-workstation.pl]] Perl script is a simple backup solution for a local workstation which compresses and 7-zip's a list of directory trees. |
| + | {{code|<perl>{{:backup-workstation.pl}}</perl>}} | ||
| + | |||
| + | == Simple server backup == | ||
| + | The following [[backup.pl]] Perl script is a simple backup solution to backup both a directory tree and all databases in a local MySQL server, and save them to a ''7-zip'' compressed file named according to the date. It also logs the backup and it's file size in [[Server log]] using [[MW:Extension:Simple Forms|SimpleForms]]. It can be executed periodically from the crontab. | ||
{{code|<perl>{{:backup.pl}}</perl>}} | {{code|<perl>{{:backup.pl}}</perl>}} | ||
Revision as of 22:53, 2 January 2008
Organic Design 2 | OD/Wikia | Wikia.php | Templates | Backup | Extensions
Backups in OD2
The organic design backups are created daily by User:Bender, compressed to .t7z and distributed over SSH to various other domains. After the upgrade to MediaWiki1.9.3, we are changing the way that wikid.pl works to integrate with MediaWiki as a proper extension on a per-wiki basis rather than a per-server basis. The backup system of OD2 is designed to stay much more up-to-date and to allow simple restoration of backups into functioning wiki's again. This same mechanism will be used to create template-wiki's.
== Simple workstation backup The following backup-workstation.pl Perl script is a simple backup solution for a local workstation which compresses and 7-zip's a list of directory trees.
Simple server backup
The following backup.pl Perl script is a simple backup solution to backup both a directory tree and all databases in a local MySQL server, and save them to a 7-zip compressed file named according to the date. It also logs the backup and it's file size in Server log using SimpleForms. It can be executed periodically from the crontab.
Up to date backup
MySQL 5.0 offers a new thing called replication which allows many slave databases to be kept syncronised with a single master. Even running the slaves locally is of benefit because regular backup and distribution can be made from the slaves so that the master never needs to be locked.
- 20 November 2006 - regarding changeover to LZMA (.t7z)
- 23 November 2006 - 7z server backups which can be unpacked the same as our template in Debian Conversion
- 20 December 2006 - backup corruption in transfer
Backup list
The following list of backup files are currently being managed:
- od-wiki-db
- od-server-image
- od-access-log
- peer-logs
- peerd-win32
- nad-org
- nad-kb
- zenia-org
- zenia-docs-settings
- fowin-dunkley
- fowin-luck
About LZMA
LZMA is an extremely good compression method which compresses our backups to about one third of the size of the gzip or bzip. I have tested it with the free 7-zip file manager from www.7-zip.org and od-wiki-db-2006-11-20 is 268MB uncompressed, 54.9MB gzipped and only 21.7MB as a 7z! But I'm unable to get the Debian port to work due to dependency issues with low level C libraries that I don't want to mess with.
- I've found a standalone version at http://sourceforge.net/projects/p7zip and that's compressed it to 24.8MB, not quite as small as the windows one, but still very good.
- Using switches -t7z -m0=lzma -mx=9 has got it down to 21.1MB - slightly smaller than the windows version :-)
Statistics
7Zip is extremely good at compressing wiki data compared to other algorthims, perhaps due to compressing the history more efficiently, here's a size comparison for compressing a server image which is a standard linux file structure containing no database or web site content.
| Compression | server image | wiki backup |
|---|---|---|
| none | 517MB | 269MB |
| 7z | 122MB (76%) | 21.1MB (92%) |
| RAR | 140MB (72%) | 24.9MB (90%) |
| Bzip2 | 176MB (66%) | 38.0MB (86%) |
| Gzip | 197MB (62%) | 54.5MB (80%) |
| Zip | 197MB (62%) | 54.5MB (80%) |
