Difference between revisions of "Backup"
m |
(Asyncronous replication in MySQL5) |
||
| Line 1: | Line 1: | ||
[[Category:IT Support]][[Category:Nad]][[Category:Organic Design]] | [[Category:IT Support]][[Category:Nad]][[Category:Organic Design]] | ||
__NOTOC__ | __NOTOC__ | ||
| − | The organic design backups are created daily by [[User:Bender]], compressed to . | + | == Backups in OD2 == |
| + | The organic design backups are created daily by [[User:Bender]], compressed to .t7z and distributed over SSH to various other domains. After the upgrade to MediaWiki1.9.3, we are changing the way that [[wikid.pl]] works to integrate with MediaWiki as a proper extension on a per-wiki basis rather than a per-server basis. The backup system of [[OD2]] is designed to stay much more up-to-date and to allow simple restoration of backups into functioning wiki's again. This same mechanism will be used to create template-wiki's. | ||
| − | = Backup related news items = | + | == Up to date backup == |
| + | MySQL 5.0 offers a new thing called [http://dev.mysql.com/doc/refman/5.0/en/replication-implementation.html replication] which allows many ''slave'' databases to be kept syncronised with a single ''master''. Even running the slaves locally is of benefit because regular backup and distribution can be made from the slaves so that the master never needs to be locked. | ||
| + | |||
| + | == Backup related news items == | ||
*[[20 November 2006]] ''- regarding changeover to LZMA (.t7z)'' | *[[20 November 2006]] ''- regarding changeover to LZMA (.t7z)'' | ||
*[[23 November 2006]] ''- 7z server backups which can be unpacked the same as our template in [[Debian Conversion]]'' | *[[23 November 2006]] ''- 7z server backups which can be unpacked the same as our template in [[Debian Conversion]]'' | ||
*[{{SERVER}}/wiki/index.php?title=20_December_2006&oldid=44019 20 December 2006] ''- backup corruption in transfer'' | *[{{SERVER}}/wiki/index.php?title=20_December_2006&oldid=44019 20 December 2006] ''- backup corruption in transfer'' | ||
| − | = Backup list = | + | == Backup list == |
The following list of backup files are currently being managed: | The following list of backup files are currently being managed: | ||
*od-wiki-db | *od-wiki-db | ||
| Line 22: | Line 26: | ||
*fowin-luck | *fowin-luck | ||
| − | = About LZMA = | + | == About LZMA == |
[[w:LZMA|LZMA]] is an extremely good compression method which compresses our backups to about one third of the size of the gzip or bzip. I have tested it with the free 7-zip file manager from [http://www.7-zip.org www.7-zip.org] and <tt>od-wiki-db-2006-11-20</tt> is 268MB uncompressed, 54.9MB gzipped and only 21.7MB as a 7z! But I'm unable to get the Debian port to work due to dependency issues with low level C libraries that I don't want to mess with. | [[w:LZMA|LZMA]] is an extremely good compression method which compresses our backups to about one third of the size of the gzip or bzip. I have tested it with the free 7-zip file manager from [http://www.7-zip.org www.7-zip.org] and <tt>od-wiki-db-2006-11-20</tt> is 268MB uncompressed, 54.9MB gzipped and only 21.7MB as a 7z! But I'm unable to get the Debian port to work due to dependency issues with low level C libraries that I don't want to mess with. | ||
*I've found a standalone version at http://sourceforge.net/projects/p7zip and that's compressed it to 24.8MB, not quite as small as the windows one, but still very good. | *I've found a standalone version at http://sourceforge.net/projects/p7zip and that's compressed it to 24.8MB, not quite as small as the windows one, but still very good. | ||
*Using switches <tt>-t7z -m0=lzma -mx=9</tt> has got it down to 21.1MB - slightly smaller than the windows version :-) | *Using switches <tt>-t7z -m0=lzma -mx=9</tt> has got it down to 21.1MB - slightly smaller than the windows version :-) | ||
| − | = Statistics = | + | == Statistics == |
7Zip is extremely good at compressing wiki data compared to other algorthims, perhaps due to compressing the history more efficiently, here's a size comparison for compressing a server image which is a standard linux file structure containing no database or web site content. | 7Zip is extremely good at compressing wiki data compared to other algorthims, perhaps due to compressing the history more efficiently, here's a size comparison for compressing a server image which is a standard linux file structure containing no database or web site content. | ||
<table border cellspacing=0 cellpadding=5> | <table border cellspacing=0 cellpadding=5> | ||
Revision as of 00:10, 30 April 2007
Backups in OD2
The organic design backups are created daily by User:Bender, compressed to .t7z and distributed over SSH to various other domains. After the upgrade to MediaWiki1.9.3, we are changing the way that wikid.pl works to integrate with MediaWiki as a proper extension on a per-wiki basis rather than a per-server basis. The backup system of OD2 is designed to stay much more up-to-date and to allow simple restoration of backups into functioning wiki's again. This same mechanism will be used to create template-wiki's.
Up to date backup
MySQL 5.0 offers a new thing called replication which allows many slave databases to be kept syncronised with a single master. Even running the slaves locally is of benefit because regular backup and distribution can be made from the slaves so that the master never needs to be locked.
- 20 November 2006 - regarding changeover to LZMA (.t7z)
- 23 November 2006 - 7z server backups which can be unpacked the same as our template in Debian Conversion
- 20 December 2006 - backup corruption in transfer
Backup list
The following list of backup files are currently being managed:
- od-wiki-db
- od-server-image
- od-access-log
- peer-logs
- peerd-win32
- nad-org
- nad-kb
- zenia-org
- zenia-docs-settings
- fowin-dunkley
- fowin-luck
About LZMA
LZMA is an extremely good compression method which compresses our backups to about one third of the size of the gzip or bzip. I have tested it with the free 7-zip file manager from www.7-zip.org and od-wiki-db-2006-11-20 is 268MB uncompressed, 54.9MB gzipped and only 21.7MB as a 7z! But I'm unable to get the Debian port to work due to dependency issues with low level C libraries that I don't want to mess with.
- I've found a standalone version at http://sourceforge.net/projects/p7zip and that's compressed it to 24.8MB, not quite as small as the windows one, but still very good.
- Using switches -t7z -m0=lzma -mx=9 has got it down to 21.1MB - slightly smaller than the windows version :-)
Statistics
7Zip is extremely good at compressing wiki data compared to other algorthims, perhaps due to compressing the history more efficiently, here's a size comparison for compressing a server image which is a standard linux file structure containing no database or web site content.
| Compression | server image | wiki backup |
|---|---|---|
| none | 517MB | 269MB |
| 7z | 122MB (76%) | 21.1MB (92%) |
| RAR | 140MB (72%) | 24.9MB (90%) |
| Bzip2 | 176MB (66%) | 38.0MB (86%) |
| Gzip | 197MB (62%) | 54.5MB (80%) |
| Zip | 197MB (62%) | 54.5MB (80%) |
