Feature Proposals/AutoBackup - Revision history

Justincc: commercial label is irrelevant

2014-07-08T16:15:21Z

commercial label is irrelevant

Justincc: /* Basic Info */

2014-07-08T16:14:57Z

‎Basic Info

MakoBot: Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name

2012-03-04T06:23:34Z

Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name

MakoBot: Robot: Cosmetic changes

2012-03-04T03:02:09Z

Robot: Cosmetic changes

MakoBot: bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

2011-06-06T14:02:26Z

bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

Makopoppo: Reverted edits by MakoBot (Talk) to last version by Makopoppo

2011-06-06T14:02:11Z

Reverted edits by MakoBot (Talk) to last version by Makopoppo

MakoBot: bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

2011-06-06T13:59:23Z

bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

Makopoppo: Reverted edits by MakoBot (Talk) to last version by Allquixotic

2011-06-06T13:58:47Z

Reverted edits by MakoBot (Talk) to last version by Allquixotic

MakoBot: bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

2011-06-06T13:52:38Z

bot test - killing 'Template:' prefix from template includings and/or changing external-link to internal-link

Allquixotic: /* Basic Info */

2011-05-12T04:08:35Z

‎Basic Info

@@ Line 8: / Line 8: @@
 * '''Status''': Committed to git master as of May 12, 2011, Experimental
 * '''Branches Targeted''': git master
 * '''Code Repository''': Maintained in OpenSimulator git master

@@ Line 6: / Line 6: @@
 * '''Developer Sponsor(s)''': [[User:Allquixotic|allquixotic]], [[User:Justincc|justincc]]
 * '''Start Date''': February 18, 2011
-* '''Status''': Committed to git master as of May 12, 2011
+* '''Status''': Committed to git master as of May 12, 2011, Experimental
 * '''Branches Targeted''': git master
 * '''Commercial''': No

@@ Line 9: / Line 9: @@
 * '''Branches Targeted''': git master
 * '''Commercial''': No
-* '''Code Repository''': Maintained in OpenSim git master
+* '''Code Repository''': Maintained in OpenSimulator git master
 == High-Level Design ==
@@ Line 15: / Line 15: @@
 === Idea ===
-Currently, saving OARs in an organized manner requires significant hackery in the way of scripts or other external wrappers. This project is an attempt to get automatic OAR backup as a core capability of OpenSim.
+Currently, saving OARs in an organized manner requires significant hackery in the way of scripts or other external wrappers. This project is an attempt to get automatic OAR backup as a core capability of OpenSimulator.
 * On a '''Per-Region''' basis, we want to give the user the opportunity:
@@ Line 52: / Line 52: @@
 === User Experience ===
-This shouldn't be visible to end-users at all, but it will be very visible to OpenSim sysadmins. Sysadmins will edit Regions.ini to manage the config settings on a per-region basis. Maybe, for user-friendliness, we can have global config options in OpenSim.ini that serve as "default" options for the ones in Regions.ini, if they are not specified. The default will be to disable auto-backup altogether, but if a user enables auto-backup without specifying any other auto-backup config options, it'd be nice if they could set the rest of the options globally in OpenSim.ini. This would reduce the size of Regions.ini for very large numbers of regions, and make it trivial to merge the timers into one.
+This shouldn't be visible to end-users at all, but it will be very visible to OpenSimulator sysadmins. Sysadmins will edit Regions.ini to manage the config settings on a per-region basis. Maybe, for user-friendliness, we can have global config options in OpenSim.ini that serve as "default" options for the ones in Regions.ini, if they are not specified. The default will be to disable auto-backup altogether, but if a user enables auto-backup without specifying any other auto-backup config options, it'd be nice if they could set the rest of the options globally in OpenSim.ini. This would reduce the size of Regions.ini for very large numbers of regions, and make it trivial to merge the timers into one.
 == Low-Level Design ==

@@ Line 93: / Line 93: @@
 * Faster compression. Use a lower compression ratio for Gzip, or use a faster algorithm like LZO. Reference credible papers on lossless compression performance; find the codec with a compatibly-licensed implementation (BSD/MIT) with the least compression performance cost; implement. Changing the OAR file format might require some kind of transition, e.g., detect which format the `load oar' is in, and automatically support both Gzip and the new format (whichever that turns out to be). That shouldn't be too hard with the use of extremely conspicuous file format magic numbers that are very common with these formats.
-* JSON instead of XML. I've heard from sources in the embedded world that you can save quite a lot of CPU cycles by using JSON instead of XML, because there are fewer and less complex structural elements to parse  (on the read) and generate (on the write).
+* JSON instead of XML. I've heard from sources in the embedded world that you can save quite a lot of CPU cycles by using JSON instead of XML, because there are fewer and less complex structural elements to parse (on the read) and generate (on the write).
 * Faster asset serialization -- this is probably where a lot of the time is consumed, writing out assets. We should really profile this code and figure out where most of the time is spent. Are images being encoded into compressed file formats, and then compressed again in the OAR? Note that compressing already-compressed data is a spectacular waste of time, because you bring out worst-case behavior in the outer compression codec as it tries to extract a little more entropy out of the already-compressed data. Usually it ends up making the file size even larger due to file format overhead.
 ** My recommendation is that we leave already-compressed assets as-is and just tar them up together, and have a .gz file inside the tar with the compressible data, like XML or JSON. This will minimize file size and possibly speed up compression, since the outer compression codec won't have to sweat the high-entropy compressed data.
 * Use the operating system scheduler to our advantage. Instead of writing the GZipStream data as quickly as possible, use asynchronous writes, and put some yields in at appropriate places. This would effectively make the archive process take longer, but give up some scheduling slots for other threads to execute. The other option would be to offload the most resource-intensive parts of the operation to a separate process (e.g. compression), and set that process's scheduling priority to minimal. The improvement you'd see from this would be extremely dependent on the OS: on Windows, starting a process has an extremely high overhead, especially with virus scanners and having to load another instance of .NET. Recent Linux 2.6 is much more efficient at sharing data and spawning processes quickly, and low priority scheduling has a noticeable impact on resource usage on Linux.
 * Amortize the cost of preparing data for archiving by using spare cycles to prepare the assets' memory stream. This could open up a whole different opportunity for activity-based backup scheduling: when the region is at a particularly low point of activity, you could just take a backup whenever you are able to reach a high level of confidence that things are not busy.

@@ Line 4: / Line 4: @@
 * '''Summary''': A new module for periodically saving per-region backup OARs according to rules defined in Regions.ini.
-* '''Developer Sponsor(s)''': [http://opensimulator.org/wiki/User:Allquixotic allquixotic], [http://opensimulator.org/wiki/User:Justincc justincc]
+* '''Developer Sponsor(s)''': [[User:Allquixotic|allquixotic]], [[User:Justincc|justincc]]
 * '''Start Date''': February 18, 2011
 * '''Status''': Committed to git master as of May 12, 2011