Duplicity 

The Duplicity backup run for the DS VPS has completed. 49GB of space is being used on the S3 storage, which is 51GB of content, compressed, encrypted and signed. Clearly a lot of our content is already in compressed formats, such as JPEG, AVI and so on.

The interesting part that I couldn't find information about is how much meta-data Duplicity needs to store locally about the backup. During the backup run, the working folder was climbing above 2GB. On completion, the "sigtar" file was gzipped, and in the end has ended up as 1.2GB. This works out at approximately 2% the size of the total compressed data. I consider this a suitable cost to be able to have off-host incremental backups without a local duplicate copy of the backup data.

Also, something I realized that made obvious sense once I realized it: nice duplicity. It just makes everything happier. While duplicity isn't CPU intensive, it just seemed to help with the system load. ionice would have been better, but Debian Etch doesn't have this.
[ add comment ] ( 35 views ) permalink
Duplicity 

A note regarding Duplicity and failed full backups - It noticed the failed backup, displayed what file and block it was up to, and got back to it.
[ add comment ] ( 35 views ) permalink
VPS, Qmail & SpamAssassin, EvoGrid, OpenSim 

Logged in this morning to a number of emails telling me the DS VPS had been shut down overnight. Investigated a little, after bringing it up. Logs suggest it was shut down, not failed or powered off in some manner. I haven't yet determined if this was in some way related to the backup being performed, as I haven't received a reply to the support ticket yet. It seems that I'll be able to test how duplicity handles failure. I hope it will just see all the files not yet backed up as new, and continue.

Spent time working on getting SpamAssassin to do SPF checking when used with QMail, I needed to set always_trust_envelope_sender to 1. I also set envelope_sender_header to Return-Path, but this may not be needed since SpamAssassin checks this head by default.

Implemented pending/history in the Simulation Manager for EvoGrid. Re-arranged the "lib" folder into libevogrid and split it into src/include/lib/obj etc.

Realised that in OpenSim, the script function llStartAnimation uses strings to specify default animations, such as the ones I had replaced. However I hadn't seen any references to this when implementing my changes. I followed the logic of this through and found a function that duplicates some of the functionality of AvatarAnimations by reading the same file. I've submitted an email to the opensim-dev list and will see if anyone can shed some light on this.
[ add comment ] ( 35 views ) permalink
Amazon S3, Duplicity and OpenSim 

Spent the day setting up DS VPS backups to Amazon S3. No, I haven't brought into the "cloud" hype, they just have VERY cheap rates for data storage.
Set up Duplicity to perform the backups, which uses a clever use of the rysnc library to detect modified files, without needing the original files in place, only hash data made from the last full backup. Setting this up involved updating the version on the server (Debian Etch, so something in the 0.4 generation ) to the current release. This was eventually done by building from the source package for latest debian (Lenny). Fortunately Duplicity is Python, with an adapter class to librsync, so there were not any real version conflicts, just cases where packages said they needed the version of other packages that are current, but still worked with older versions.

Almost finished the work on the OpenSim codebase, moving from using constant strings to an Enum for avatar states. The code is written, supports all the states defined in the external file (not just the ones actually in use), compiles and runs. And works with my in-development server-side "AO" functionality. This isn't an over-ride as SL users know them, rather it changes the animations used for the states at the lower level. So no polling.
Only step remaining is working out how to push or otherwise publish these changes for evaluation. This will include me writing a post going into the details of the change, and why they are good ideas, and any caveats added by the change.
[ add comment ] ( 36 views ) permalink
Joomla and OpenSim 

Evaluated Joomla modules for splash/slide-show functionality for DS.com revamping. Working with the web developer to decide the best way to move the previously static and hand updated pages into categories to use Joomla's functionality for updating and adding new content.

Did work on the OpenSim core code to replace the use of magic strings ("STAND", "WALK", "SIT" and similar) for states with an Enum. The possible values are defined in an external data file, and any states not in this file are not functional, so any flexibility added by using strings is limited. The file has not been updated since 2007 (based on its internal comments) so the values may be considered finalized.
Additionally the strings were not documented, are defined across multiple points of code, and had minimal error checking. They were also being used as look-up keys and comparison values, which is inefficient at best and breaking at worse if the string is updated and not changed in all of its separate instances.

This code has not been made publicly available yet, as it doesn't yet include all the states which were supported and not used. It does support all the states that were used.
[ add comment ] ( 42 views ) permalink

<<First <Back | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | Next> Last>>