[Opensim-users] Approach to get fresh content for an OpenSim Grid on New Servers

[Melanie]

Hi,

this requires a sweep and mark approach.

Create a table that parallels your asset table. It should contain
the asset ID and a date field which would default to the asset's
creation date.

Next you need to iterate over all rezzed objects in all sims and
collect the UUID of any used textures and the UUIDs of all inventory
items. Keep those that refer to objects separately, as well as those
that refer to scripts and wearables. Also make a list of all
notecards. Don't forget to include ground textures and parcel
textures. Mark all textures and wearables with the current date.

Then, similarly iterate all user inventories. Mark all UUIDs of
simple objects like wearables and textures, then add the UUIDs of
the objects, wearables, notecards and scripts to the UUID lists
created in the first step. Mark all "simple" (non-object) assets
with the current date.

Next, iterate all notecards. Extract all attached (embedded) objects
and add the wearables and objects and other notecards to the lists
above. Mark all simple assets found. Repeat until no more notecards
are left.

Finally, iterate all object assets, retrieving and parsing them to
glean the same info as you obtained from the sim. Mark textures and
wearables and also add wearables, objects and scripts to the above
lists. Mark all object assets with the current date.

Repeat until no more objects are left in the object list.

If notecards were collected, go back to iterating the notecards.

Last, iterate wearables (extract texture ids and mark them) and
scripts (regex-match UUID-like things and if they are a known asset,
mark them).

Depending on the grid size, this should leave you anywhere from a
few days to a few months on. Assuming the sweep & mark took less
than 6 months, all UUID in the secondary table that have a date
stamp OLDER than 6 months are unreferenced and the assets they name
can be deleted.

Since old assets are static, limiting future scans to only look at
things changed after the last scan was done will allow you to keep
this dated list and use it for further purging, dramatically
shortening the next sweeping run.

A lot of this code is present in the flotsam cache and the
archivers. It's a matter of putting it together properly and filling
in a few tricky spots.

Melanie

On 13/04/2013 02:40, Jeff Kelley wrote:
> At 4:20 PM -0500 4/12/13, Chris wrote:
> 
>>  As far as I can tell it does also pick up assets recursively although how
>>  deeply nested I am unsure of; I have not tested more than a few objects deep.
> 
> Well, there is no recursivity, strictly speaking. "Nested" is more 
> accurate. An object in itself is stramlined in the XML (not a 
> reference pointing to self). Recursivity is achieved in-world  with 
> the "rez and give a copy of self" trick.
> 
> I'm searching a way to clean the database more deeply than with SQL queries.
> That implies parsing SceneObjectGroups, as well as the binary UUIDs 
> in primShapes:Texture. Not a simple task.
> 
> 
> -- Jeff
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-users
> 
>