[Opensim-dev] AssetServer Observations and Suggestions

Mon Feb 9 16:41:13 UTC 2009

Hi,

again, i believe three criteria need to be met for an asset to be 
reaped:

- It must not be referenced by user inventory directly or indirectly
- It must not be referenced by an inworld object, or anything in it
- It must be unaccessed for a certain period of time (6 months being 
my suggestioin)

This last case covers references by UUID. UUIDs can reference 
textures and sounds only. So, all assets satisfying criteria 1 and 2 
can be removed immediately, unless they are textures or sounds, in 
which case criterion 3, 6 months without access, would apply.

UUIDs can not reference object assets, animations or wearables, even 
though their UUID can be obtained, it can't be used in any LSL 
function call.
UUIDs can also not reference script source, as that is specifically 
blocked, you can't view script source unless the script is in your 
or the object's inventory with proper permissions.

Therefore, unreferenced object, script, animation and wearables 
assets can be removed immediately upon detecting that they are 
totally unreferenced.

Further, texture assets with the temporary flag set can be removed 
immediately when unreferenced, unless referenced by the regions table.

Melanie

Alan M Webb wrote:
> A newly created region contains no assets (except for terrain?). Objects 
> get rezzed from OAR files and by avatars moving things from their 
> inventories, right? These objects reference other objects and assets 
> (textures et al) either directly or via object inventory.
> 
> In principle, it is sufficient for the asset database to reflect only 
> those assets required to support the set of objects considered to be part 
> of the permanent region state at any given point in time. Of course, I'm 
> assuming here that the assets needed by an objects that are subsequently 
> rezzed can be resolved somehow. The approach we have been experimenting 
> with locally is to extend the "Not Found" handling to search for an asset 
> in a static/dynamic set of other known asset repositories. This should 
> make the process transparent to all.
> 
> Weeding out abandoned assets can, as everyone has said, by examining the 
> point-in-time configuration of the region's permanent state. This could be 
> done by a separate process or low-priority thread. As part of this 
> approach it would seem logical to have a secondary asset database. Moving 
> the unreferenced assets to this "apparently unused asset" database. If an 
> asset is not resolved in the local asset database then the first 
> alternative location checked is the "apparently unused asset" database. 
> The search continues with other independent asset servers according  to 
> whatever search algorithm is in effect.
> 
> Best regards
> Alan
> -------------------
> T.J. Watson Research Center, Hawthorne, NY
> 1-914-784-7286
> alan_webb at us.ibm.com
> 
> 
> 
> From:
> Melanie <melanie at t-data.com>
> To:
> opensim-dev at lists.berlios.de
> Date:
> 02/09/2009 10:33 AM
> Subject:
> Re: [Opensim-dev] AssetServer Observations and Suggestions
> 
> 
> 
> In a normal grid use scenario (OSGRID) I believe that most assets 
> are _initially_ created by uploading them through the viewer. Even 
> though they can be deleted, few people delete their textures, since 
> they are always needed for further building. The savings would be 
> minuscule, IMHO. Assets being implicitly shared, any textures passed 
> to another user will link to the same inventory item, so that 
> deleting something from one user's inventory doesn't mean that it is 
> deleted from all users' inventories. There is no way to determine a 
> reference but by scanning all inventories and all object assets.
> 
> Melanie
> 
> Justin Clark-Casey wrote:
>> Melanie wrote:
>>> All assets have, at some point, been on a user's inventory. There is 
>>> no way to upload anything without having it in inventory at some point.
>>> The largest assets are textures, and they will always pass through 
>>> user inventories. Also, they will remain in user inventories despite 
>>> _also_ being on region objects. Making regions hold their own assets 
>>> authoritatively would cause massive data duplication, albeit 
>>> distributed, and not really unclutter the central database.
>> 
>> Assets can be loaded via OAR and other mechanisms that never flow 
> through inventory.  Also, I believe it's quite 
>> possible to completely delete items from user inventory.
>> 
>>> 
>>> I believe a tag and reap mechanism that slow-scans all assets for 
>>> contained assets and all inventories for referencs and tags all 
>>> referenced assets, as well as regions transmitting "asset lists" of 
>>> assets currently referenced inworld, and the last use flag. Let that 
>>> run for 6 months, then reap all unflagged, reset all flags, and 
>>> start a new cycle.
>>> 
>>> This doesn't have to be mutually exclusive, in fact it can be 
>>> combined with persistent local asset caching (e.g. local texture 
>>> storage for inworld objects) as well as a HG model.
>>> 
>>> Melanie
>>> 
>>> Diva Canto wrote:
>>>> Diva Canto wrote:
>>>>> Justin Clark-Casey wrote:
>>>>> 
>>>>>> An even more radical solution would be to switch osgrid to a pure 
> Hypergrid model.  The osgrid UAI services could still 
>>>>>> act as the home services for many people who don't want to run their 
> own regions, but the responsibilty for maintaining 
>>>>>> region-side assets would shift to other OpenSim instances (and some 
> people would also use them for their home services 
>>>>>> instead of osgrid).
>>>>>>
>>>>>> 
>>>>>> 
>>>>> I've been thinking about this too. I think it should be possible, 
>>>>> although we need to rethink a little bit the association between 
> users 
>>>>> and UGAIM servers. Right now it's sort of bundled; we need to 
> unbundle. 
>>>>> We could make an interface for the User server that would allow users 
> to 
>>>>> set their servers.
>>>>> 
>>>> Actually, we don't even need to change the user base data. The 
> inventory 
>>>> assets are only a [relatively small?] percentage of all the assets in 
>>>> the grid asset server. Most of the assets, I would say, are inworld 
>>>> things that aren't on any inventory. They are on the "region's 
>>>> inventory" so to speak. So we could move all regions into HG mode, 
>>>> setting the proper local servers in their OpenSim.ini's. The users 
>>>> inventory would still be OSGrid's Inventory server.
>>>>
>>>> Even so, there are a few things that need changing/improving for this 
> to 
>>>> happen. But it's not that far out.
>>>>
>>>>
>>>>
>>>>
>>>> 
> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Opensim-dev mailing list
>>>> Opensim-dev at lists.berlios.de
>>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at lists.berlios.de
>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>> 
>> 
>> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev