[Opensim-dev] Proposal to subdivide the assets table

[Melanie]

Hi,

I'm sorry, but I don't see this being possible in quite the way 
described. Actually, there are quite a few good points there, though.

Pros:
- Decrease network traffic
- Decrease dependency on external servers
- Increase reliability
- Increase responsiveness
- Save space on asset server

Cons:
- Prim crossings are much more difficult
- ID space management is near impossible
- Asset transfer between sims becomes a large issue
- UUID referencing in scripts is no longer possible

Well, to make a long story short: Regions don't own assets. Regions 
use assets. Users own assets.

So, a centralized asset storage is certainly needed, tied into the 
concepts around asset portability, possibly several centralized 
asset servers.

But assets are immutable and a region is a finite space. This makes 
it an ideal situation for a persistent cache. While we cannot 
totally remove centralized asset storage, almost all the above 
points can be achieved by locally caching a region's used assets on 
disk. This cache can be two-tiered:

Tier 1: referenced assets, indefinite retention
Tier 2: unreferenced assets, short term retention

This means, that textures on a prim would be cached locally forever, 
but textures that make up the clothing of an avatar passing through 
would be purged after a day or so.
For sounds, playing them would re-trigger the retention period, so 
that looped sounds will effectively be retained forever as well. 
Same goes for any frequently used asset, however, we should probably 
not cache object assets.

The one thing that this doesn't address is the issue of asset server 
disk space usage. However, with the proposed approach of hashing and 
de-duping, this becomes a much smaller problem already.

However, because of the ability to reference an asset by it's UUID, 
letting a region "own" an asset is still not possible.

Assume a texture, which I uploaded.
I placed it on a prim, then, later, deleted the uploaded texture 
from my inventory.
Technically, one could say that a region now holds the only 
reference, and the texture would be owned by that region. It could 
be removed from the central server as soon as it is no longer 
referenced by any _agent_ inventory links.

But, now there are three scenarios:

1) The prim is pushed across a region boundary. The destination 
region requests the texture from the asset server and fails. The 
prim will turn gray.

2) The prim has previously been "take"n. A copy exists in egent 
inventory, but the list of the resources needed to reconstitute this 
"object" is hidden in the asset, actually, multiple asset records. 
If the texture has been removed from the central server, rezzing 
this object in any region where the texture is not in use will fail.

3) If i deleted the texture, there is no guarantee that I have also 
lost it's UUID. I may have a notecard with all my texture UUIDs and 
use them in scripts. Such scripts, taken to other regions, would fail.

Issue 1 could be addressed by transferring the needed resources 
along with the object, however, on a distributed grid like OSGrid, 
this can be time-consuming, as home DSL may be involved.

Issue 3 could be addressed by creating a list of resources needed at 
the time of the "take" and making that part of the inventory record. 
This would significantly bloat the inventory table, as each prim's 
inventory, face textures and all sounds/animations it might play are 
listed in the database.

Issue 3 cannot be reasonably addressed.

Therefore, caching, which is something I have looked into and am 
going to write some code for, is an absolutely viable solution for 
many of the raised issues, but weeding the central server's contents 
in such a way seems to be impracticable.

Melanie

liu xiaolu wrote:
> Hi everyone,
> 
> my proposal is to subdivide current assets table into 2 tables:
> user_assets  : for user owned assets (mainly in inventory, appearance, 
> attachment, ...)
>   user_asset is used by UGAI servers
> region_assets: for region owned assets, sounds, textures(including sun, 
> moon, ...)
>   region_asset is used by region servers
> 
> I wrote the whole, detailed idea at:
> http://opensimulator.org/wiki/User:Lulurun#proposal_to_subdivide_the_assets_table
> Please have a look at it, here is only a brief introduction.
> 
> This could bring us many advantages such as:
> 0. increase the dispersity of asset data, clarify the responsibility of 
> each server.
> Currently, all of the asset data is stored in assets table, managed by 
> assetserver.
> This would cause some problems:
>  1. assets table keeps growing in a fast pace.
>  2. hard to determine which asset is belongs to inactive regions (when 
> you want to "clean"
> the assets table).
>  3. assetserver not only serves users' data but also regions' data, if 
> something happened to
> assetserver both of avatar and prims will become gray.
> To separate assets table can not only solve these problems, but also 
> brings minor benefits
> such like: reduce the responsibility of assetserver, disperse the asset 
> requests(especially
> for big grid service), lower the risk of losing data, ...
> 
> 1. for Data Portability in virtual world
> For grid admin,
>  * you can concentrate on managing only user(not region owner) relavent 
> information
>    like "users, avatarappearance, inventory"
>  * it can reduce the vulnerability caused by foreign assets.
>  * when a region leaves your grid, you don't have to worry about its 
> assets becoming garbage.
>   - on the other hand, this also good for region owners.
> For when a user registers their region server with a grid,
>  * When you want to switch your sim to a new grid, "prims, primshapes, 
> terrain, region_asset"
>    are always tied-up together. the only thing you need to do is to edit 
> UGAI servers URI in
>    your OpenSim.ini
> 
> For more advantages, please have a look at
> http://opensimulator.org/wiki/User:Lulurun#Advantages
> 
> If this is a good way to go, not so many changes is needed:
> * UGAI servers would remain as they used to be.
> * VW user won't feel anything different.
> * Of course this should be a option in OpenSim.ini so that we can 
> enable/disable/config
> * Just need to add supports for a new table "RegionAssets" with the same 
> structure as "assets".
> * AssetClient in RegionServer should be improved to be able to get 
> assets from mutilple servers.
> 
> For more detailed plan of the implementation, please have a look at
> http://opensimulator.org/wiki/User:Lulurun#Implementation
> 
> Any advice, comment, thought is appreciate.
> 
> -- 
> Lulurun
> regards
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev