[Opensim-dev] Roaming Assets [Was: Proposal to subdivide the assets table]

Fri Jun 27 07:10:09 UTC 2008

<Musing class="vague">

For some time, I've been thinking about what would be needed to create a distributed asset structure.

It's quite clear that you can have a case where all assets are local, if you have a region that only serves local content. In deed, I consider that the base case. (The standalone region serving only local content to anonymous visitors)
Of course, that would mean that 'external' assets would be broken/missing, you wouldn't have any inventory but what's local et c.

So the next step would be to discern between 'local assets' and 'roaming assets'. Which shouldn't be that hard - either it's local or it's roaming. I guess we kind of half-have that today.

(Consider a region where you could 'take' none of the local content. What's in it from the beginning, stays there.)

Now, if there was some way of encoding or resolving the asset so that the region could resolve the origin of that asset, you could have distributed roaming assets. It can't be impossible to track assets via guid, if we have full control over it all the way, from the upload?

Tech question: Are child agents fetching assets? I mean, if I land in a region, and it has a neighbour, is the neighbours texture content downloaded via my root agent or via the child agent I have in that neighbour?

Anywhichway; what I'm thinking is that you could come to the scenario where each region serves its own assets to those who needs it.

Then, any asset, like a t-shirt, could have a 'home asset server' that serves its textures.

Much like the web works today. It's up to the system architect to partition the content resource serving given the application scenario.

Of course, since assets are, and should be, immutable, a local copy cache is prudent.
</Musing>
Best regards,Stefan AnderssonTribal Media AB Join the 3d web revolution : http://tribalnet.se/ 

Date: Fri, 27 Jun 2008 11:42:03 +0900From: lulurun at gmail.comTo: opensim-dev at lists.berlios.deSubject: Re: [Opensim-dev] Proposal to subdivide the assets tableThank you for the "Cons", that makes my mind more clear about the subdividing proposal,Maybe my proposal about "how to subdivide" is not good or not completed or wrong.but I think there are *necessities* of subdividing assets table as I wrote in this mail.This mail is not going to talk about "HOW", I will send another mail later, to discuss aboutmy throughts on your comments.There are 2 basic reasons of the subdividing:1. No matter to use centralized or distributed assetservers, due to the *maintenancibility* of the asset data,   even my proposal is wrong, we have to consider a way to store assets but not just simply put assets   all together, assets have to be well managed.   assets are identified by UUID, but should not be managed (only) by UUID, assets should also be managed   by its purpose(for avatar, for region) and by its location(for which region).2. Since regionserver maybe not only owned by gridservice, but also owned by individual region owners, Making   regionserver more indenpendent increases regionserver(sim)'s *flexibility* and *portability*. (regionserver is not   suppose to be tied up with only one gridservice)I think to be compared to secondlife(even someone do not like to do so), OpenSim's main advantage is "OPEN",this is also the reason I feel happy to be involved in the project :)So we have to care about problems only happened in a *open* world:1. "maintenancibility" is a issue for grid admin, there is a big difference between a close world(secondlife) and   a opengrid(OSGrid).   in secondlife, since all of the data is kept by linden, no matter easy or difficult, they can find out "all textures   of a region", if a region resigned or want to be moved under another assetservice, linden can delete all of its   assets without disturbing other regions.   But in OSGrid, individual regionservers are open to register, so once the assetservice is started, it can not   be stopped.(however, it a free service ...) and the number of assets grows forever.2. "portability" is a issue mainly for region owner. in secondlife there is no "real" region owner, but in OSGrid,   when a region owner want to switch to anther grid service, they can not get thier assets back(easily).   And originally, prims are supposed to be closly with its assets(texture), It is OK to store them separatly, but   prims are well managed by "regionUUID", while assets are mixed together.Lulurun,regards
2008/6/27 Melanie <melanie at t-data.com>:
Hi,I'm sorry, but I don't see this being possible in quite the waydescribed. Actually, there are quite a few good points there, though.Pros:- Decrease network traffic- Decrease dependency on external servers- Increase reliability- Increase responsiveness- Save space on asset serverCons:- Prim crossings are much more difficult- ID space management is near impossible- Asset transfer between sims becomes a large issue- UUID referencing in scripts is no longer possibleWell, to make a long story short: Regions don't own assets. Regionsuse assets. Users own assets.So, a centralized asset storage is certainly needed, tied into theconcepts around asset portability, possibly several centralizedasset servers.But assets are immutable and a region is a finite space. This makesit an ideal situation for a persistent cache. While we cannottotally remove centralized asset storage, almost all the abovepoints can be achieved by locally caching a region's used assets ondisk. This cache can be two-tiered:Tier 1: referenced assets, indefinite retentionTier 2: unreferenced assets, short term retentionThis means, that textures on a prim would be cached locally forever,but textures that make up the clothing of an avatar passing throughwould be purged after a day or so.For sounds, playing them would re-trigger the retention period, sothat looped sounds will effectively be retained forever as well.Same goes for any frequently used asset, however, we should probablynot cache object assets.The one thing that this doesn't address is the issue of asset serverdisk space usage. However, with the proposed approach of hashing andde-duping, this becomes a much smaller problem already.However, because of the ability to reference an asset by it's UUID,letting a region "own" an asset is still not possible.Assume a texture, which I uploaded.I placed it on a prim, then, later, deleted the uploaded texturefrom my inventory.Technically, one could say that a region now holds the onlyreference, and the texture would be owned by that region. It couldbe removed from the central server as soon as it is no longerreferenced by any _agent_ inventory links.But, now there are three scenarios:1) The prim is pushed across a region boundary. The destinationregion requests the texture from the asset server and fails. Theprim will turn gray.2) The prim has previously been "take"n. A copy exists in egentinventory, but the list of the resources needed to reconstitute this"object" is hidden in the asset, actually, multiple asset records.If the texture has been removed from the central server, rezzingthis object in any region where the texture is not in use will fail.3) If i deleted the texture, there is no guarantee that I have alsolost it's UUID. I may have a notecard with all my texture UUIDs anduse them in scripts. Such scripts, taken to other regions, would fail.Issue 1 could be addressed by transferring the needed resourcesalong with the object, however, on a distributed grid like OSGrid,this can be time-consuming, as home DSL may be involved.Issue 3 could be addressed by creating a list of resources needed atthe time of the "take" and making that part of the inventory record.This would significantly bloat the inventory table, as each prim'sinventory, face textures and all sounds/animations it might play arelisted in the database.Issue 3 cannot be reasonably addressed.Therefore, caching, which is something I have looked into and amgoing to write some code for, is an absolutely viable solution formany of the raised issues, but weeding the central server's contentsin such a way seems to be impracticable.Melanie

liu xiaolu wrote:> Hi everyone,>> my proposal is to subdivide current assets table into 2 tables:> user_assets  : for user owned assets (mainly in inventory, appearance,> attachment, ...)>   user_asset is used by UGAI servers> region_assets: for region owned assets, sounds, textures(including sun,> moon, ...)>   region_asset is used by region servers>> I wrote the whole, detailed idea at:> http://opensimulator.org/wiki/User:Lulurun#proposal_to_subdivide_the_assets_table> Please have a look at it, here is only a brief introduction.>> This could bring us many advantages such as:> 0. increase the dispersity of asset data, clarify the responsibility of> each server.> Currently, all of the asset data is stored in assets table, managed by> assetserver.> This would cause some problems:>  1. assets table keeps growing in a fast pace.>  2. hard to determine which asset is belongs to inactive regions (when> you want to "clean"> the assets table).>  3. assetserver not only serves users' data but also regions' data, if> something happened to> assetserver both of avatar and prims will become gray.> To separate assets table can not only solve these problems, but also> brings minor benefits> such like: reduce the responsibility of assetserver, disperse the asset> requests(especially> for big grid service), lower the risk of losing data, ...>> 1. for Data Portability in virtual world> For grid admin,>  * you can concentrate on managing only user(not region owner) relavent> information>    like "users, avatarappearance, inventory">  * it can reduce the vulnerability caused by foreign assets.>  * when a region leaves your grid, you don't have to worry about its> assets becoming garbage.>   - on the other hand, this also good for region owners.> For when a user registers their region server with a grid,>  * When you want to switch your sim to a new grid, "prims, primshapes,> terrain, region_asset">    are always tied-up together. the only thing you need to do is to edit> UGAI servers URI in>    your OpenSim.ini>> For more advantages, please have a look at> http://opensimulator.org/wiki/User:Lulurun#Advantages>> If this is a good way to go, not so many changes is needed:> * UGAI servers would remain as they used to be.> * VW user won't feel anything different.> * Of course this should be a option in OpenSim.ini so that we can> enable/disable/config> * Just need to add supports for a new table "RegionAssets" with the same> structure as "assets".> * AssetClient in RegionServer should be improved to be able to get> assets from mutilple servers.>> For more detailed plan of the implementation, please have a look at> http://opensimulator.org/wiki/User:Lulurun#Implementation>> Any advice, comment, thought is appreciate.>> --> Lulurun> regards>>> ------------------------------------------------------------------------>> _______________________________________________> Opensim-dev mailing list> Opensim-dev at lists.berlios.de> https://lists.berlios.de/mailman/listinfo/opensim-dev_______________________________________________Opensim-dev mailing listOpensim-dev at lists.berlios.dehttps://lists.berlios.de/mailman/listinfo/opensim-dev-- Lulurun 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080627/677f1672/attachment-0001.html>