[Opensim-dev] Proposal to subdivide the assets table

[liu xiaolu]

Thank you for the "Cons", that makes my mind more clear about the
subdividing proposal,
Maybe my proposal about "how to subdivide" is not good or not completed or
wrong.
but I think there are *necessities* of subdividing assets table as I wrote
in this mail.
This mail is not going to talk about "HOW", I will send another mail later,
to discuss about
my throughts on your comments.

There are 2 basic reasons of the subdividing:
1. No matter to use centralized or distributed assetservers, due to the
*maintenancibility* of the asset data,
   even my proposal is wrong, we have to consider a way to store assets but
not just simply put assets
   all together, assets have to be well managed.
   assets are identified by UUID, but should not be managed (only) by UUID,
assets should also be managed
   by its purpose(for avatar, for region) and by its location(for which
region).
2. Since regionserver maybe not only owned by gridservice, but also owned by
individual region owners, Making
   regionserver more indenpendent increases regionserver(sim)'s
*flexibility* and *portability*. (regionserver is not
   suppose to be tied up with only one gridservice)

I think to be compared to secondlife(even someone do not like to do so),
OpenSim's main advantage is "OPEN",
this is also the reason I feel happy to be involved in the project :)
So we have to care about problems only happened in a *open* world:
1. "maintenancibility" is a issue for grid admin, there is a big difference
between a close world(secondlife) and
   a opengrid(OSGrid).
   in secondlife, since all of the data is kept by linden, no matter easy or
difficult, they can find out "all textures
   of a region", if a region resigned or want to be moved under another
assetservice, linden can delete all of its
   assets without disturbing other regions.
   But in OSGrid, individual regionservers are open to register, so once the
assetservice is started, it can not
   be stopped.(however, it a free service ...) and the number of assets
grows forever.
2. "portability" is a issue mainly for region owner. in secondlife there is
no "real" region owner, but in OSGrid,
   when a region owner want to switch to anther grid service, they can not
get thier assets back(easily).
   And originally, prims are supposed to be closly with its assets(texture),
It is OK to store them separatly, but
   prims are well managed by "regionUUID", while assets are mixed together.

Lulurun,
regards


2008/6/27 Melanie <melanie at t-data.com>:

> Hi,
>
> I'm sorry, but I don't see this being possible in quite the way
> described. Actually, there are quite a few good points there, though.
>
> Pros:
> - Decrease network traffic
> - Decrease dependency on external servers
> - Increase reliability
> - Increase responsiveness
> - Save space on asset server
>
> Cons:
> - Prim crossings are much more difficult
> - ID space management is near impossible
> - Asset transfer between sims becomes a large issue
> - UUID referencing in scripts is no longer possible
>
> Well, to make a long story short: Regions don't own assets. Regions
> use assets. Users own assets.
>
> So, a centralized asset storage is certainly needed, tied into the
> concepts around asset portability, possibly several centralized
> asset servers.
>
> But assets are immutable and a region is a finite space. This makes
> it an ideal situation for a persistent cache. While we cannot
> totally remove centralized asset storage, almost all the above
> points can be achieved by locally caching a region's used assets on
> disk. This cache can be two-tiered:
>
> Tier 1: referenced assets, indefinite retention
> Tier 2: unreferenced assets, short term retention
>
> This means, that textures on a prim would be cached locally forever,
> but textures that make up the clothing of an avatar passing through
> would be purged after a day or so.
> For sounds, playing them would re-trigger the retention period, so
> that looped sounds will effectively be retained forever as well.
> Same goes for any frequently used asset, however, we should probably
> not cache object assets.
>
> The one thing that this doesn't address is the issue of asset server
> disk space usage. However, with the proposed approach of hashing and
> de-duping, this becomes a much smaller problem already.
>
> However, because of the ability to reference an asset by it's UUID,
> letting a region "own" an asset is still not possible.
>
> Assume a texture, which I uploaded.
> I placed it on a prim, then, later, deleted the uploaded texture
> from my inventory.
> Technically, one could say that a region now holds the only
> reference, and the texture would be owned by that region. It could
> be removed from the central server as soon as it is no longer
> referenced by any _agent_ inventory links.
>
> But, now there are three scenarios:
>
> 1) The prim is pushed across a region boundary. The destination
> region requests the texture from the asset server and fails. The
> prim will turn gray.
>
> 2) The prim has previously been "take"n. A copy exists in egent
> inventory, but the list of the resources needed to reconstitute this
> "object" is hidden in the asset, actually, multiple asset records.
> If the texture has been removed from the central server, rezzing
> this object in any region where the texture is not in use will fail.
>
> 3) If i deleted the texture, there is no guarantee that I have also
> lost it's UUID. I may have a notecard with all my texture UUIDs and
> use them in scripts. Such scripts, taken to other regions, would fail.
>
> Issue 1 could be addressed by transferring the needed resources
> along with the object, however, on a distributed grid like OSGrid,
> this can be time-consuming, as home DSL may be involved.
>
> Issue 3 could be addressed by creating a list of resources needed at
> the time of the "take" and making that part of the inventory record.
> This would significantly bloat the inventory table, as each prim's
> inventory, face textures and all sounds/animations it might play are
> listed in the database.
>
> Issue 3 cannot be reasonably addressed.
>
> Therefore, caching, which is something I have looked into and am
> going to write some code for, is an absolutely viable solution for
> many of the raised issues, but weeding the central server's contents
> in such a way seems to be impracticable.
>
> Melanie
>
> liu xiaolu wrote:
> > Hi everyone,
> >
> > my proposal is to subdivide current assets table into 2 tables:
> > user_assets  : for user owned assets (mainly in inventory, appearance,
> > attachment, ...)
> >   user_asset is used by UGAI servers
> > region_assets: for region owned assets, sounds, textures(including sun,
> > moon, ...)
> >   region_asset is used by region servers
> >
> > I wrote the whole, detailed idea at:
> >
> http://opensimulator.org/wiki/User:Lulurun#proposal_to_subdivide_the_assets_table
> > Please have a look at it, here is only a brief introduction.
> >
> > This could bring us many advantages such as:
> > 0. increase the dispersity of asset data, clarify the responsibility of
> > each server.
> > Currently, all of the asset data is stored in assets table, managed by
> > assetserver.
> > This would cause some problems:
> >  1. assets table keeps growing in a fast pace.
> >  2. hard to determine which asset is belongs to inactive regions (when
> > you want to "clean"
> > the assets table).
> >  3. assetserver not only serves users' data but also regions' data, if
> > something happened to
> > assetserver both of avatar and prims will become gray.
> > To separate assets table can not only solve these problems, but also
> > brings minor benefits
> > such like: reduce the responsibility of assetserver, disperse the asset
> > requests(especially
> > for big grid service), lower the risk of losing data, ...
> >
> > 1. for Data Portability in virtual world
> > For grid admin,
> >  * you can concentrate on managing only user(not region owner) relavent
> > information
> >    like "users, avatarappearance, inventory"
> >  * it can reduce the vulnerability caused by foreign assets.
> >  * when a region leaves your grid, you don't have to worry about its
> > assets becoming garbage.
> >   - on the other hand, this also good for region owners.
> > For when a user registers their region server with a grid,
> >  * When you want to switch your sim to a new grid, "prims, primshapes,
> > terrain, region_asset"
> >    are always tied-up together. the only thing you need to do is to edit
> > UGAI servers URI in
> >    your OpenSim.ini
> >
> > For more advantages, please have a look at
> > http://opensimulator.org/wiki/User:Lulurun#Advantages
> >
> > If this is a good way to go, not so many changes is needed:
> > * UGAI servers would remain as they used to be.
> > * VW user won't feel anything different.
> > * Of course this should be a option in OpenSim.ini so that we can
> > enable/disable/config
> > * Just need to add supports for a new table "RegionAssets" with the same
> > structure as "assets".
> > * AssetClient in RegionServer should be improved to be able to get
> > assets from mutilple servers.
> >
> > For more detailed plan of the implementation, please have a look at
> > http://opensimulator.org/wiki/User:Lulurun#Implementation
> >
> > Any advice, comment, thought is appreciate.
> >
> > --
> > Lulurun
> > regards
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>



-- 
Lulurun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20080627/efeb19fb/attachment-0001.html>