[Opensim-dev] Prim (Task) Inventory - initial thoughts for an implementation in OpenSim 0.6

[Justin Clark-Casey]

Sean Dague wrote:
> On Tue, Dec 11, 2007 at 12:54:31AM +0000, Justin Clark-Casey wrote:
>> As MW has outlined, there are two aspects involved in implementing 
>> persistent prim inventories - persisting them in an avatar's inventory 
>> and persisting them in a region.
>>
>> I propose that in both of these cases we store the prim's inventory as 
>> part of its XML serialization, which I believe is stored as an asset 
>> when a prim (or prim group) is taken into an avatar's inventory.  
>> Metadata about items in the prim's inventory (name, permissions, etc) 
>> would be stored as part of the serialization

>> I believe this will work without too many changes for the avatar 
>> inventory, since an asset server is already available in both standalone 
>> and grid modes.  The same asset server is also available locally to a 
>> region in standalone mode.  However, I expect it would be a bad idea to 
>> fetch assets from the grid asset server every time a grid mode region 
>> started up - both in terms of unscalable load and bandwidth costs.
>>
>> So I would propose that as part of the implementation of persistent prim 
>> inventories, we need a local asset server that acts largely as a cache 
>> for the grid asset server.  When a prim is initially rezzed to a region, 
>> the only additional data other than that which we should supply already 
>> (name, description, etc), is the asset id representing the serialized 
>> parts of the prim. (Actually, I suspect that we're serializing things 
>> such as name and description which we arguably shouldn't be - as 
>> evidenced by the fact that if I change the name of a prim in inventory 
>> and rerez it, it comes back with the name it had when I originally took 
>> it from the region).
>>
>> On receiving the asset ID, the region immediately makes a request to its 
>> local server asset cache for the serialized data in order to run any 
>> scripts in the prim inventory.  If the asset is not found locally, then 
>> the request goes out to the grid server.  Once the grid server supplies 
>> the asset, it can be cached persistently locally (as assets never change).
>>
>> Once the asset is received by the original region calling code, it's 
>> deserialized and loaded into memory.  The asset ids contained within the 
>> prim inventory can be requested as required.
>>
>> If any of the metadata for those contents changes, then a new serialized 
>> asset is stored locally.  If any of the prim contents change (e.g. a 
>> script is updated), then both the contents information (with the new 
>> asset id) and the new script asset are stored locally.
>>
>> If a user takes the prim into their inventory, we need to pass to the 
>> grid the asset id holding the serialized aspects of the prim.  If this 
>> doesn't exist on the grid asset server, the grid will need to make a 
>> request for that asset to the local region asset server.  It will need 
>> to deserialize this data and make requests in turn for any of the prim 
>> inventory asset ids it's missing.
>>
>> I believe the major cost of this scheme is that associated with the 
>> [de]serialization of the prim.  I've thought through a few alternative 
>> proposals which avoid this by establishing task inventory tables and the 
>> like, but these all seem to become rapidly quite complicated as one 
>> tries to account for things such as prims which exist inside themselves 
>> and the reuse of item inventory entries (so that identical items don't 
>> end up unnecessarily duplicating inventory structures).  The 
>> serialization scheme has the wonderful virtue of simplicity (assuming it 
>> works :-).
>>     
>
> I think that if you specify a parent_id relationship, you can do this
> pretty easily and recursively in tables in a reasonable way.  Having a
> seperate parent_child inventory table will work reasonably well here as
> well.  Perhaps if I have a few examples I can propose table models and
> queries that will give you what you want here.
>
> The inventory normalization (keeping a minimum number of inventory
> items) needs to be thought of carefully, because unless they are
> no-moddify, you will need to make copies at some point.  Delayed copy
> might save some storage, but may make the code much more complicated.
>
>   
You may well be right - I was getting seduced by the relative ease of 
[de]serialization compared to storing data in database tables using our 
current database coding patterns.  But then if you take this pattern to 
its extreme, our database tables would consist mainly of tables with 
only two fields, UUIDs and blobs.  Some of the recent commercial 
database versions can search within XML but for the open source 
databases that I think we'll continue to support, this would make the 
data pretty opaque.  And it does negate some of the benefits of using a 
database in the first place.

So I'm going to go down the path of object inventory tables instead, 
though if possible I hope to code it in such a way that different 
persistent strategies can be plugged in.  I had done some initial 
thinking on this approach (reusing as much existing inventory code as 
possible).  But rather than trying to think it all out fully in advance, 
I'm just going to start working with the codebase instead with the 
occasional mailing list posting.  If anybody feels I should be posting 
more details (apart from what crops up in the changelogs) please let me 
know.  All this changes will be marked experimental until 0.5 is released.

However, I do want to restate that I think a local persistent asset 
cache for regions running under grid assets will be important for prim 
inventory, since I wouldn't anticipate that these would be stored 
directly in the region persistence in the same way that prim textures 
are currently stored (i.e. inlined in the XML SceneObjectPart 
serialization).  This is for the same scalability reasons as stated 
originally.  Such an implementation shouldn't consist of much more than 
using the existing asset code to decorate the GridAssetClient 
IAssetServer interface.

--
justincc