[Opensim-dev] Standardizing OpenSim data formats

[Justin Clark-Casey]

Hurliman, John wrote:
> Currently, OpenSim server<->server communication is primarily made up of .NET XML serialization. As a side effect of this, I've seen the wire format between simulator and grid services or grid service to grid service change many times since I started working on backend services. The OAR file format has also changed at least once while retaining the same version number because the underlying prim XML format changed. I am working on a libomv OAR reader/writer at the moment, and trying to deal with hacks for supporting pre and post change OAR files.
> 

OAR was intended more as a packaging mechanism than a complete format.  This is partly why I didn't bump the version 
number when the serialization format changed.

In principle, I think it would be good if the serializations themselves identified themselves as one format or another 
(as I think you've already indicated in a previous post).  Then one could have different serializations (e.g. mesh) 
without having to produce different OAR structures or different formats entirely.  I'm not sure how pragmatic this is - 
perhaps some indication of the format of serialization could be placed in the OAR control file (though even then there 
might be regions containing multiple different serializations).

The XML serialization 'XML2' format was used for objects in the OAR partly just because it was already there and it 
worked, even if it is pretty horrible.  I feel that one nasty point about it for offline content transfer is that it 
doesn't contain any versioning information to identify it (necessitating heuristics I guess).  I feel that there may 
also come a point where an offline format will diverge from an on-the-wire format (e.g. inventory serial seems 
unncessary for offline purposes - though many other things will remain for the backup use case as opposed to the 
transfer to some one else use case).

> To increase compatibility between versions of OpenSim and with third party services I'd like to propose a standardization of (eventually) all over-the-wire communication in OpenSim. I started with an attempt at an LLIDL definition for primitives:
> 
> http://opensimulator.org/wiki/PrimitiveFormatProposal
> 
> Is anything missing? Does this look like a reasonable start?

On a quick look through, I don't see any attribute for sculpt information.  At the moment this is stored inline in the 
serialization.  Also, I suspect state information for scripts may have crept in recently (not sure).

Also, at least for archiving purposes, the creator id will at some point become a string rather than a uuid in order to 
allow identification information to be retrieved on grids other than the one on which the archive was saved.

-- 
justincc
Justin Clark-Casey
http://justincc.wordpress.com