[Opensim-dev] what to use for serializing structures? LLSD? XmlSerializer? JSON?

[Kyle Hamilton]

XML is not "lightweight".  It takes a LOT to be able to use it
properly, and this reduces its utility and increases the barrier to
entry.

(Yes, it's possible to prove of XML that 'it matches the schema'
[presuming the schema is defined], but even then I tend to think that
values usually aren't checked for sanity before use.)

JSON has the advantage of being easy to parse, decreasing network
protocol overhead, having many implementations, being directly
mappable to C# primitives and classes without expensive overhead, not
requiring a predefined schema, and allowing for more useful data
handling...  as well as being much more easily read by people who
understand C or C# without having to worry about learning some kind of
markup that has begun to rival SGML for sheer attempted generality
(and thus complexity).

To get a grasp of what it is I'm talking about, look at the number of
pages in the XML standard.  Then look at the number of pages in the
JSON description (RFC 4627).  And then look at http://json.org/ to
figure out how many implementations of JSON parsing there are for C#.
(not to mention how few diagrams are actually required to describe the
language.)

I believe that XML, while useful in many contexts, suffers from the
same problems that the Electronic Data Interchange standard has (such
as "the schema must be known at both ends").  As well, It is far too
formalized, the tag lengths are far too arbitrary, and the sheer
number of times those tags must be duplicated is insane (and, while it
makes adaptive-dictionary compression work really well, that's the
kind of redundancy that is really inappopriate for any kind of network
protocol).

I'm forced to remind everyone of something that they've all seemed to
forget: increasing the amount of data to be transferred increases the
number of packets that must be transmitted.  Increasing the number of
packets transmitted increases the amount of work that the network must
do.  Increasing the amount of data to be transferred also increases
memory usage on the client and server -- as well as the amount of
memory that must be used on each router and gateway in between.  (Why
can't the designers of new systems be nice to the network operators
for once, and not force them to non-trivial expansions of their
network capacity?)

JSON is MUCH easier to parse than XML.  While, yes, our systems have
more raw processing power than ever before in our history, our systems
have been getting slower because everyone has been much more lax about
"we have lots of memory and processing power to play with".  This
includes the W3C.  This also includes the people who use the standards
promulgated by them.  This includes the people who have written the
parsers which have been created for use with XML.

I would vote that we should reduce the processing power required, so
that more simultaneous connections can be dealt with.  I would vote
that we should reduce the protocol overhead, so that it is easier to
process and transfer.  I would vote that we keep in mind that just
because there's more processing power to play with doesn't mean that
we should extend the sandbox beyond the very minimum necessary to make
it easy to play within.

And above all, I would vote that it be made a guiding principle in the
design to minimize resource impact -- both data storage (hard
disk/database) and data transmission (network).

-Kyle H

On Wed, May 14, 2008 at 5:55 AM, Justin Clark-Casey
<jjustincc at googlemail.com> wrote:
> Dr Scofield wrote:
>  > Stefan Andersson wrote:
>  >> Ok, next time, I'll half the dose of that cough syrup.
>  >>
>  >> Aaanyway : I still think it's a REALLY bad idea to put forward as a
>  >> 'standard region API' something that is more
>  >> application-and-environment specific (JSON) when there is something
>  >> that is less application-and-environment specific (XML)
>  > oops...this is not a standard region API, this is supposed to be a
>  > control API to allow us to control region specific information of an
>  > OpenSim instance.
>  >> Is there a concrete win with Json over Xml that I'm missing?
>  > i'd say, easy to parse from AJAX side, less verbose, doesn't require
>  > knowledge of the class definition to make sense of it (well, at least to
>  > some extent).
>
>   From what I've read of JSON, I don't believe it's tied to any specific
>  language - to some extent it just happens to be directly evaluatable in
>  Javascript since that was from where it originated.
>
>  However, I do personally prefer XML - there are just more tools and
>  libraries out there for it and it's more familiar.  I'm not sure the
>  possible advantages of JSON (e.g. less verbosity, though I'm not sure
>  that's an issue here) outweigh that.
>
>  Oh, and regarding LLSD - yes, let's confine that to linden interop and
>  not use it as our core REST format.
>
>
>  >>
>  >> And, it would be great if the api was formulated as a set of abstract
>  >> classes, interfaces and event hooks, so however you decide to
>  >> implement the protocol layer, somebody else could implement it
>  >> differently. (SOAP, anybody?)
>  > hmm...not sure whether i really want to add lots and lots of code around
>  > a simple control API.
>  >
>  > we've got a specific need in mind: control an OpenSim instance via REST
>  > instead of via console. REST appears to be resource/state centric,
>  > XmlRpc/SOAP/et al seems to be more object--method oriented.
>  >
>
>  I would agree.  Let's have a nice simple REST interface (alongside the
>  existing XMLRPC for now).  If we need to elaborate this can happen later
>  on...
>
>  --
>  justincc
>  Justin Clark-Casey
>  http://justincc.wordpress.com
>
>
> _______________________________________________
>  Opensim-dev mailing list
>  Opensim-dev at lists.berlios.de
>  https://lists.berlios.de/mailman/listinfo/opensim-dev
>