[Opensim-dev] ScriptServer communication

Sun Oct 14 15:05:28 UTC 2007

Thanks.

Good point on not locking it to LSL commands only.

I've started implementing a binary protocol for communication. It is the only way to get fast and scalable communication. Commands will be sent in packets like:

(Total packet length,) parameter count, command length, command name (string), param1 type, param1 length, param1 data, param2 type, param2 length, param2 data, etc.

This is what any RPC/Remoting has to go through anyway.

(should be noted that I have previous experience with tcp communication and binary protocols, so the implementation itself will be clean and will work)

But now I'm concerned about the data types in parameters... doing it this way I have to "hardcode" every data type (send/recv) and cast it from byte stream to the actual data type upon receive.

The only things I can think of except normal int/long/double/string/etc is vectors/rotation.

But feedback on this would be great.

If someone wants to share their thoughts on serialization too, go ahead.

Using a binary protocol will allow for efficient continuous command stream in both directions through a single TCP channel. At receiving endpoint the function with params will be checked against an interface, if found it will be cached for future execution (using a dictionary). Protocol will have a version negotiation at startup, so we can in the future improve communication by having remote party learn an ID of function and use that instead. Special packets can be used for function calls that don't contain strings, got no params, etc. (room for improvement is always good)

Also the feedback required, I think a script should be completely paused until it receives feedback .. because the reason why its not receiving it quickly enough is most likely communications delay or lack of CPU/resources on either ends - meaning something should wait. This will be a job for the microthreader.

For commands that require feedback we will use an ID in the packet. Server will respond with this ID in a packet. This way there is no blocking operation (in code) while waiting for reply (although script must be blocked).

Sincerely,

 Tedd

________________________________

Fra: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-bounces at lists.berlios.de] På vegne av Stefan Andersson
Sendt: 12. oktober 2007 15:50
Til: opensim-dev at lists.berlios.de
Emne: Re: [Opensim-dev] ScriptServer communication

Tedd,

first of all I'd recommend you to build your approach so this isn't tied hard to LSL; much could be gained by inserting a provider interface in there somewhere.

The 'expect immediate reply' is easier and harder than what one would think; but the easiest approach would be just to halt script execution until the reply comes back.

I'd suggest you do this by dividing your script executions into 'pieces of work' in such a way that the thread can easily yield if it encounters an synch operation (consider, for example, to do those using the asynch pattern) and let the current thread just go on with the next piece of work.

Also, I'd say that if you want to code proof, you'd halt the incoming requests until the next update sweep, as otherwise you'll have inconsistent data between sweeps (and between sequential reads)

Or, you could hook into the UpdateMovement / Simulate / Update chain to Perform Writes / Halt / Perform Reads. (having two request queues)

Also, you need to decide whether one script host serves many regions, or the opposite; allowing a many-to-many relation would in all probability create a lot of small updates and queries, but choosing a 1-to-many relation would give you an option to queue and send chunks of updates. On a side note, you'd really want some statistics on what events are the most common; world updates or script updates.

I still feel you will get a lot more bang for your buck by leaving the LSL event/object model and tailor something new.

It's a dead cool initiative though.

All the best,
/Stefan

________________________________

Date: Sat, 6 Oct 2007 02:08:59 +0200
From: tedd at konge.net
To: opensim-dev at lists.berlios.de
Subject: [Opensim-dev] ScriptServer communication

[23:15:56] <Tedd> ok... so I got this neato plan on getting stand alone ScriptEngine working, and making good progress...

[23:15:56] <Tedd> BUT - I need some feedback

[23:15:56] <Tedd> Communication between stand alone ScriptEngine and the Region... How to handle... here are a few keywords:

[23:15:56] <Tedd> * Event from region to ScriptEngine (to script) for create new script and for the 16 events like touch_start.

[23:15:56] <Tedd> * LSL command to manipulate something in-world from script to region

[23:15:56] <Tedd> * LSL command to query something in-world and have it returned as an LSL event

[23:15:56] <Tedd> nevermind local LSL commands like timers or http get, those will be handled in scriptengine internally

[23:15:56] <Tedd> * Now the problem: LSL Commands that query/set parameters in-world and expect imediate reply

[23:15:56] <Tedd> And of course:

[23:15:56] <Tedd> * Latency

[23:15:56] <Tedd> * Being able to put many commands (from one specific scriptengine to one specific region) into one request

I have been doing some thinking and had a good discussion with Tleiades.

Would like more inputs?

*	EVENTS: Region -> Script
*	FUNCTION CALL (no feedback): Script -> Region
*	FUNCTION CALL (feedback): Script -> Region

With 10.000 scripts running, would probably be nice with lazy write on network and sending an array of function calls/events at the time.

I'm thinking of going for a .Net Remoting with simple function calling (one by one) as a first implementation.

Sincerely,

 Tedd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20071014/5386d1c5/attachment-0001.html>