[Opensim-dev] Synchronous or Asynchronous.. Packet handler synchronicity

[Frisby, Adam]

Since we're defaulting to async - we should probably only make a list of what is known to run fast and what isn't.

Just as a quick example - all the terrain packets require some heavyish number crunching and should all be async.

Adam

> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
> bounces at lists.berlios.de] On Behalf Of Teravus Ovares
> Sent: Sunday, 13 December 2009 12:08 AM
> To: opensim-dev
> Subject: [Opensim-dev] Synchronous or Asynchronous.. Packet handler
> synchronicity
> 
> Hello Again
> 
> I just wanted to update the people subscribed to opensim-dev about the
> current situation with regards to packet handler synchronicity and
> thread context.
> 
> The History:
> Shortly after OpenSimulator started, it was decided that each user
> should have it's own packet processing thread.  This worked well at
> the time, but limited concurrency because of lock and thread
> contention.   5 UDP connections = 5 dedicated threads that spent most
> of their time waiting(this got expensive when child agents were
> considered).
> 
> Recently:
> A few months ago, Intel provided some test results and provided a few
> developers to re-write the UDP server and client handling stack.
> One of the things that they did was make each UDP server use two
> threads.  One for the incoming packet processing and one for the
> outgoing packet processing, regardless of how many clients were
> connected to OpenSimulator.    Essentially, this made it 'one thread
> per network connection resource'.   This was great because it reduced
> the amount of threads that opensimulator uses quite significantly.
> There were some problems though.  Some code handling the upper level
> processing (Scene, SceneObjectGroup, SceneObjectPart..  etc.. )  of
> the data in the packet took long enough to process..   to cause the
> simulator to slowly come to a crawl over time and disconnect users.
> 
> Because the infrastructure wasn't in place to selectively delegate
> which packets are handled asynchronously and synchronously, the choice
> was made to process each packet on the threadpool so that a long
> running packet processor wouldn't cause the UDP server to get
> backlogged and cause all users connected to get disconnected.   This
> solution wasn't really optimal..    because a single client, moving
> around, causes ~15 AgentUpdate packets to be sent to the server per
> second.      15 packets means that up to 15 threadpool requests were
> happening a second per connected root agent.       There are various
> reasons why this isn't optimal, I won't go into them here :).
> 
> Now:
> I'm happy to announce, that the infrastructure for selectively
> delegating whether a packet will be processed synchronously or
> asynchronously is in place as of
> 4ef8dc7d96fa2d4efd992ff7d304b8894f004c4f.   You can find the way to
> set a packet handler as synchronous in
> OpenSim.Region.ClientStack.LindenUDP.LLClientView.RegisterLocalPacketHa
> ndler.
>       Note that packet handlers with 3 parameters with the third one
> set to 'false' means that the packet handler is synchronous.    Packet
> handlers in that method with 2 parameters or 3 parameters and the
> third one set to 'true' means that the packet is processed
> asynchronously on the threadpool using the method defined by
> async_call_method in OpenSim.ini
> 
> I've also taken the liberty of setting the AgentUpdate Packet, the
> ViewerEffect Packet, and a few others as synchronous.   The
> AgentUpdate Packet and ViewerEffect Packet are the two packets that
> are sent from the client the most often, so, these alone should reduce
> the threads required per client significantly.
> 
> Testers:
> What users and testers should look for, however, is the Simulator
> suddenly disconnecting all users.    If this occurs, then, likely, the
> UDP server is not moving fast enough to effectively process packets
> for the current amount of users.
> 
> Developers:
> It's time to look at the length of time it takes for packet processors
> to finish processing and decide if the packet handler should be
> executed synchronously or asynchronously.
> Any synchronously processed packet is processed in the thread context
> beginning with the UDPServer.
> 
> Hopefully, this will allow us to increase the number of concurrently
> connected users further.
> 
> Regards
> 
> Teravus
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev