<div dir="ltr">I am not sure the HTTP server handles UDP, I want to say it does not handle that.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 16, 2015 at 11:50 AM, Myron Curtis <span dir="ltr"><<a href="mailto:myronjc@virtualworldsgrid.com" target="_blank">myronjc@virtualworldsgrid.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Could this be a case of the Http server in opensim just being too slow to<br>
respond to this traffic quickly enough?<br>
<div class="HOEnZb"><div class="h5"><br>
-----Original Message-----<br>
From: <a href="mailto:opensim-dev-bounces@opensimulator.org">opensim-dev-bounces@opensimulator.org</a><br>
[mailto:<a href="mailto:opensim-dev-bounces@opensimulator.org">opensim-dev-bounces@opensimulator.org</a>] On Behalf Of Michael Heilmann<br>
Sent: Monday, March 16, 2015 6:55 AM<br>
To: <a href="mailto:opensim-dev@opensimulator.org">opensim-dev@opensimulator.org</a><br>
Subject: [Opensim-dev] UDP async performance<br>
<br>
Opensim Devs<br>
<br>
I have been working on an external process that I hope to link to an opensim<br>
plugin I am authoring. As a sanity check, I ran a simple socket test<br>
against the components to test for obvious problems before I get to the<br>
heavy lifting. This test was not meant to reveal anything, just to confirm<br>
that the pieces are in place. However, I am not seeing the expected<br>
behavior.<br>
<br>
My simple sanity test: send 1000000 small udp packets at a server, and have<br>
it count packets received. This is trivial, and the sending client is<br>
python, so all packets should be received.<br>
<br>
While running this on localhost: (~160K packets per second)<br>
> Go, c++, nodejs, c# synchronous : negligible to no packet loss, as<br>
> expected<br>
><br>
> c# async: 80% packet loss<br>
<br>
<br>
I have reviewed msdn documentation, and went through the opensim code (as it<br>
uses c# async) to clean up and run my tests. That 80% packet loss is after<br>
forcing my threadpool and iocp thread counts up, and giving c# async extra<br>
time before and after the test to warm up threadpools and process any<br>
queues.<br>
<br>
I have read that in some cases C# and .Net has a packet loss bug for udp on<br>
localhost.<br>
<br>
I re-ran this test between two linux servers with a 10gbe LAN interconnect<br>
to remove any .net localhost packet loss issues, with the following results:<br>
(~115K packets per second)<br>
> c# synchronous: ~998K messages received (average over 3 runs) Go :<br>
> ~980K messages received (average over 3 runs)<br>
><br>
> c# async: ~40K messages received (average over 3 runs)<br>
disclosure: Go on my workstation is version 1.4.2, but on the servers is<br>
version 1.2.1. Mono versions are identical<br>
<br>
Now, the 3 runs were very nearly identical, with differences in the<br>
thousands of packets. As before, C# async was given extra treatment to help<br>
it along.<br>
<br>
I am on linux 64 bit, so I had a coworker write his own version of this test<br>
using Visual Studio 2013 on Windows (no code sharing), and he saw the same<br>
behavior: c# async suffering massive packet loss while c# sync is keeping<br>
up easily.<br>
<br>
Is there anything in the opensim client stack that I am missing, that allows<br>
for higher async performance? Or am I on a witch hunt, and this performance<br>
is within Opensim bounds? I have been increasing the threadpool and iocp<br>
threads as Opensim does. I have pasted important pieces of the code below,<br>
as the entirety could be too long for this<br>
messageboard: (The UDPPacketBuffer class is coped from opensim code)<br>
<br>
> private void AsyncBeginReceive()<br>
> {<br>
> UDPPacketBuffer buf = new UDPPacketBuffer();<br>
> this.u.BeginReceiveFrom(<br>
> buf.Data,<br>
> 0,<br>
> UDPPacketBuffer.BUFFER_SIZE,<br>
> SocketFlags.None,<br>
> ref buf.RemoteEndPoint,<br>
> AsyncEndReceive,<br>
> buf);<br>
> }<br>
><br>
> private void AsyncEndReceive(IAsyncResult iar)<br>
> {<br>
> //schedule another receive<br>
> AsyncBeginReceive();<br>
><br>
> lock(this)<br>
> {<br>
> this.i++;<br>
> }<br>
> }<br>
><br>
> public static void Main (string[] args)<br>
> {<br>
> int iocpThreads;<br>
> int workerThreads;<br>
> ThreadPool.GetMinThreads(out workerThreads, out iocpThreads);<br>
> workerThreads = 32;<br>
> iocpThreads = 32;<br>
> Console.WriteLine(workerThreads);<br>
> ThreadPool.SetMinThreads(workerThreads,iocpThreads);<br>
> MainClass mc = new MainClass();<br>
> mc.AsyncBeginReceive();<br>
> //manually trigger packet report after test run completion<br>
> Console.ReadLine();<br>
> Console.WriteLine(mc.i);<br>
> }<br>
<br>
I have run this test repeatedly, varying workerThread and iocpThreads from<br>
8, to 16, 32, 64 ... 512, without any change in packets received.<br>
If anyone had any insight into why this is suffering from so much packet<br>
loss, I would appreciate it. Thanks.<br>
<br>
--<br>
Michael Heilmann<br>
Research Associate<br>
Institute for Simulation and Training<br>
University of Central Florida<br>
<br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@opensimulator.org">Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
<br>
_______________________________________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@opensimulator.org">Opensim-dev@opensimulator.org</a><br>
<a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">Michael Emory Cerquoni</div>
</div>