<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
color:black;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='color:black'>Good point. I will explore that more.<o:p></o:p></span></p><p class=MsoNormal><span style='color:black'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'> opensim-dev-bounces@opensimulator.org [mailto:opensim-dev-bounces@opensimulator.org] <b>On Behalf Of </b>Michael Emory Cerquoni<br><b>Sent:</b> Monday, March 16, 2015 10:13 AM<br><b>To:</b> opensim-dev@opensimulator.org<br><b>Subject:</b> Re: [Opensim-dev] UDP async performance<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>I am not sure the HTTP server handles UDP, I want to say it does not handle that.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Mon, Mar 16, 2015 at 11:50 AM, Myron Curtis <<a href="mailto:myronjc@virtualworldsgrid.com" target="_blank">myronjc@virtualworldsgrid.com</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal>Could this be a case of the Http server in opensim just being too slow to<br>respond to this traffic quickly enough?<o:p></o:p></p><div><div><p class=MsoNormal><br>-----Original Message-----<br>From: <a href="mailto:opensim-dev-bounces@opensimulator.org">opensim-dev-bounces@opensimulator.org</a><br>[mailto:<a href="mailto:opensim-dev-bounces@opensimulator.org">opensim-dev-bounces@opensimulator.org</a>] On Behalf Of Michael Heilmann<br>Sent: Monday, March 16, 2015 6:55 AM<br>To: <a href="mailto:opensim-dev@opensimulator.org">opensim-dev@opensimulator.org</a><br>Subject: [Opensim-dev] UDP async performance<br><br>Opensim Devs<br><br>I have been working on an external process that I hope to link to an opensim<br>plugin I am authoring. As a sanity check, I ran a simple socket test<br>against the components to test for obvious problems before I get to the<br>heavy lifting. This test was not meant to reveal anything, just to confirm<br>that the pieces are in place. However, I am not seeing the expected<br>behavior.<br><br>My simple sanity test: send 1000000 small udp packets at a server, and have<br>it count packets received. This is trivial, and the sending client is<br>python, so all packets should be received.<br><br>While running this on localhost: (~160K packets per second)<br>> Go, c++, nodejs, c# synchronous : negligible to no packet loss, as<br>> expected<br>><br>> c# async: 80% packet loss<br><br><br>I have reviewed msdn documentation, and went through the opensim code (as it<br>uses c# async) to clean up and run my tests. That 80% packet loss is after<br>forcing my threadpool and iocp thread counts up, and giving c# async extra<br>time before and after the test to warm up threadpools and process any<br>queues.<br><br>I have read that in some cases C# and .Net has a packet loss bug for udp on<br>localhost.<br><br>I re-ran this test between two linux servers with a 10gbe LAN interconnect<br>to remove any .net localhost packet loss issues, with the following results:<br>(~115K packets per second)<br>> c# synchronous: ~998K messages received (average over 3 runs) Go :<br>> ~980K messages received (average over 3 runs)<br>><br>> c# async: ~40K messages received (average over 3 runs)<br>disclosure: Go on my workstation is version 1.4.2, but on the servers is<br>version 1.2.1. Mono versions are identical<br><br>Now, the 3 runs were very nearly identical, with differences in the<br>thousands of packets. As before, C# async was given extra treatment to help<br>it along.<br><br>I am on linux 64 bit, so I had a coworker write his own version of this test<br>using Visual Studio 2013 on Windows (no code sharing), and he saw the same<br>behavior: c# async suffering massive packet loss while c# sync is keeping<br>up easily.<br><br>Is there anything in the opensim client stack that I am missing, that allows<br>for higher async performance? Or am I on a witch hunt, and this performance<br>is within Opensim bounds? I have been increasing the threadpool and iocp<br>threads as Opensim does. I have pasted important pieces of the code below,<br>as the entirety could be too long for this<br>messageboard: (The UDPPacketBuffer class is coped from opensim code)<br><br>> private void AsyncBeginReceive()<br>> {<br>> UDPPacketBuffer buf = new UDPPacketBuffer();<br>> this.u.BeginReceiveFrom(<br>> buf.Data,<br>> 0,<br>> UDPPacketBuffer.BUFFER_SIZE,<br>> SocketFlags.None,<br>> ref buf.RemoteEndPoint,<br>> AsyncEndReceive,<br>> buf);<br>> }<br>><br>> private void AsyncEndReceive(IAsyncResult iar)<br>> {<br>> //schedule another receive<br>> AsyncBeginReceive();<br>><br>> lock(this)<br>> {<br>> this.i++;<br>> }<br>> }<br>><br>> public static void Main (string[] args)<br>> {<br>> int iocpThreads;<br>> int workerThreads;<br>> ThreadPool.GetMinThreads(out workerThreads, out iocpThreads);<br>> workerThreads = 32;<br>> iocpThreads = 32;<br>> Console.WriteLine(workerThreads);<br>> ThreadPool.SetMinThreads(workerThreads,iocpThreads);<br>> MainClass mc = new MainClass();<br>> mc.AsyncBeginReceive();<br>> //manually trigger packet report after test run completion<br>> Console.ReadLine();<br>> Console.WriteLine(mc.i);<br>> }<br><br>I have run this test repeatedly, varying workerThread and iocpThreads from<br>8, to 16, 32, 64 ... 512, without any change in packets received.<br>If anyone had any insight into why this is suffering from so much packet<br>loss, I would appreciate it. Thanks.<br><br>--<br>Michael Heilmann<br>Research Associate<br>Institute for Simulation and Training<br>University of Central Florida<br><br>_______________________________________________<br>Opensim-dev mailing list<br><a href="mailto:Opensim-dev@opensimulator.org">Opensim-dev@opensimulator.org</a><br><a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><br><br>_______________________________________________<br>Opensim-dev mailing list<br><a href="mailto:Opensim-dev@opensimulator.org">Opensim-dev@opensimulator.org</a><br><a href="http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev" target="_blank">http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev</a><o:p></o:p></p></div></div></blockquote></div><p class=MsoNormal><br><br clear=all><br>-- <o:p></o:p></p><div><p class=MsoNormal>Michael Emory Cerquoni<o:p></o:p></p></div></div></div></body></html>