<div class="gmail_extra">Hi Justin, <br><br>I am still investigating the Slow GetTexture problem and the resulting instability under high Load.<br><br>What i did so far: <br><br><ol><li>I'm still using opensim-0007711 ( i didn't have the time to upgrade, the first upgrade was not so good due an error on my side and since then i did not look too much into it)</li>
<li>I've created a Windows Instance in the Amazon cloud in order to be able to connect some profiling tools.</li><li>I've run the last two Friday Parties from there the first Party was quite okay ( MaxPoolThreads=90 in the SmartPoolThreads settings, but i saw more on peaks, strange )</li>
<li>The second party from the 20. April went crazy after 3 hours here's a picture:</li></ol><p><a href="http://farm9.staticflickr.com/8017/6957771466_4412ee83c4_b_d.jpg">http://farm9.staticflickr.com/8017/6957771466_4412ee83c4_b_d.jpg</a></p>
<p>Most of the threads had a stack trace like this:</p><p><a href="http://farm9.staticflickr.com/8155/6957875232_0203631ed0_b_d.jpg">http://farm9.staticflickr.com/8155/6957875232_0203631ed0_b_d.jpg</a></p><p>Wondering why this increase started after approx 3 hours. We had at max about 18 avies on the region/sim with various different viewers. Because I did not attach this amazon Cloud instance to my splunk server i have no statistics about the viewers during the party ... i probably should do that in future. <br>
</p><p>I will upgrade to a more recent version next week ... <br></p><p>Thanks a lot!</p><p>Akira<br></p><p><br></p><div class="gmail_quote">Am 19. März 2012 01:57 schrieb Justin Clark-Casey <span dir="ltr"><<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It's quite possible the 3rd party HTTP server doesn't use the threadpool though I've never looked in detail.<br>
<br>
You could supply any other http address in the GetTexture cap (e.g. the asset service directly with a suitable handler). However, I'm not sure that asset serving is such a bottleneck at the moment compared with scripting and physics issues.<div class="im">
<br>
<br>
On 17/03/12 21:10, Dahlia Trimble wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
I've done a bit of tracing through the code and I can't seem to find where the http server in OpenSimulator uses<br>
threadpool threads. I did find them used in the LLUDP server and in asyncronous requests from the asset service, but I<br>
have yet to find any other uses. is it possible that the http server is still using system threads, bypassing the<br>
threadpool? I'm rather curious as I use the built-in http server in a few personal applications and I'm concerned about<br>
performance.<br>
<br>
On another note, I believe part of the impetus behind LL designing the texture fetch capability was to allow a separate<br>
service from the simulator to supply assets to viewers, thereby reducing load on the individual simulator processes.<br>
Perhaps this is something OpenSimulator can take advantage of? Probably some kind of asset proxy cache could do a much<br>
better job of serving textures and other assets to viewers than the existing monolithic process? I believe it could even<br>
be moved to a separate server with a different IP address.<br>
<br>
<br></div><div class="im">
On Fri, Mar 16, 2012 at 8:36 PM, Justin Clark-Casey <<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.com</a> <mailto:<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.<u></u>com</a>>> wrote:<br>
<br>
Hi Akira. I have now updated the "show threads" method to show threadpool statistics for the main threadpool.<br>
Please note that each XEngine script engine will also have it's own threadpool (which can be seen using the<br>
"xengine status" command). Might need to improve this further.<br>
<br>
"show threads" will also show all in-use threads. However, at least on mono 2.6.7 this isn't reported by the VM so<br>
won't be shown.<br>
<br>
I'm not sure whether this will help you or not in tracking down performance issues. In some situations it could<br>
help (e.g. if threads are encountering deadlock the number of 'in use' threads will leap up, though you've probably<br>
already noticed deadlock by the long-running threads reporting monitoring failures and the sim locking up).<br>
<br>
So I'd be happy to hear suggestions for additional data and I'll implement them if I can, since I think this is<br>
going to be a growing area of concern. Unfortunately pinning down performance issues with a system as complex as<br>
OpenSimulator (with massive numbers of threads and user generated scripts) is likely to remain a significant<br>
challenge for the forseeable future.<br>
<br>
<br>
On 11/03/12 19:15, Akira Sonoda wrote:<br>
<br>
Am 10. März 2012 03:25 schrieb Justin Clark-Casey <<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.com</a> <mailto:<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.<u></u>com</a>><br>
</div>
<mailto:<a href="mailto:jjustincc@googlemail." target="_blank">jjustincc@googlemail.</a>_<u></u>_com <mailto:<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.<u></u>com</a>>>>:<div class="im">
<br>
<br>
<br>
I'm sorry to say that you'll have to take the ThreadPool numbers with a very very very large pinch of salt. I<br>
believe they only refer to the built-in mono thread pool and not the SmartThreadPool which is the one<br>
actually used<br>
(and beyond that the core simulator and xengine use separate pools). I will try and improve this situation<br>
soon.<br>
<br>
<br>
Thank you Justin!<br>
<br>
Would be nice to have some meaningful statistics for all those ThreadPools! Maybe there is a possibility to<br>
write those<br>
statistics to the log from time to time ( e.g. every 30 seconds). Together with some documented "best practices"<br>
from<br>
those who operate Sims, with lots of avatars on it - I'm thinking mainly the OSgrid Plazas are good references -<br>
this<br>
could be highly valuable information for those who operate Sims for similar purposes ( meeting points, parties,<br>
concerts<br>
etc. )<br>
<br>
<br>
<br>
<br>
<br>
<br></div>
______________________________<u></u>___________________<div class="im"><br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.berlios.de</a> <mailto:<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.<u></u>berlios.de</a>><br>
</div>
<a href="https://lists.berlios.de/__mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/__<u></u>mailman/listinfo/opensim-dev</a> <<a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/<u></u>mailman/listinfo/opensim-dev</a>><div class="im">
<br>
<br>
<br>
<br>
--<br>
Justin Clark-Casey (justincc)<br>
<a href="http://justincc.org/blog" target="_blank">http://justincc.org/blog</a><br>
<a href="http://twitter.com/justincc" target="_blank">http://twitter.com/justincc</a><br></div>
______________________________<u></u>___________________<div class="im"><br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.berlios.de</a> <mailto:<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.<u></u>berlios.de</a>><br></div>
<a href="https://lists.berlios.de/__mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/__<u></u>mailman/listinfo/opensim-dev</a> <<a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/<u></u>mailman/listinfo/opensim-dev</a>><div class="im">
<br>
<br>
<br>
<br>
<br>
______________________________<u></u>_________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.berlios.de</a><br>
<a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/<u></u>mailman/listinfo/opensim-dev</a><br>
</div></blockquote><div class="HOEnZb"><div class="h5">
<br>
<br>
-- <br>
Justin Clark-Casey (justincc)<br>
<a href="http://justincc.org/blog" target="_blank">http://justincc.org/blog</a><br>
<a href="http://twitter.com/justincc" target="_blank">http://twitter.com/justincc</a><br>
______________________________<u></u>_________________<br>
Opensim-dev mailing list<br>
<a href="mailto:Opensim-dev@lists.berlios.de" target="_blank">Opensim-dev@lists.berlios.de</a><br>
<a href="https://lists.berlios.de/mailman/listinfo/opensim-dev" target="_blank">https://lists.berlios.de/<u></u>mailman/listinfo/opensim-dev</a><br>
</div></div></blockquote></div><br></div>