I may be crazy, but I swear I'm seeing an immediate and very noticeable improvement in performance from the end user perspective. Avatar movement not as sluggish, almost snappy even, textures rezzing more quickly.. <div>
<br></div><div>The CPU usage is about the same as before the split (57-59% on average with no activity) but the Page File usage is higher. I'll be interested to see how things go when the HGAC visits tomorrow to give a good test with higher concurrency.</div>
<div><br></div><div>Thanks again!</div><div><br></div><div>- Chris/Fleep</div><div><br></div><div><br></div><div>Chris M. Collins (SL/OS: Fleep Tuque)</div><div>Center for Simulations & Virtual Environments Research (UCSIM)</div>
<div>UCIT Instructional & Research Computing</div><div>University of Cincinnati</div><div>406A Zimmer Hall</div><div>315 College Drive</div><div>PO BOX 210088</div><div>Cincinnati, OH 45221-0088</div><div><a href="mailto:chris.collins@uc.edu">chris.collins@uc.edu</a></div>
<div>(513) 556-3018</div><div><br></div><div><a href="http://ucsim.uc.edu">http://ucsim.uc.edu</a><br><div><br></div><div><br><div><br><div class="gmail_quote">On Mon, Feb 20, 2012 at 7:28 AM, Gary Banham <span dir="ltr"><<a href="mailto:banham1@telus.net">banham1@telus.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Your more than welcome<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">TC<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gary<u></u><u></u></span></p>
<p class="MsoNormal"><a name="1359abcad49f3b84__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></a></p><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:opensim-users-bounces@lists.berlios.de" target="_blank">opensim-users-bounces@lists.berlios.de</a> [mailto:<a href="mailto:opensim-users-bounces@lists.berlios.de" target="_blank">opensim-users-bounces@lists.berlios.de</a>] <b>On Behalf Of </b>Fleep Tuque<br>
<b>Sent:</b> Monday, February 20, 2012 5:26 AM</span></p><div><div class="h5"><br><b>To:</b> <a href="mailto:opensim-users@lists.berlios.de" target="_blank">opensim-users@lists.berlios.de</a><br><b>Subject:</b> Re: [Opensim-users] Intermittent crashing - System.OutofMemoryException<u></u><u></u></div>
</div><p></p></div><div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Wow, it really was that simple. I successfully split off a few regions into a separate instance on FleepGrid with no trouble at all. <u></u><u></u></p>
<div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">The key, as Gary mentioned, was uncommenting the http_listener_port line (267 in my opensim.ini file) and incrementing the TCP port (in my case to 9001) for the new instance.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Thanks Gary! I'll have to work out how to split the regions up logically for our UCSIM grid but I'm surprised it's that simple! Yay! :)<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">- Chris/Fleep<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div>
<div><p class="MsoNormal">Chris M. Collins (SL/OS: Fleep Tuque)<u></u><u></u></p></div><div><p class="MsoNormal">Center for Simulations & Virtual Environments Research (UCSIM)<u></u><u></u></p></div><div><p class="MsoNormal">
UCIT Instructional & Research Computing<u></u><u></u></p></div><div><p class="MsoNormal">University of Cincinnati<u></u><u></u></p></div><div><p class="MsoNormal">406A Zimmer Hall<u></u><u></u></p></div><div><p class="MsoNormal">
315 College Drive<u></u><u></u></p></div><div><p class="MsoNormal">PO BOX 210088<u></u><u></u></p></div><div><p class="MsoNormal">Cincinnati, OH 45221-0088<u></u><u></u></p></div><div><p class="MsoNormal"><a href="mailto:chris.collins@uc.edu" target="_blank">chris.collins@uc.edu</a><u></u><u></u></p>
</div><div><p class="MsoNormal"><a href="tel:%28513%29%20556-3018" value="+15135563018" target="_blank">(513) 556-3018</a><u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt">
<a href="http://ucsim.uc.edu" target="_blank">http://ucsim.uc.edu</a><u></u><u></u></p><div><p class="MsoNormal">On Mon, Feb 20, 2012 at 6:48 AM, Gary Banham <<a href="mailto:banham1@telus.net" target="_blank">banham1@telus.net</a>> wrote:<u></u><u></u></p>
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">PS</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">just an additional note I think the problem is the single instance of Opensim will only access 2 gigs of memory </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">with a large number of regions and lots of building and scripts the single instance can't count that high it hits a wall</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">by placing regions over a number of instances each can use 2 gigs</span><u></u><u></u></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">so yes you use more memory EG 5 instances could potentially use 10 gigs of memory but one instance only address 2 Gigs</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">TC</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gary</span><u></u><u></u></p><p class="MsoNormal"><a name="1359abcad49f3b84_1359a979bf408a91__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></a><u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:opensim-users-bounces@lists.berlios.de" target="_blank">opensim-users-bounces@lists.berlios.de</a> [mailto:<a href="mailto:opensim-users-bounces@lists.berlios.de" target="_blank">opensim-users-bounces@lists.berlios.de</a>] <b>On Behalf Of </b>Fleep Tuque</span><u></u><u></u></p>
<div><p class="MsoNormal"><br><b>Sent:</b> Monday, February 20, 2012 4:34 AM<br><b>To:</b> <a href="mailto:opensim-users@lists.berlios.de" target="_blank">opensim-users@lists.berlios.de</a><u></u><u></u></p></div><p class="MsoNormal">
<b>Subject:</b> Re: [Opensim-users] Intermittent crashing - System.OutofMemoryException<u></u><u></u></p></div><div><div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">I'm worried that you may be right about that and it's just a matter of time before adding more content becomes a problem, in which case it sounds like perhaps instead of running all the regions on a single opensim instance I should split them up across several as Gary mentioned above?<u></u><u></u></p>
<div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">I'm not sure exactly how that works, I guess, and I'm trying to understand how that would use less memory instead of more.. Would I simply copy the contents of the opensim directory to multiple locations and break the regions up in the regions.ini file or..? <u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">I'd like to experiment with it on FleepGrid first before I hose our campus install, but the HGAC group is coming for a visit in a couple days so if anyone has pointers about how to do that splitting up process before I break anything, I'm all ears. :)<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">- Chris/Fleep<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div>
<div><p class="MsoNormal">Chris M. Collins (SL/OS: Fleep Tuque)<u></u><u></u></p></div><div><p class="MsoNormal">Center for Simulations & Virtual Environments Research (UCSIM)<u></u><u></u></p></div><div><p class="MsoNormal">
UCIT Instructional & Research Computing<u></u><u></u></p></div><div><p class="MsoNormal">University of Cincinnati<u></u><u></u></p></div><div><p class="MsoNormal">406A Zimmer Hall<u></u><u></u></p></div><div><p class="MsoNormal">
315 College Drive<u></u><u></u></p></div><div><p class="MsoNormal">PO BOX 210088<u></u><u></u></p></div><div><p class="MsoNormal">Cincinnati, OH 45221-0088<u></u><u></u></p></div><div><p class="MsoNormal"><a href="mailto:chris.collins@uc.edu" target="_blank">chris.collins@uc.edu</a><u></u><u></u></p>
</div><div><p class="MsoNormal"><a href="tel:%28513%29%20556-3018" target="_blank">(513) 556-3018</a><u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal"><a href="http://ucsim.uc.edu" target="_blank">http://ucsim.uc.edu</a><u></u><u></u></p>
</div><div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"> <u></u><u></u></p><div><p class="MsoNormal">On Thu, Feb 16, 2012 at 10:19 PM, Justin Clark-Casey <<a href="mailto:jjustincc@googlemail.com" target="_blank">jjustincc@googlemail.com</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">I would actually be rather surprised if errors during the OAR import were related to your memory problems. I think that it's more likely that the content of the OAR in some way pushed you beyond memory boundaries and wiping it has reset things.<br>
<br>Still, if the problem is solved then that's what matters :)<u></u><u></u></p><div><div><p class="MsoNormal"><br><br>On 16/02/12 22:50, Fleep Tuque wrote:<u></u><u></u></p></div></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div><div><p class="MsoNormal">Hi all,<br><br>After going through the log files and trying to figure out what changed in the time period when the out of memory errors<br>began, we traced it back to an OAR file import. (Thanks to whoever suggested that might be an issue earlier in this<br>
thread!)<br><br>I'd ported over a copy of the FleepGrid Plaza region so my students could easily grab some of the freebies I'd created,<br>and it looks like something got horked during the import to the UCSIM grid:<br>
<br>012-02-07 09:12:05,991 ERROR - Flotsam.RegionModules.AssetCache.FlotsamAssetCache [FLOTSAM ASSET CACHE]: Failed to<br>update cache for asset 3d0b3a52-40e5-48fe-8595-b72e163917fa. Exception The process cannot access the file<br>
'c:\Opensim\bin\assetcache\3d0\3d0b3a52-40e5-48fe-8595-b72e163917fa' because it is being used by another process. at<br>System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)<br> at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights,<br>
FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)<br> at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)<br>
at System.IO.File.OpenFile(String path, FileAccess access, SafeFileHandle& handle)<br> at System.IO.File.SetLastAccessTimeUtc(String path, DateTime lastAccessTimeUtc)<br> at Flotsam.RegionModules.AssetCache.FlotsamAssetCache.UpdateFileCache(String key, AssetBase asset)<br>
<br>Shortly after that, we begin to see the first out of memory errors..<br><br>2012-02-07 09:15:17,453 DEBUG - OpenSim.Region.ScriptEngine.XEngine.XEngine [XEngine] Loaded script LK-Stool-Sit<br>#2.poseball-zero-lag, script UUID e73e1114-297a-43a7-b4b8-dd423cd83b95, prim UUID 2ae59628-943d-49ec-9673-ee3465bd0242 @<br>
<168.1464, 156.7953, 26.90887>.UCSIM Staff 1<br>2012-02-07 09:15:17,984 ERROR - OpenSim.Region.ScriptEngine.XEngine.XEngine [XEngine] Exception creating app domain:<br> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.<br>
at System.AppDomain.nCreateDomain(String friendlyName, AppDomainSetup setup, Evidence providedSecurityInfo, Evidence<br>creatorsSecurityInfo, IntPtr parentSecurityDescriptor)<br> at System.AppDomain.CreateDomain(String friendlyName, Evidence securityInfo, AppDomainSetup info)<br>
at OpenSim.Region.ScriptEngine.XEngine.XEngine.DoOnRezScript(Object[] parms)<br><br>And we see those out of memory errors stack up until eventually the application crashed..<br><br>When we realized it was probably the OAR file that was the source of the issue, I wiped the region by importing a fresh<br>
"blank" OAR and we haven't had the out of memory exception since.<br><br>Looks like the problem is resolved and super thanks again to everyone who helped with pointers and info in this thread -<br>much appreciated!<br>
<br>- Chris/Fleep<br><br><br>Chris M. Collins (SL/OS: Fleep Tuque)<br>Center for Simulations & Virtual Environments Research (UCSIM)<br>UCIT Instructional & Research Computing<br>University of Cincinnati<br>406A Zimmer Hall<br>
315 College Drive<br>PO BOX 210088<br>Cincinnati, OH 45221-0088<u></u><u></u></p></div></div><p class="MsoNormal"><a href="mailto:chris.collins@uc.edu" target="_blank">chris.collins@uc.edu</a> <mailto:<a href="mailto:chris.collins@uc.edu" target="_blank">chris.collins@uc.edu</a>><br>
<a href="tel:%28513%29%20556-3018" target="_blank">(513) 556-3018</a><br><br><a href="http://ucsim.uc.edu" target="_blank">http://ucsim.uc.edu</a><u></u><u></u></p><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br><br>On Thu, Feb 16, 2012 at 2:16 PM, R.Gunther <<a href="mailto:rigun@rigutech.nl" target="_blank">rigun@rigutech.nl</a> <mailto:<a href="mailto:rigun@rigutech.nl" target="_blank">rigun@rigutech.nl</a>>> wrote:<br>
<br> There's a 3e option. Munin.<br> <a href="http://munin-monitoring.org/" target="_blank">http://munin-monitoring.org/</a><br><br><br> On 2012-02-16 16:49, BlueWall wrote:<br><br> The rrdtools is very good for storing and organizing the data for statistics and it will also produce graphs<br>
from that data. It relies on external scripts to collect the data - that's where Cacti comes in. It has the<br> scripts and templates to graph your system (and snmp devices) in the distribution. I have done it on Linux and<br>
they claim to work on Windows too.<br><br> You could also write scripts and templates to graph OpenSim. I haven't tried that, though. There is<br> documentation included in the distro, and scattered across the Cacti user forums.<br>
<br> -BlueWall<br><br> On 02/16/2012 09:55 AM, Rick Anderson wrote:<br><br> One question about Cacti, and RDDTools. The sites describe them as<br> general purpose graphing tools. Is there any documentation or tutorial<br>
for configuring them to monitor system status. Maybe, it's really<br> simple, and I'm over complicating it.<br><br> -_Rick<u></u><u></u></p></div><p class="MsoNormal"> _________________________________________________<u></u><u></u></p>
<div><p class="MsoNormal"><br> Opensim-users mailing list<br> <a href="mailto:Opensim-users@lists.berlios.de" target="_blank">Opensim-users@lists.berlios.de</a> <mailto:<a href="mailto:Opensim-users@lists.berlios.de" target="_blank">Opensim-users@lists.berlios.de</a>><u></u><u></u></p>
</div><p class="MsoNormal"> <a href="https://lists.berlios.de/__mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/__mailman/listinfo/opensim-users</a> <<a href="https://lists.berlios.de/mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-users</a>><u></u><u></u></p>
<div><p class="MsoNormal"><br><br><br><br><br>_______________________________________________<br>Opensim-users mailing list<br><a href="mailto:Opensim-users@lists.berlios.de" target="_blank">Opensim-users@lists.berlios.de</a><br>
<a href="https://lists.berlios.de/mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-users</a><u></u><u></u></p></div></blockquote><p class="MsoNormal"> <u></u><u></u></p><div>
<p class="MsoNormal"><br>-- <br>Justin Clark-Casey (justincc)<br><a href="http://justincc.org/blog" target="_blank">http://justincc.org/blog</a><br><a href="http://twitter.com/justincc" target="_blank">http://twitter.com/justincc</a><u></u><u></u></p>
</div><div><div><p class="MsoNormal">_______________________________________________<br>Opensim-users mailing list<br><a href="mailto:Opensim-users@lists.berlios.de" target="_blank">Opensim-users@lists.berlios.de</a><br><a href="https://lists.berlios.de/mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-users</a><u></u><u></u></p>
</div></div></div><p class="MsoNormal"> <u></u><u></u></p></div></div></div></div></div></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>_______________________________________________<br>Opensim-users mailing list<br>
<a href="mailto:Opensim-users@lists.berlios.de" target="_blank">Opensim-users@lists.berlios.de</a><br><a href="https://lists.berlios.de/mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-users</a><u></u><u></u></p>
</div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div><br>_______________________________________________<br>
Opensim-users mailing list<br>
<a href="mailto:Opensim-users@lists.berlios.de">Opensim-users@lists.berlios.de</a><br>
<a href="https://lists.berlios.de/mailman/listinfo/opensim-users" target="_blank">https://lists.berlios.de/mailman/listinfo/opensim-users</a><br>
<br></blockquote></div><br></div></div></div>