[Opensim-dev] Any Ideas??

Sat Nov 29 15:10:06 UTC 2014

I learnt from another source today that I made a mistake in commit 690f30c and wasn't actually logging the exception. 
I've corrected this in git master 432f0e8 so please update to this.

On 29/11/14 02:21, Butch Arnold wrote:
> Hi Justin,
>
> Thanks for the reply, sorry for the error on the links.
>
> The non working pic is at: http://3rdrockgrid.com/new/testpics/WelcomeNW.jpg
>
> I will check out the new code.
>
> Thanks again.
>
>
>
> On 11/28/2014 9:14 PM, Justin Clark-Casey wrote:
>> Hi Butch.  You actually linked to the 'ok' pic twice below.  It's good to see the actual output as that can sometimes
>> give extra clues to someone looking at the code as to what's going wrong.
>>
>> However, on reviewing the code, I believe it is possible for an exception in the script compilation process to
>> permanently stop all future compilation until the simulator is restarted.  I also believe any such exception would be
>> silently dropped and not logged.
>>
>> So to help identify the issue, I have both logged any exception and changed the code to increase the chance that
>> future script compilation will still happen even if something does go wrong. This is in commit 690fe0c in master.
>>
>> If you see an exception logged with this code then please report. If the problem still occurs with no extra log clues,
>> then my next hypothesis would be that compilation is hanging indefinitely, possibly due to some Mono bug.  In this
>> case, the ideal thing to do would be to take a thread dump so we can see if a thread is stuck.  On Mono I have just
>> documented the process at [1]. However, I suspect you're on Windows and I'm not sure how one would do it there (or
>> even if that kind of thing is possible).
>>
>> If we find the bug in master a fix can be backported to 0.8-post-fixes branch.
>>
>> [1] http://opensimulator.org/wiki/Debugging#Thread_Dumps
>>
>> On 28/11/14 16:46, Butch Arnold wrote:
>>> Hello Justin,
>>>
>>> Thank you.
>>>
>>> A little more info...
>>> It seems not "All" scripts stop working... those which were running will remain running, but any that should start
>>> "After" scripts break will not start - instead they are shown in the "xengine status" report from the console as
>>> "Scripts Waiting For Load" with the number of scripts waiting to load... see screen captures here:
>>>
>>> Working fine:
>>> This is after a fresh restart of the instance....
>>> http://3rdrockgrid.com/new/testpics/WelcomeWorking.jpg
>>>
>>> This is immediately after we were able to break the same "Fresh Restart" from above....
>>> Broken:
>>> http://3rdrockgrid.com/new/testpics/WelcomeWorking.jpg
>>>
>>> This seems to happen on any of our instances as it should since they are all configured the same.
>>>
>>> When I upgraded, I did not use anything from the previous grid setup... ie.. I did not "Merge files" - I built a new
>>> grid beside the old one in case I needed to roll back to the old.
>>>
>>> Everything is fresh in this setup.
>>> I've installed a new database, but reloaded with a dump from the old one and new folders with no previously compiled
>>> scripts.
>>> Our setup recompiles scripts on every instance start.
>>>
>>> I thought maybe one of our custom .dll's were to blame so I removed them, running only "Bone Stock" OpenSim and still
>>> have the same issue.
>>>
>>>   - I have made the changes you've suggested with the same results.
>>>
>>>   - I have upgraded to the recent Ghost/Trunk merge 2 days ago with the same results.
>>>
>>>   - I have tried "Stopping and Restarting" scripts from the console when the issue starts... same results.
>>>
>>>   - I have tried "Suspending and Resuming" scripts from the console when the issue starts... same results.
>>>
>>>   - The only way to get scripts running correctly again is to restart the OpenSim instance.
>>>
>>> It seems a 32bit OpenSim instance will break instantly... ie we have found a combination which reliably crashes the
>>> instance each time in 32bit mode.
>>> If running a 32bit instance, any time a HyperGrid visitor arrives it causes the issue.
>>>
>>> In 64bit mode, we have not yet identified an event, a chain of events which cause the issue.
>>> Last evening, I made some changes and restarted the region in 64bit mode, we tried several times to break scripts and
>>> was unable to, so we were hopeful things were fixed.
>>> We left the region run overnight and this morning I checked and scripts were broken again.
>>> Logs do not indicate any issues.
>>>
>>> We are still trying to find the combination which will break scripts every time in 64bit mode.
>>>
>>> You can see it in action for yourself at:  (providing it isn't already broken)
>>> grid.3rdrockgrid.com:8002:welcome
>>>
>>> We have a "Rez Zone" there where you can rez objects...
>>>
>>>
>>> Our machine is running Windows Server 2008 R2 64bit.
>>>
>>> I have tried numerous configurations, but currently I am running:
>>> [BulletSim]
>>>      UseSeparatePhysicsThread = true
>>>      TerrainImplementation=0
>>>      AvatarToAvatarCollisionsByDefault = true
>>>
>>> [XEngine]
>>>      ;# {Enabled} {} {Enable the XEngine scripting engine?} {true false} true
>>>      ;; Enable this engine in this OpenSim instance
>>>      Enabled = true
>>>
>>>      ;; How many threads to keep alive even if nothing is happening
>>>      MinThreads = 2
>>>
>>>      ;; How many threads to start at maximum load
>>>      MaxThreads = 250
>>>
>>>     IdleTimeout = 60
>>>
>>>      Priority = "AboveNormal"
>>>
>>>      MaxScriptEventQueue = 300
>>>
>>>      ThreadStackSize = 462144
>>>
>>>     AppDomainLoading = true
>>>
>>>      ScriptStopStrategy = co-op
>>>
>>>      DeleteScriptsOnStartup = true
>>>
>>>      DefaultCompileLanguage = "lsl"
>>>
>>>      AllowedCompilers = "lsl"
>>>
>>>      CompileWithDebugInformation = true
>>>
>>>      AllowMODFunctions = false
>>>
>>>     AllowOSFunctions = true
>>>
>>>      Allow_osTeleportAgent = true
>>>      Allow_osAvatarName2Key = true
>>>      Allow_osNpcSay = true
>>>
>>>      AllowLightShareFunctions = false
>>>
>>>     OSFunctionThreatLevel = VeryHigh
>>>
>>>     EventLimit = 1800
>>>
>>>      KillTimedOutScripts = true
>>>
>>>      ScriptDelayFactor = 1.0
>>>
>>>      ScriptDistanceLimitFactor = 1.0
>>>
>>>      NotecardLineReadCharsMax = 255
>>>
>>>     SensorMaxRange = 96.0
>>>      SensorMaxResults = 16
>>>
>>>     DisableUndergroundMovement = true
>>>
>>>      ScriptEnginesPath = "ScriptEngines"
>>>
>>> Thank you.
>>>
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at opensimulator.org
>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>>
>>
>>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>

-- 
Justin Clark-Casey (justincc)
OSVW Consulting
http://justincc.org
http://twitter.com/justincc