XEngine

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
m (Typo correction "never" -> "newer")
 
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
+
{{Quicklinks}}
{{Template:Quicklinks}}
+
 
<br />
 
<br />
  
== The XEngine Script Engine ==
+
=Summary=
  
 
This is a fully functional, full featured script engine which supports persistent script states and script state serialization.
 
This is a fully functional, full featured script engine which supports persistent script states and script state serialization.
  
This engine is not enabled by default. To enable it, add the following line to your OpenSim.ini [Startup] section:
+
This engine is enabled by default.
  
DefaultScriptEngine = "XEngine"
+
For each unique script asset, XEngine will create up to 4 files.  These are:
  
You may have a similar line, but referencing the "ScriptEngine.DotNetEngine" there. If so, please comment it, since two script engines can not be run in parallel.
+
* '''CommonCompiler_compiled_<script-asset-id>.dll''', the actual executable DLL for that script asset.
Also, newer revisions will have that line already in place, but commented. Simply uncomment it in this case.
+
* On mono, '''CommonCompiler_compiled_<script-asset-id>.mdb''', the debug data file (necessary for resolving line numbers).
 +
* On Windows, '''CommonCompiler_compiled_<script-asset-id>.pdb''', the debug data file (necessary for resolving line numbers).
 +
* '''CommonCompiler_source_CommonCompiler_compiled_<script-asset-id>.cs''', if WriteScriptSourceToDebugFile = true in [XEngine] in OpenSim.ini.  This is the C# source file as translated from the source language (e.g. from LSL) before compilation.
  
At this time, you can delete the compiled scripts from your ScriptEngines directory, because the XEngine uses a different binary format.
+
If scripts contain identical source code then there can be more than one script using the same code.  However, they will maintain separate state files to record the script's individual state. These are labelled
  
The XEngine will create up to 4 files for each script, these are:
+
* '''<script-item-id>.state'''
  
.dll, the actual executable file
+
For more information on state files, see below.
  .mdb, the debug info, if requested
+
 
.state, the saved state
+
= Configuration =
  .cs, the converted source file, if requested
+
 
 +
XEngine is enabled by default in OpenSimulator. If you do need to enable it manually, make sure that the following line is present in your OpenSim.ini [Startup] section:
 +
 
 +
  DefaultScriptEngine = "XEngine"
  
The XEngine has a number of configuration options in OpenSim.ini, which are listed below, with an explanation and the default values:
+
XEngine has a number of configuration options in OpenSim.ini, which are listed below, with an explanation and the default values:
  
=== WARNING! Do NOT copy the section below to your OpenSim.ini! ===
+
''' WARNING! Do NOT copy the section below to your OpenSim.ini! '''
 
''Use the template provided in OpenSim.ini.example. The section below contains syntactically incorrect documentation text, which is not meant to be placed in the OpenSim.ini file!
 
''Use the template provided in OpenSim.ini.example. The section below contains syntactically incorrect documentation text, which is not meant to be placed in the OpenSim.ini file!
  
 +
<source lang="ini">
 
  [XEngine]
 
  [XEngine]
 
  ; Allow the XEngine to run
 
  ; Allow the XEngine to run
Line 58: Line 63:
 
  ThreadStackSize = 262144
 
  ThreadStackSize = 262144
 
     ''The stack size each script thread runs with. The default also represents the minimum''
 
     ''The stack size each script thread runs with. The default also represents the minimum''
 +
 +
; Set this to true (the default) to load each script into a separate
 +
; AppDomain. Setting this to false will load all script assemblies into the
 +
; current AppDomain, which will reduce the per-script overhead at the
 +
; expense of reduced security and the inability to garbage collect the
 +
; script assemblies
 +
AppDomainLoading = true
 +
 +
; Controls whether previously compiled scripts DLLs are deleted on sim restart. If you set this to false
 +
; then startup will be considerably faster since scripts won't need to be recompiled. However, then it becomes your responsibility to delete the
 +
; compiled scripts if you're recompiling OpenSim from source code and internal interfaces used
 +
; by scripts have changed.
 +
DeleteScriptsOnStartup = false
 +
 +
; Controls whether scripts are stopped by aborting their threads externally (abort)
 +
; or by co-operative checks inserted by OpenSimulator into compiled script (co-op)
 +
; co-op will be more stable but this option is currently experimental.
 +
; If this setting is changed between co-op and abort, then existing scripts will automatically be recompiled if necessary.
 +
; However, the setting change will not take affect until the next time you restart the simulator.
 +
; Setting changes will not affect state information stored for scripts.
 +
ScriptStopStrategy = abort
 
   
 
   
 
  ; Rate to poll for asynchronous command replies (ms)
 
  ; Rate to poll for asynchronous command replies (ms)
Line 105: Line 131:
 
  OSFunctionThreatLevel = VeryLow
 
  OSFunctionThreatLevel = VeryLow
 
     ''osFunctions are rated by exploitability. This allows balancing functionality and risk
 
     ''osFunctions are rated by exploitability. This allows balancing functionality and risk
   
+
</source>
===OS Functions enable/disable===
+
 
 +
== ScriptStopStrategy ==
 +
This setting was introduced in OpenSimulator 0.7.6 (http://opensimulator.org/pipermail/opensim-dev/2013-January/023985.html).
 +
 
 +
There are two possible values, "abort" (the default as of OpenSimulator 0.8) and "co-op".
 +
 
 +
In abort mode, a script thread that takes too long to stop when requested (e.g. because the object is de-rezzed), has its thread aborted via the Thread.Abort() SDK method.
 +
 
 +
Although this is fine in the vast majority of cases, in heavily loaded simulators where many scripts are often being stopped, such an abort can destabilize the virtual machine and either crash the VM entirely or lead to odd problems such as 100% CPU usage with no apparent cause. Aborting threads in this way is highly discouraged.
 +
 
 +
The "co-op" setting instead inserts checks into the compiled script so that it can be co-operatively shut down rather than aborted.  These checks are placed in such a way that the script cannot ignore them.  Despite the name of the setting, co-op doesn't require any changes to the script source code itself - the checks are inserted entirely by XEngine under the hood.
 +
 
 +
As of OpenSimulator 0.7.6, this setting is technically experimental but should be considered operational, as it has been used in many settings and in places such as the OSGrid plazas without incident.
 +
 
 +
From OpenSimulator 0.7.6 to 0.8, if this setting was changed from "abort" to "co-op", then the compiled script cache would either need to be manually cleared or OpenSimulator started with DeleteScriptsOnStartup = true for one session.
 +
 
 +
As of git master d7b92604 OpenSimulator development code (11th July 2014), this setting can now be switched without any other user action being required.  The recompiled DLLs will be used on the next simulator restart.
 +
 
 +
== OS Functions enable/disable ==
 
For each function, you can add one line, as shown
 
For each function, you can add one line, as shown
 
   
 
   
Line 126: Line 170:
 
* Stopping/starting other scripts (XEngine only)
 
* Stopping/starting other scripts (XEngine only)
  
===State persistence===
 
  
XEngine persists state and variable values across region restarts
+
= Event Execution =
  Note: Sim crossings are not yet supported
+
XEngine maintains its own SmartThreadPool to execute script events.  The maximum number of threads for this pool (and hence the maximum simultaneous events) is set by [XEngine] MaxThreads.  You can also see this information in the "xengine status" command.
 +
 
 +
When a script event needs to be run (e.g. state_entry, touch_start, etc.), it is placed on the ScriptInstance.EventQueue.  If there is no work item event already scheduled, then it is placed as a work item on the threadpool and m_CurrentWorkItem is set.
 +
 
 +
When a threadpool thread becomes available, it runs the event through the ScriptInstance.EventProcessor() method.  At the end of this method, if events are still waiting (i.e. if EventQueue.Count > 0) then another work item is scheduled.
 +
 
 +
= State persistence =
 +
 
 +
XEngine persists state and variable values for individual script instances across region restarts. This is stored in a .state file.  When a script is deleted, the .state file is also deleted.
 +
 
 +
[[Category:Modules]]

Latest revision as of 13:16, 8 January 2015


Contents

[edit] Summary

This is a fully functional, full featured script engine which supports persistent script states and script state serialization.

This engine is enabled by default.

For each unique script asset, XEngine will create up to 4 files. These are:

  • CommonCompiler_compiled_<script-asset-id>.dll, the actual executable DLL for that script asset.
  • On mono, CommonCompiler_compiled_<script-asset-id>.mdb, the debug data file (necessary for resolving line numbers).
  • On Windows, CommonCompiler_compiled_<script-asset-id>.pdb, the debug data file (necessary for resolving line numbers).
  • CommonCompiler_source_CommonCompiler_compiled_<script-asset-id>.cs, if WriteScriptSourceToDebugFile = true in [XEngine] in OpenSim.ini. This is the C# source file as translated from the source language (e.g. from LSL) before compilation.

If scripts contain identical source code then there can be more than one script using the same code. However, they will maintain separate state files to record the script's individual state. These are labelled

  • <script-item-id>.state

For more information on state files, see below.

[edit] Configuration

XEngine is enabled by default in OpenSimulator. If you do need to enable it manually, make sure that the following line is present in your OpenSim.ini [Startup] section:

DefaultScriptEngine = "XEngine"

XEngine has a number of configuration options in OpenSim.ini, which are listed below, with an explanation and the default values:

WARNING! Do NOT copy the section below to your OpenSim.ini! Use the template provided in OpenSim.ini.example. The section below contains syntactically incorrect documentation text, which is not meant to be placed in the OpenSim.ini file!

[XEngine]
 ; Allow the XEngine to run
 Enabled = true
    ''Without this, it won't run at all
 
 ; How many threads to keep alive even if nothing is happening
 MinThreads = 2
    ''XEngine will keep at least this many threads waiting''
 
 ; How many threads to start at maximum load
 MaxThreads = 100
    ''XEngine will not start more than this many scripts, even if that means blocking script execution''
 
 ; Time a thread must be idle (in seconds) before it dies
 IdleTimeout = 60
    ''Threads will die when they are not used for a while. This is the amount of time they must be'
    ''idle for that to happen''
 
 ; Thread priority ("Lowest", "BelowNormal", "Normal", "AboveNormal", "Highest")
 Priority = "BelowNormal"
    ''Script threads will run at this priority''
 
 ; Maximum number of events to queue for a script (excluding timers)
 MaxScriptEventQueue = 300
    ''Each script has a queue for events. This is the length of the queue, determining the number of''
    ''events that will be queued up before events are dropped''
 
 ; Stack size per thread created
 ThreadStackSize = 262144
    ''The stack size each script thread runs with. The default also represents the minimum''
 
 ; Set this to true (the default) to load each script into a separate
 ; AppDomain. Setting this to false will load all script assemblies into the
 ; current AppDomain, which will reduce the per-script overhead at the
 ; expense of reduced security and the inability to garbage collect the
 ; script assemblies
 AppDomainLoading = true
 
 ; Controls whether previously compiled scripts DLLs are deleted on sim restart. If you set this to false
 ; then startup will be considerably faster since scripts won't need to be recompiled. However, then it becomes your responsibility to delete the 
 ; compiled scripts if you're recompiling OpenSim from source code and internal interfaces used
 ; by scripts have changed.
 DeleteScriptsOnStartup = false
 
 ; Controls whether scripts are stopped by aborting their threads externally (abort) 
 ; or by co-operative checks inserted by OpenSimulator into compiled script (co-op) 
 ; co-op will be more stable but this option is currently experimental.
 ; If this setting is changed between co-op and abort, then existing scripts will automatically be recompiled if necessary.
 ; However, the setting change will not take affect until the next time you restart the simulator.
 ; Setting changes will not affect state information stored for scripts.
 ScriptStopStrategy = abort
 
 ; Rate to poll for asynchronous command replies (ms)
 AsyncLLCommandLoopms = 50
    ''Interval between polls for asynchronous command results. Smaller values mean better response, but''
    ''also more burned CPU cycles''
 
 ; Save the source of all compiled scripts
 WriteScriptSourceToDebugFile = false
    ''Create a source file (.cs) on ScriptEngines, containing the preprocessed script''
 
 ; Default language for scripts
 DefaultCompileLanguage = lsl
    ''The default language to compile''
 
 ; List of allowed languages (lsl,vb,js,cs)
 AllowedCompilers = lsl,vb,js,cs
    ''The languages to allow for compiling''
 
 ; Compile debug info (line numbers) into the script assemblies
 CompileWithDebugInformation = true
    ''Create the .mdb debug file, which allows reporting of line numbers in runtime errors''
 
 ; Allow the use of os* functions (some are dangerous)
 AllowOSFunctions = false
    ''Allow the use if functions in the os* namespace''
 
 ; Interval (s) between background save of script states
 SaveInterval = 120
    ''Interval for script state persistence save.''
 
 ; Interval (s) between maintenance runs (0 = disable)
 MaintenanceInterval = 10
    ''Interval in seconds between maintenance runs. The maintenance run checks for long''
    ''running script events and terminates the scripts''
 
 ; Time a script can spend in an event handler before it is interrupted
 EventLimit = 30
    ''If a script spends more time in a single event handler, it will be interrupted.''
    ''This includes delays produced by llSleep() calls''
 
 ; If a script overruns it's event limit, kill the script?
 KillTimedOutScripts = false
    ''If a script gets terminated by the above timeout, suspend it pending a recompile?''
 
 ; Threat level to allow, one of None, VeryLow, Low, Moderate, High, VeryHigh, Severe
 OSFunctionThreatLevel = VeryLow
    ''osFunctions are rated by exploitability. This allows balancing functionality and risk

[edit] ScriptStopStrategy

This setting was introduced in OpenSimulator 0.7.6 (http://opensimulator.org/pipermail/opensim-dev/2013-January/023985.html).

There are two possible values, "abort" (the default as of OpenSimulator 0.8) and "co-op".

In abort mode, a script thread that takes too long to stop when requested (e.g. because the object is de-rezzed), has its thread aborted via the Thread.Abort() SDK method.

Although this is fine in the vast majority of cases, in heavily loaded simulators where many scripts are often being stopped, such an abort can destabilize the virtual machine and either crash the VM entirely or lead to odd problems such as 100% CPU usage with no apparent cause. Aborting threads in this way is highly discouraged.

The "co-op" setting instead inserts checks into the compiled script so that it can be co-operatively shut down rather than aborted. These checks are placed in such a way that the script cannot ignore them. Despite the name of the setting, co-op doesn't require any changes to the script source code itself - the checks are inserted entirely by XEngine under the hood.

As of OpenSimulator 0.7.6, this setting is technically experimental but should be considered operational, as it has been used in many settings and in places such as the OSGrid plazas without incident.

From OpenSimulator 0.7.6 to 0.8, if this setting was changed from "abort" to "co-op", then the compiled script cache would either need to be manually cleared or OpenSimulator started with DeleteScriptsOnStartup = true for one session.

As of git master d7b92604 OpenSimulator development code (11th July 2014), this setting can now be switched without any other user action being required. The recompiled DLLs will be used on the next simulator restart.

[edit] OS Functions enable/disable

For each function, you can add one line, as shown

; true is the default for all functions, and allows them if below threat level
Allow_osSetRegionWaterHeight = true
   This allows any user to use it, if the threat level permits

or

; false disables the function completely
Allow_osSetRegionWaterHeight = false
   Prevents anyone from using it

or

; Comma separated list of UUIDS allows the function for that list of UUIDS
Allow_osSetRegionWaterHeight = 888760cb-a3cf-43ac-8ea4-8732fd3ee2bb
   Allows only the listed user(s) to use these functions (unconditionally)

Both engines share the same parser and LSL implementation, so any script that runs on one can be expected to run on the other, with a few notable exceptions:

  • Resetting other scripts (XEngine only)
  • Stopping/starting other scripts (XEngine only)


[edit] Event Execution

XEngine maintains its own SmartThreadPool to execute script events. The maximum number of threads for this pool (and hence the maximum simultaneous events) is set by [XEngine] MaxThreads. You can also see this information in the "xengine status" command.

When a script event needs to be run (e.g. state_entry, touch_start, etc.), it is placed on the ScriptInstance.EventQueue. If there is no work item event already scheduled, then it is placed as a work item on the threadpool and m_CurrentWorkItem is set.

When a threadpool thread becomes available, it runs the event through the ScriptInstance.EventProcessor() method. At the end of this method, if events are still waiting (i.e. if EventQueue.Count > 0) then another work item is scheduled.

[edit] State persistence

XEngine persists state and variable values for individual script instances across region restarts. This is stored in a .state file. When a script is deleted, the .state file is also deleted.

Personal tools
General
About This Wiki