ScriptEngines

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
(OpenSim.ini Settings)
 
(42 intermediate revisions by 11 users not shown)
Line 1: Line 1:
 +
{{Quicklinks}}
 +
<br />
 +
 
== Script Engines ==
 
== Script Engines ==
  
  Note: This page represents an implementation goal. Much of this already works, but some parts are not
+
OpenSimulator supports scripting via script engines. Script engines are normal region modules.
      implemented yet.
+
 
 +
This page deals with common parts of script engines.<br>
 +
OpenSimulator currently has two public script engines XEngine and YEngine.<br>
 +
For more information, please see [[XEngine]] and [[YEngine]].
 +
 
 +
For information on the old DotNetEngine which was deprecated in OpenSimulator 0.6.8 and subsequently removed from the core tree, please see [[DotNetEngine]] and [[OpenSim.Region.ScriptEngine]].
 +
 
 +
:Note: This page represents an implementation goal. Much of this already works, but some parts are not implemented yet.
 +
 
 +
== OpenSim.ini Settings ==
 +
Multiple script engines can now be active at once. The script_engine directive in OpenSim.ini is no longer used!
 +
Instead, in the [Startup] section, you will now find
 +
<source lang="ini">
 +
DefaultScriptEngine = "XEngine"
 +
</source>
 +
XEngine is the default, to change it to any other engine, change the comments to activate the line
 +
<source lang="ini">
 +
DefaultScriptEngine = (some engine name)
 +
</source>
 +
XEngine can also be disabled by setting
 +
<source lang="ini">
 +
[XEngine]
 +
Enabled = false
 +
</source>
 +
These default to true.
 +
 +
OpenSimulator should be able to run multiple script engines simultaneously, but this should be avoid
 +
 
 +
== Multiple script engine usage ==
  
OpenSim supports scripting via script engines.
+
To run a script on a specific engine, begin the script with a first line like:
 +
<source lang="lsl">
 +
//XEngine: or //someOtherEngineName:
 +
</source>
 +
Optionally, this can be followed by a language code:
 +
<source lang="lsl">
 +
//XEngine:lsl
 +
//someOtherEngineName:lsl
 +
</source>
 +
Beware: there is a little side effect. If you start your script with anything like
 +
<source lang="lsl">
 +
//any text you want:
 +
</source>
 +
(starting with "//" and ending with ":") you'll get an error
  
Script engines are special region modules, which cannot be shared modules.
+
''"Selected engine unavailable. Running script on XEngine"''
  
 
A script engine defines a way to load and run a script. It uses compilers and runtimes to accomplish this.
 
A script engine defines a way to load and run a script. It uses compilers and runtimes to accomplish this.
Line 12: Line 56:
 
Compilers take script text and convert it to a .NET assembly. Such an assembly needs to reference a runtime, which provides API stubs.
 
Compilers take script text and convert it to a .NET assembly. Such an assembly needs to reference a runtime, which provides API stubs.
  
This calls out of the appdomain to the API implementation.
+
Currently, a compiler exists for lsl.
  
Currently, a compiler exists for lsl, c#, j# and vb. YieldProlog (yp) can be added to c# scripts.
+
A runtime exists for the LSL API and for the OSSL API. [[LSL Status/Functions]]
  
A runtime exists for the LSL API and for the OSSL API. [[LSL_Status/Functions]]
+
The following describes the directory structure(under /OpenSim/Region/):
  
The runtime and compiler are not dependent on each other, and also not dependent on the script engine.
+
{|style="margin-left:20px;"
 +
|ScriptEngines/XEngine/||The engine itself. Methods to manage threads, AppDomains, etc
 +
|-
 +
|ScriptEngines/Interfaces/||Common interfaces used to create script engines and components
 +
|-
 +
|ScriptEngines/Shared/CodeTools/||The currrent compiler
 +
|-
 +
|ScriptEngines/Shared/Api/Interface/||The Api interfaces (see below)
 +
|-
 +
|ScriptEngines/Shared/Api/Implementation/||The Api implementations (see below)
 +
|-
 +
|ScriptEngines/Shared/Api/Runtime/||The Api runtimes (see below)
 +
|}
  
The following describes the directory structure:
 
  
ScriptEngines/DotNetEngine/                      The engine itself. Methods to manage threads, AppDomains, etc
+
An API is defined as an implementation, and interface and a runtime. The reason for this seemingly complicated approach has to do with scripts running in AppDomains apart from the normal OpenSimulator code.
ScriptEngines/XEngine/                          The engine itself. Methods to manage threads, AppDomains, etc
+
ScriptEngines/Interfaces/                        Common interfaces used to create script engines and components
+
ScriptEngines/Shared/CodeTools/                  The currrent compiler
+
ScriptEngines/Shared/Api/Interface/              The Api interfaces (see below)
+
ScriptEngines/Shared/Api/Implementation/        The Api implementations (see below)
+
ScriptEngines/Shared/Api/Runtime/                The Api runtimes (see below)
+
ScriptEngines/Shared/Api/Runtime/YieldProlog/    The Yield Prolog runtime
+
 
+
An API is defined as an implementation, and interface and a runtime. The reason for this seemingly conplicated approach has to do with scripts running in AppDomains apart from the normal OpenSim code.
+
  
 
Creating an API is pretty straightforward, but a few naming conventions must be observed for the Api to function correctly. The script runtime uses reflection to match Api runtimes to implementations, and it uses names to find these matches.
 
Creating an API is pretty straightforward, but a few naming conventions must be observed for the Api to function correctly. The script runtime uses reflection to match Api runtimes to implementations, and it uses names to find these matches.
Line 37: Line 83:
 
First, create your API implementation.
 
First, create your API implementation.
  
 +
<source lang="csharp">
 
  namespace OpenSim.Region.ScriptEngine.Shared.Api
 
  namespace OpenSim.Region.ScriptEngine.Shared.Api
 
  {
 
  {
     public class XXX_Api: MasrshalByRefObject, IXXX_Api, IScriptApi
+
     public class XXX_Api: MarshalByRefObject, IXXX_Api, IScriptApi
 
     {
 
     {
 
         internal IScriptEngine m_ScriptEngine;
 
         internal IScriptEngine m_ScriptEngine;
 
         internal SceneObjectPart m_host;
 
         internal SceneObjectPart m_host;
 
         internal uint m_localID;
 
         internal uint m_localID;
         internal LLUUID m_itemID;
+
         internal UUID m_itemID;
 
   
 
   
         public void Initialize(IScriptEngine ScriptEngine, SceneObjectPart host, uint localID, LLUUID itemID)
+
         public void Initialize(IScriptEngine ScriptEngine, SceneObjectPart host, uint localID, UUID itemID)
 
         {
 
         {
 
             m_ScriptEngine = ScriptEngine;
 
             m_ScriptEngine = ScriptEngine;
Line 59: Line 106:
 
     }
 
     }
 
  }
 
  }
 +
</source>
  
 
Here, the class name (XXX_Api) is used by reflection, it must end in "_Api" to be recognized as an Api.
 
Here, the class name (XXX_Api) is used by reflection, it must end in "_Api" to be recognized as an Api.
Line 69: Line 117:
 
Next, create an interface to link the Api to the runtime:
 
Next, create an interface to link the Api to the runtime:
  
 +
<source lang="csharp">
 
  namespace OpenSim.Region.ScriptEngine.Shared.Api.Interfaces
 
  namespace OpenSim.Region.ScriptEngine.Shared.Api.Interfaces
 
  {
 
  {
Line 76: Line 125:
 
     }
 
     }
 
  }
 
  }
 +
</source>
  
 
Place this file in OpenSim/Region/ScriptEngines/Api/Interface, named IXXX_Api.
 
Place this file in OpenSim/Region/ScriptEngines/Api/Interface, named IXXX_Api.
Line 82: Line 132:
 
Now, the stub file is needed to connect the script to the Api. This stub file will be loaded into the script AppDomain, and should not contain any processing. It merely forwards the calls to the Api outside of the AppDomain.
 
Now, the stub file is needed to connect the script to the Api. This stub file will be loaded into the script AppDomain, and should not contain any processing. It merely forwards the calls to the Api outside of the AppDomain.
  
 +
<source lang="csharp">
 
  namespace OpenSim.Region.ScriptEngine.Shared.ScriptBase
 
  namespace OpenSim.Region.ScriptEngine.Shared.ScriptBase
 
  {
 
  {
Line 102: Line 153:
 
     }
 
     }
 
  }
 
  }
 +
</source>
  
 
Save this in OpenSim/Region/ScriptEngines/Shared/Api/Runtime, name it XXX_Stub.cs
 
Save this in OpenSim/Region/ScriptEngines/Shared/Api/Runtime, name it XXX_Stub.cs
Line 115: Line 167:
  
 
The directory OpenSim/Region/ScriptEngines/Shared/CodeTools contains all compilers, converters and other code manipulation tools. It is shared between all script engines. Because of the shared compiler, the script engines can also share compiled assemblies, eliminating the need to recompile each script for each script engine.
 
The directory OpenSim/Region/ScriptEngines/Shared/CodeTools contains all compilers, converters and other code manipulation tools. It is shared between all script engines. Because of the shared compiler, the script engines can also share compiled assemblies, eliminating the need to recompile each script for each script engine.
 +
 +
== Code generation ==
 +
 +
LSL to C# code generation is common to all script engines.  This is currently done by translating LSL to C# code and then compiling it.  Compiled code then calls methods in LSL_Api.cs (OpenSim/Region/ScriptEngine/Shared/Api/Implementation/LSL_Api.cs) that correspond to LSL functions or methods in OSSL_Api.cs for OSSL functions, etc.
 +
 +
Since OpenSimulator 0.7.4, there is also a mechanism for making script functions available from region modules without patching OpenSimulator itself.  See [[OSSL_Script_Library/ModInvoke]] for more details.
 +
 +
Parsing the LSL script is kicked off by CSCodeGenerator.Convert() (OpenSim/Region/ScriptEngine/Shared/CodeTools/CSCodeGenerator.cs).  This takes the LSL and uses a parser/lexer to convert into in an abstract syntax tree (AST), before then converting that AST to C# via the LSL2CSCodeTransformer.
 +
 +
The parser (lsl.parser.cs) and lexer (lsl.lexer.cs) are generated using Malcolm Crowe's compiler tools in C# from the trunk/managed/lsl2cs directory in the [[Opensim-libs_git_repository|opensim-libs repository]] from the lsl.parser and lsl.lexer files.  '''Note that if you regenerate this, you will need to change the penultimate number on the arr array in the generated ArgumentDeclarationList_5 class in lsl.parser.cs from 1 to 0.  We need to find out why this bug occurs.'''
 +
 +
The converted C# is then compiled to a DLL by the OpenSimulator Compiler class (via the PerformScriptCompile() method that also triggered LSL -> C# conversion as described above).
 +
 +
You can see the tranformed C# that was compiled by setting
 +
 +
<source lang='ini'>
 +
[XEngine]
 +
WriteScriptSourceToDebugFile = true
 +
</source>
 +
 +
in your OpenSim.ini file.  This will appear in the bin/ScriptEngines/<region-id>/ directory with the filename CommonCompiler_source_CommonCompiler_compiled_<script-asset-id>.lsl.
 +
 +
[[Category:Scripting]]

Latest revision as of 04:22, 22 September 2020


Contents

[edit] Script Engines

OpenSimulator supports scripting via script engines. Script engines are normal region modules.

This page deals with common parts of script engines.
OpenSimulator currently has two public script engines XEngine and YEngine.
For more information, please see XEngine and YEngine.

For information on the old DotNetEngine which was deprecated in OpenSimulator 0.6.8 and subsequently removed from the core tree, please see DotNetEngine and OpenSim.Region.ScriptEngine.

Note: This page represents an implementation goal. Much of this already works, but some parts are not implemented yet.

[edit] OpenSim.ini Settings

Multiple script engines can now be active at once. The script_engine directive in OpenSim.ini is no longer used! Instead, in the [Startup] section, you will now find

DefaultScriptEngine = "XEngine"

XEngine is the default, to change it to any other engine, change the comments to activate the line

DefaultScriptEngine = (some engine name)

XEngine can also be disabled by setting

[XEngine]
Enabled = false

These default to true.

OpenSimulator should be able to run multiple script engines simultaneously, but this should be avoid

[edit] Multiple script engine usage

To run a script on a specific engine, begin the script with a first line like:

//XEngine: or //someOtherEngineName:

Optionally, this can be followed by a language code:

//XEngine:lsl
//someOtherEngineName:lsl

Beware: there is a little side effect. If you start your script with anything like

//any text you want:

(starting with "//" and ending with ":") you'll get an error

"Selected engine unavailable. Running script on XEngine"

A script engine defines a way to load and run a script. It uses compilers and runtimes to accomplish this.

Compilers take script text and convert it to a .NET assembly. Such an assembly needs to reference a runtime, which provides API stubs.

Currently, a compiler exists for lsl.

A runtime exists for the LSL API and for the OSSL API. LSL Status/Functions

The following describes the directory structure(under /OpenSim/Region/):

ScriptEngines/XEngine/ The engine itself. Methods to manage threads, AppDomains, etc
ScriptEngines/Interfaces/ Common interfaces used to create script engines and components
ScriptEngines/Shared/CodeTools/ The currrent compiler
ScriptEngines/Shared/Api/Interface/ The Api interfaces (see below)
ScriptEngines/Shared/Api/Implementation/ The Api implementations (see below)
ScriptEngines/Shared/Api/Runtime/ The Api runtimes (see below)


An API is defined as an implementation, and interface and a runtime. The reason for this seemingly complicated approach has to do with scripts running in AppDomains apart from the normal OpenSimulator code.

Creating an API is pretty straightforward, but a few naming conventions must be observed for the Api to function correctly. The script runtime uses reflection to match Api runtimes to implementations, and it uses names to find these matches.

First, create your API implementation.

namespace OpenSim.Region.ScriptEngine.Shared.Api
 {
     public class XXX_Api: MarshalByRefObject, IXXX_Api, IScriptApi
     {
        internal IScriptEngine m_ScriptEngine;
        internal SceneObjectPart m_host;
        internal uint m_localID;
        internal UUID m_itemID;
 
        public void Initialize(IScriptEngine ScriptEngine, SceneObjectPart host, uint localID, UUID itemID)
        {
            m_ScriptEngine = ScriptEngine;
            m_host = host;
            m_localID = localID;
            m_itemID = itemID;
        }
 
        public void myApiFunction()
        {
        }
     }
 }

Here, the class name (XXX_Api) is used by reflection, it must end in "_Api" to be recognized as an Api.

Place the file in OpenSim/Region/ScriptEngines/Shared/Api/Implementaton/

By convention, it should be named XXX_Api.cs


Next, create an interface to link the Api to the runtime:

namespace OpenSim.Region.ScriptEngine.Shared.Api.Interfaces
 {
     public interface IXXX_Api
     {
         void myApiFunction();
     }
 }

Place this file in OpenSim/Region/ScriptEngines/Api/Interface, named IXXX_Api.


Now, the stub file is needed to connect the script to the Api. This stub file will be loaded into the script AppDomain, and should not contain any processing. It merely forwards the calls to the Api outside of the AppDomain.

namespace OpenSim.Region.ScriptEngine.Shared.ScriptBase
 {
     public partial class ScriptBaseClass : MarshalByRefObject
     {
         public IXXX_Api m_XXX_Functions;
 
         public void ApiTypeXXX(IScriptApi api)
         {
             if(!(api is IXXX_Api))
                 return;
 
             m_XXX_Functions = (IXXX_Api)api;
         }
 
         public void myApiFunction()
         {
             m_XXX_Functions.myApiFunction();
         }
     }
 }

Save this in OpenSim/Region/ScriptEngines/Shared/Api/Runtime, name it XXX_Stub.cs

Here, the ApiTypeXXX naming is the mandatory name, by which the script engine recognizes this Api and matches it up to the API implementation.

Please note that your Api here becomes a part of a partial class, which, in turn, becomes the base class of the script.

On script load, the Api runtime (stub) is called with a reference to the IScriptApi interface of the implementation. That, cast to the custom interface, is used to access the function implementations.

The reason we cannot directly reference the implementation is that that would cause the entire implementation to be loaded into the script's AppDomain. We need to avoid this to keep the memory footprint small.


The directory OpenSim/Region/ScriptEngines/Shared/CodeTools contains all compilers, converters and other code manipulation tools. It is shared between all script engines. Because of the shared compiler, the script engines can also share compiled assemblies, eliminating the need to recompile each script for each script engine.

[edit] Code generation

LSL to C# code generation is common to all script engines. This is currently done by translating LSL to C# code and then compiling it. Compiled code then calls methods in LSL_Api.cs (OpenSim/Region/ScriptEngine/Shared/Api/Implementation/LSL_Api.cs) that correspond to LSL functions or methods in OSSL_Api.cs for OSSL functions, etc.

Since OpenSimulator 0.7.4, there is also a mechanism for making script functions available from region modules without patching OpenSimulator itself. See OSSL_Script_Library/ModInvoke for more details.

Parsing the LSL script is kicked off by CSCodeGenerator.Convert() (OpenSim/Region/ScriptEngine/Shared/CodeTools/CSCodeGenerator.cs). This takes the LSL and uses a parser/lexer to convert into in an abstract syntax tree (AST), before then converting that AST to C# via the LSL2CSCodeTransformer.

The parser (lsl.parser.cs) and lexer (lsl.lexer.cs) are generated using Malcolm Crowe's compiler tools in C# from the trunk/managed/lsl2cs directory in the opensim-libs repository from the lsl.parser and lsl.lexer files. Note that if you regenerate this, you will need to change the penultimate number on the arr array in the generated ArgumentDeclarationList_5 class in lsl.parser.cs from 1 to 0. We need to find out why this bug occurs.

The converted C# is then compiled to a DLL by the OpenSimulator Compiler class (via the PerformScriptCompile() method that also triggered LSL -> C# conversion as described above).

You can see the tranformed C# that was compiled by setting

[XEngine]
WriteScriptSourceToDebugFile = true

in your OpenSim.ini file. This will appear in the bin/ScriptEngines/<region-id>/ directory with the filename CommonCompiler_source_CommonCompiler_compiled_<script-asset-id>.lsl.

Personal tools
General
About This Wiki