Codebase overview

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
(Codebase overview)
(Packages)
(10 intermediate revisions by one user not shown)
Line 1: Line 1:
__NOTOC__
 
 
{{Quicklinks}}
 
{{Quicklinks}}
<br />
 
== Coding restrictions ==
 
  
Because OpenSimulator has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that [http://en.wikipedia.org/wiki/Mono_(software) Mono] supports.
+
= Introduction =
  
== Codebase overview ==
+
OpenSimulator is a large and organically grown project, so its architecture is still subject to change.  As such, there are very few formal internal interfaces - [[IRegionModule|region modules]] operating by plugging in directly to the core internals.
  
(This is probably quite out of date. On 2011-08-08, a few edits were made to bring pieces of it up to date, but the editor doesn't have enough of an overview in head to remove all the parts that aren't current. Edits will continue.)
+
= Architecture =
  
OpenSimulator is a large and organically grown project, so its architecture is still subject to change.  As such, there are very few formal internal interfaces - [[region modules]] operating by plugging in directly to the core internals.
+
[[Image:Opensim-block.jpg|frame|center|An diagram of the core parts of the simulator code organization.]]
  
At the top level, OpenSimulator code is broken up into seven sections. There's also a bit more detail on the most significant packages within those sections.
+
This is a very high level diagram of the main chunks of the simulator code organization.  Arrows indicate direction of request initiation.
 +
 
 +
A viewer (at the bottom) interacts with the simulator by sending messages through both UDP and HTTP.
 +
 
 +
Regions and region modules register for UDP message events (e.g. user command inputs such as move forward, inventory manipulation requests, etc).  These events are invoked when the LLUDP client stack receives the relevant requests from the viewer.  In turn, the regions and region modules will pass messages back to the LLUDP clientstack (e.g. movements of avatars and objects) for delivery back to the client.
 +
 
 +
LL capabilities are a similar message passing mechanism but with HTTP as the carrier rather than UDP.  These tend to be used for messages which are not time critical and are often used for moving large amounts of data in newer viewers (e.g. texture and inventory request and delivery, though not other inventory commands such as item move).  Again, these are handled by regions and region modules.
 +
 
 +
A simulator can pass messages back to the region via the event queue capability, which is essentially a poll by the viewer to the simulator to fetch messages.
 +
 
 +
Regions and region modules will in turn communicate with backend grid services to fetch required data (e.g. assets), manipulate inventory, etc.  In OpenSimulator standalone mode, services will be in the same process as OpenSim.exe.  In grid mode, they will be hosted in a separate Robust.exe process.
 +
 
 +
More exotic configurations of OpenSimulator would complicate this diagram.  For instance, it's possible to configure OpenSimulator to fetch assets directly from capabilities implemented in the same process as the grid service rather than go through the simulator.
 +
 
 +
= Packages =
  
 
* '''OpenSim'''. This contains the classes which launch the OpenSimulator region server and handle the console
 
* '''OpenSim'''. This contains the classes which launch the OpenSimulator region server and handle the console
Line 18: Line 29:
 
*** The Main() for <tt>OpenSim.exe</tt> can be found in the file <tt>OpenSim/Region/Application/Application.cs</tt>. It does a bunch of configuration checks, but the meat of it is the instantiation of an '''OpenSim''' object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
 
*** The Main() for <tt>OpenSim.exe</tt> can be found in the file <tt>OpenSim/Region/Application/Application.cs</tt>. It does a bunch of configuration checks, but the meat of it is the instantiation of an '''OpenSim''' object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
 
*** OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
 
*** OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
*** OpenSim.OpenSim.StartupSpecific() exists. It creates a console, registers the region console commands, makes a couple of status stream handlers, runs startup commands if any are specified in config, and hooks up its WatchdogTimeoutHandler to Watchdog. It then calls its base class' StartupSpecific(), and each base class calls ''its'' base class' StartupSpecific() (all the way back up to OpenSim.Framework.Servers.BaseOpenSimServer.StartupSpecific()).
 
*** ancestor OpenSim.OpenSimBase starts up simulation and estate data services; creates a ModuleLoader (in field <tt>m_moduleLoader</tt>); and loads plugins of type IApplicationPlugin at extension point ("/OpenSim/Startup") (I have to admit to not fully knowing what that means...), and initializes each one of them.
 
*** ancestor OpenSim.Region.ClientStack.RegionApplicationBase starts up an http server in its StartupSpecific()
 
*** ancestor OpenSim.Framework.Servers.BaseOpenSimServer starts up (something to do with) the console in its StartupSpecific()
 
  
 +
* '''OpenSim.Addons'''.  OpenSimulator components which came originally from the Diva distribution.  Hence their different organization compared to other features.
 +
** '''OpenSim.Addons.Groups'''.  Core support for user groups.
 +
** '''OpenSim.Addons.OfflineIM'''.  Core support for offline IM.
 
* '''OpenSim.ApplicationPlugins'''. Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region server. See [[RegionModules]] for more details or one of [http://justincc.wordpress.com/2008/05/08/opensim-tech-basics-my-god-its-full-of-modules/ justincc's blogposts] for a higher level overview (which should really be in this wiki).
 
* '''OpenSim.ApplicationPlugins'''. Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region server. See [[RegionModules]] for more details or one of [http://justincc.wordpress.com/2008/05/08/opensim-tech-basics-my-god-its-full-of-modules/ justincc's blogposts] for a higher level overview (which should really be in this wiki).
 +
* '''OpenSim.Capabilities'''.  Basic structures for [[capabilities]] support.
 +
* '''OpenSim.ConsoleClient'''.  [[Client|RestConsole]] for connecting remotely to the OpenSimulator console.  Not widely used as of yet - more commonly OpenSimulator is started in a [https://www.gnu.org/software/screen/ screen] instance in Linux and in separate terminals in Windows.
 +
* '''OpenSim.Data'''. This is all the low level database manipulation stuff. In here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag).
 +
* '''OpenSim.Framework'''. General framework code used by the rest of OpenSimulator which is either not specific to regions or used in common with regions and other classes.
 +
** '''OpenSim.Framework.AssetLoader.Filesystem'''.  Small DLL used to load library assets from the filesystem.
 +
** '''OpenSim.Framework.Communications'''.  Some comms related odds and ends.  Notably request limiting code in OpenSimulator master git branch post 0.7.6.
 +
** '''OpenSim.Framework.Configuration.HTTP'''.  Old code for loading configuration via HTTP.  May have decayed beyond usefulness.
 +
** '''OpenSim.Framework.Configuration.XML'''.  Code for loading XML config files (as opposed to INI).  May have decayed beyond usefulness.
 +
** '''OpenSim.Framework.Console'''.  Framework classes for the OpenSimulator console.
 +
** '''OpenSim.Framework.Monitoring'''.  Statistics gathering and monitoring infrastructure.
 +
** '''OpenSim.Framework.RegionLoader.FileSystem'''.  Code for loading region configuration from the filesystem (e.g. bin/Regions/Regions.ini).
 +
** '''OpenSim.Framework.RegionLoader.Web'''.  Code for loading region configuration remotely.
 +
** '''OpenSim.Framework.Serialization'''.  Infrastructure for OAR and IAR read and write.
 +
** '''OpenSim.Framework.Servers'''. Contains generic server code, such as the base OpenSimulator server and http servers.
 +
* '''OpenSim.Region'''. Contains most of the meaty code that it specifically concerned with regions/scenes.
 +
** '''OpenSim.Region.ClientStack'''.  Small amount of infrastructure for making different UDP client stacks available (though currently only LLUDP exists).
 +
** '''OpenSim.Region.ClientStack.LindenCaps'''.  The code necessary for communicating with clients using the HTTP (capability) part of the Second Life protocol.  Glues data going to/from the client to region modules that handle specific aspects.
 +
** '''OpenSim.Region.ClientStack.LindenUDP'''.  The code necessary for communicating with clients using the UDP part of the Second Life protocol.  Glues data going to/from the client to scene and region module code.
 +
** '''OpenSim.Region.CoreModules'''.  The region modules considered core to OpenSimulator (and thus come bundled with it).  This covers a vast range.  For example, the Avatar/Attachments/AttachmentsModule handles avatar attachments (though with many structures still in core region code).  The Framework/EntityTransfer/EntityTransferModule handles region crossings and teleports (though again in co-operation with large chunks of core region code).
 +
** '''OpenSim.Region.DataSnapshot'''.  Code for creating an XML file that summarises the contents of the region.  Could be exposed to an external search engine.
 +
** '''OpenSim.Region.Framework'''.  The framework for the region itself, including scene graph, scene presence (agent) classes, scene object classes, inventory, etc.  Could be considered the core of the simulator.
 +
** '''OpenSim.Region.OptionalModules'''.  Optional region modules that aren't considered core to the simulator.  In principle, could be removed yet have a working Second Life compatible simulator, though this has not been well tested.
 +
** '''OpenSim.Region.Physics'''. General physics framework code and plugins for specific engines (BulletSim, ODE, etc).  Only the ODE and BulletSim modules are mature or complete enough for use.
 +
** '''OpenSim.Region.ScriptEngine'''. Script engine code that powers the scripts run within OpenSimulator. Currently there is only one engine bundled with OpenSimulator, [[XEngine]].
 +
** '''OpenSim.Region.UserStatistics'''.  [[Web Statistics Module]] for exposing some simulator information via a webpage.
 +
* '''OpenSim.Server'''.  Server infrastructure.
 +
** '''OpenSim.Server.Base'''.  Common server infrastructure for the simulator and Robust grid servers.
 +
** '''OpenSim.Server.Handlers'''.  Classes for handling raw incoming data and translating these to service requests.  Used mainly in grid mode for ROBUST server instances.  For instance, Asset/AssetServerGetHandler translates internal asset fetch requests to the asset service and replies with the results.
 +
* '''OpenSim.Services'''.  Code which provides both service implementations (e.g. AssetService to read and write assets to persistent storage) interfaces and infrastructure (chiefly OpenSim.Services.Connectors which creates the actually HTTP requests to invoke services remotely).  In grid mode, these are instantiated within Robust.exe instances.  In standalone mode, these are instantiated directly in OpenSim.exe.
 +
* '''OpenSim.Tests.*'''.  Test code, both common infrastructure used in regression tests and some standalone client test code.  Actual regression tests are spread throughout the code base in separate *.Test packages (e.g. OpenSim.Region.Framework.Tests project).
 +
* '''OpenSim.Tools'''.  Some OpenSimulator associated tools.  Not used in actual OpenSimulator operation.
 +
* '''pCampbot'''.  A test tool which uses bots to simulate activity load on a simulator.  See [[pCampbot]] for more details.  Not used in actual OpenSimulator operation.
  
* '''OpenSim.Data'''. This is all the low level database manipulation stuff. In here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag), as well as code for [http://www.hibernate.org/ NHibernate] which will hopefully reduce our reliance on manipulation code for specific databases.
+
== Coding restrictions ==
 
+
* '''OpenSim.Framework'''. General framework code used by the rest of OpenSimulator which is felt to be not specific to a region.
+
** ''OpenSim.Framework.Communications''. Contains asset and inventory caching code (to cut down on communication with a remote asset or inventory service), and contains CAPS code, a Linden approach to splitting some aspects of the Second Life protocol into independent, secured services. See a [http://mrtopf.de/blog/secondlife/slga-capabilities-explained-technical/ blog post] from Tao for more background on this.
+
** ''OpenSim.Framework.Servers''. Contains generic server code, such as the base OpenSimulator server and http servers.
+
** ''OpenSim.Framework.Stats''. Collects statistical information on OpenSimulator operations.
+
 
+
* '''OpenSim.Server''': these are things that are part of the R.O.B.U.S.T. services, including grid servers, avatar servers, asset servers, inventory servers, and the like.
+
 
+
* '''OpenSim.Region'''. Contains most of the meaty code that it specifically concerned with region manipulation.
+
** ''OpenSim.Region.ClientStack.LindenUDP''. The code necessary for communicating with clients using the Second Life protocol.
+
** ''OpenSim.Region.Communications.Local''. Code used for communicating with non-region services (user, asset, grid, etc.) when OpenSimulator is running in standalone mode.
+
** ''OpenSim.Region.Communications.OGS1''. Code for communicating with remote UGAI services using the Open Grid Services 1 protocol.
+
** ''OpenSim.Region.Environment''. A large package containing both the vast majority of the region modules (categorized under modules) and the central scene manipulation code (under Scenes, appropriately enough). Modules here do everything from handling region chat to loading and saving terrain.
+
** ''OpenSim.Region.Modules''. More region module code. Actually, I'm not too sure why these aren't in the OpenSim.Region.Environment package.
+
** ''OpenSim.Region.Physics''. General physics framework code and plugins for specific engines (BulletX, ODE, etc). The ODE code is by far the most reliable and stable.
+
** ''OpenSim.Region.ScriptEngine''. Script engine code that powers the scripts run within OpenSimulator. Currently there are two engines, the classic [[DotNetEngine]] and the relatively new [[XEngine]].
+
** ''[[Overview of How RegionsWork|OpenSim.Region.Framework.Scenes.Scene]]'' is the "heart" of OpenSimulator functionality. It has the "main loop" for a region, in the method <tt>Update()</tt>.
+
 
+
  
* '''OpenSim.Tests'''. Test code. Really, we don't actually have much of this (yet).
+
Because OpenSimulator has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that the minimum version of [http://en.wikipedia.org/wiki/Mono_(software) Mono] we require supports.

Revision as of 02:07, 6 April 2019

Contents

Introduction

OpenSimulator is a large and organically grown project, so its architecture is still subject to change. As such, there are very few formal internal interfaces - region modules operating by plugging in directly to the core internals.

Architecture

An diagram of the core parts of the simulator code organization.

This is a very high level diagram of the main chunks of the simulator code organization. Arrows indicate direction of request initiation.

A viewer (at the bottom) interacts with the simulator by sending messages through both UDP and HTTP.

Regions and region modules register for UDP message events (e.g. user command inputs such as move forward, inventory manipulation requests, etc). These events are invoked when the LLUDP client stack receives the relevant requests from the viewer. In turn, the regions and region modules will pass messages back to the LLUDP clientstack (e.g. movements of avatars and objects) for delivery back to the client.

LL capabilities are a similar message passing mechanism but with HTTP as the carrier rather than UDP. These tend to be used for messages which are not time critical and are often used for moving large amounts of data in newer viewers (e.g. texture and inventory request and delivery, though not other inventory commands such as item move). Again, these are handled by regions and region modules.

A simulator can pass messages back to the region via the event queue capability, which is essentially a poll by the viewer to the simulator to fetch messages.

Regions and region modules will in turn communicate with backend grid services to fetch required data (e.g. assets), manipulate inventory, etc. In OpenSimulator standalone mode, services will be in the same process as OpenSim.exe. In grid mode, they will be hosted in a separate Robust.exe process.

More exotic configurations of OpenSimulator would complicate this diagram. For instance, it's possible to configure OpenSimulator to fetch assets directly from capabilities implemented in the same process as the grid service rather than go through the simulator.

Packages

  • OpenSim. This contains the classes which launch the OpenSimulator region server and handle the console
    • Startup
      • The Main() for OpenSim.exe can be found in the file OpenSim/Region/Application/Application.cs. It does a bunch of configuration checks, but the meat of it is the instantiation of an OpenSim object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
      • OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
  • OpenSim.Addons. OpenSimulator components which came originally from the Diva distribution. Hence their different organization compared to other features.
    • OpenSim.Addons.Groups. Core support for user groups.
    • OpenSim.Addons.OfflineIM. Core support for offline IM.
  • OpenSim.ApplicationPlugins. Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region server. See RegionModules for more details or one of justincc's blogposts for a higher level overview (which should really be in this wiki).
  • OpenSim.Capabilities. Basic structures for capabilities support.
  • OpenSim.ConsoleClient. RestConsole for connecting remotely to the OpenSimulator console. Not widely used as of yet - more commonly OpenSimulator is started in a screen instance in Linux and in separate terminals in Windows.
  • OpenSim.Data. This is all the low level database manipulation stuff. In here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag).
  • OpenSim.Framework. General framework code used by the rest of OpenSimulator which is either not specific to regions or used in common with regions and other classes.
    • OpenSim.Framework.AssetLoader.Filesystem. Small DLL used to load library assets from the filesystem.
    • OpenSim.Framework.Communications. Some comms related odds and ends. Notably request limiting code in OpenSimulator master git branch post 0.7.6.
    • OpenSim.Framework.Configuration.HTTP. Old code for loading configuration via HTTP. May have decayed beyond usefulness.
    • OpenSim.Framework.Configuration.XML. Code for loading XML config files (as opposed to INI). May have decayed beyond usefulness.
    • OpenSim.Framework.Console. Framework classes for the OpenSimulator console.
    • OpenSim.Framework.Monitoring. Statistics gathering and monitoring infrastructure.
    • OpenSim.Framework.RegionLoader.FileSystem. Code for loading region configuration from the filesystem (e.g. bin/Regions/Regions.ini).
    • OpenSim.Framework.RegionLoader.Web. Code for loading region configuration remotely.
    • OpenSim.Framework.Serialization. Infrastructure for OAR and IAR read and write.
    • OpenSim.Framework.Servers. Contains generic server code, such as the base OpenSimulator server and http servers.
  • OpenSim.Region. Contains most of the meaty code that it specifically concerned with regions/scenes.
    • OpenSim.Region.ClientStack. Small amount of infrastructure for making different UDP client stacks available (though currently only LLUDP exists).
    • OpenSim.Region.ClientStack.LindenCaps. The code necessary for communicating with clients using the HTTP (capability) part of the Second Life protocol. Glues data going to/from the client to region modules that handle specific aspects.
    • OpenSim.Region.ClientStack.LindenUDP. The code necessary for communicating with clients using the UDP part of the Second Life protocol. Glues data going to/from the client to scene and region module code.
    • OpenSim.Region.CoreModules. The region modules considered core to OpenSimulator (and thus come bundled with it). This covers a vast range. For example, the Avatar/Attachments/AttachmentsModule handles avatar attachments (though with many structures still in core region code). The Framework/EntityTransfer/EntityTransferModule handles region crossings and teleports (though again in co-operation with large chunks of core region code).
    • OpenSim.Region.DataSnapshot. Code for creating an XML file that summarises the contents of the region. Could be exposed to an external search engine.
    • OpenSim.Region.Framework. The framework for the region itself, including scene graph, scene presence (agent) classes, scene object classes, inventory, etc. Could be considered the core of the simulator.
    • OpenSim.Region.OptionalModules. Optional region modules that aren't considered core to the simulator. In principle, could be removed yet have a working Second Life compatible simulator, though this has not been well tested.
    • OpenSim.Region.Physics. General physics framework code and plugins for specific engines (BulletSim, ODE, etc). Only the ODE and BulletSim modules are mature or complete enough for use.
    • OpenSim.Region.ScriptEngine. Script engine code that powers the scripts run within OpenSimulator. Currently there is only one engine bundled with OpenSimulator, XEngine.
    • OpenSim.Region.UserStatistics. Web Statistics Module for exposing some simulator information via a webpage.
  • OpenSim.Server. Server infrastructure.
    • OpenSim.Server.Base. Common server infrastructure for the simulator and Robust grid servers.
    • OpenSim.Server.Handlers. Classes for handling raw incoming data and translating these to service requests. Used mainly in grid mode for ROBUST server instances. For instance, Asset/AssetServerGetHandler translates internal asset fetch requests to the asset service and replies with the results.
  • OpenSim.Services. Code which provides both service implementations (e.g. AssetService to read and write assets to persistent storage) interfaces and infrastructure (chiefly OpenSim.Services.Connectors which creates the actually HTTP requests to invoke services remotely). In grid mode, these are instantiated within Robust.exe instances. In standalone mode, these are instantiated directly in OpenSim.exe.
  • OpenSim.Tests.*. Test code, both common infrastructure used in regression tests and some standalone client test code. Actual regression tests are spread throughout the code base in separate *.Test packages (e.g. OpenSim.Region.Framework.Tests project).
  • OpenSim.Tools. Some OpenSimulator associated tools. Not used in actual OpenSimulator operation.
  • pCampbot. A test tool which uses bots to simulate activity load on a simulator. See pCampbot for more details. Not used in actual OpenSimulator operation.

Coding restrictions

Because OpenSimulator has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that the minimum version of Mono we require supports.

Personal tools
General
About This Wiki