Codebase overview

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
(Packages)
(15 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
 
 
{{Quicklinks}}
 
{{Quicklinks}}
<br />
 
==Coding restrictions==
 
  
Because OpenSim has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that [http://en.wikipedia.org/wiki/Mono_(software) Mono] supports.
+
= Introduction =
  
==Codebase overview==
+
OpenSimulator is a large and organically grown project, so its architecture is still subject to change.  As such, there are very few formal internal interfaces - [[IRegionModule|region modules]] operating by plugging in directly to the core internals.
  
(This is probably quite out of date.  On 2011-08-08, a few edits were made to bring pieces of it up to date, but the editor doesn't have enough of an overview in head to remove all the parts that aren't current.  Edits will continue.)
+
= Architecture =
  
OpenSim is already a fairly large and intricate project, so patience is required to start understanding it.
+
[[Image:Opensim-block.jpg|frame|center|An diagram of the core parts of the simulator code organization.]]
  
At the top level, OpenSim code is broken up into seven sectionsThere's also a bit more detail on the most significant packages within those sections.
+
This is a very high level diagram of the main chunks of the simulator code organizationArrows indicate direction of request initiation.
  
* '''OpenSim'''.  This contains the classes which launch the OpenSim region server and handle the console
+
A viewer (at the bottom) interacts with the simulator by sending messages through both UDP and HTTP.
** Startup
+
*** The Main() for <tt>OpenSim.exe</tt> can be found in the file <tt>OpenSim/Region/Application/Application.cs</tt>.  It does a bunch of configuration checks, but the meat of it is the instantiation of an '''OpenSim''' object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
+
*** OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
+
*** OpenSim.OpenSim.StartupSpecific() exists, but much of the meat of really getting a region running is found in the base class' method OpenSim.OpenSimBase.StartupSpecific() (and in other versions of StartupSpecific() on up the inheritance hierarchy).
+
  
* '''OpenSim.ApplicationPlugins'''Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region serverSee [[RegionModules]] for more details or one of [http://justincc.wordpress.com/2008/05/08/opensim-tech-basics-my-god-its-full-of-modules/ justincc's blogposts] for a higher level overview (which should really be in this wiki).
+
Regions and region modules register for UDP message events (e.g. user command inputs such as move forward, inventory manipulation requests, etc)These events are invoked when the LLUDP client stack receives the relevant requests from the viewerIn turn, the regions and region modules will pass messages back to the LLUDP clientstack (e.g. movements of avatars and objects) for delivery back to the client.
  
* '''OpenSim.Data'''.  This is all the low level database manipulation stuffIn here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag), as well as code for [http://www.hibernate.org/ NHibernate] which will hopefully reduce our reliance on manipulation code for specific databases.
+
LL capabilities are a similar message passing mechanism but with HTTP as the carrier rather than UDPThese tend to be used for messages which are not time critical and are often used for moving large amounts of data in newer viewers (e.g. texture and inventory request and delivery, though not other inventory commands such as item move). Again, these are handled by regions and region modules.
  
* '''OpenSim.Framework'''.  General framework code used by the rest of OpenSim which is felt to be not specific to a region. 
+
A simulator can pass messages back to the region via the event queue capability, which is essentially a poll by the viewer to the simulator to fetch messages.
** ''OpenSim.Framework.Communications''.  Contains asset and inventory caching code (to cut down on communication with a remote asset or inventory service), and contains CAPS code, a Linden approach to splitting some aspects of the Second Life protocol into independent, secured services.  See a [http://mrtopf.de/blog/secondlife/slga-capabilities-explained-technical/ blog post] from Tao for more background on this.
+
** ''OpenSim.Framework.Servers''.  Contains generic server code, such as the base OpenSim server and http servers.
+
** ''OpenSim.Framework.Stats''.  Collects statistical information on OpenSim operations.
+
  
* '''OpenSim.Server''': these are things that are part of the R.O.B.U.S.T. services, including grid servers, avatar servers, asset servers, inventory servers, and the like.
+
Regions and region modules will in turn communicate with backend grid services to fetch required data (e.g. assets), manipulate inventory, etc.  In OpenSimulator standalone mode, services will be in the same process as OpenSim.exe.  In grid mode, they will be hosted in a separate Robust.exe process.
 +
 
 +
More exotic configurations of OpenSimulator would complicate this diagram.  For instance, it's possible to configure OpenSimulator to fetch assets directly from capabilities implemented in the same process as the grid service rather than go through the simulator.
 +
 
 +
= Packages =
 +
 
 +
* '''OpenSim'''. This contains the classes which launch the OpenSimulator region server and handle the console
 +
** Startup
 +
*** The Main() for <tt>OpenSim.exe</tt> can be found in the file <tt>OpenSim/Region/Application/Application.cs</tt>. It does a bunch of configuration checks, but the meat of it is the instantiation of an '''OpenSim''' object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
 +
*** OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
  
* '''OpenSim.Region'''.  Contains most of the meaty code that it specifically concerned with region manipulation.
+
* '''OpenSim.Addons'''.  OpenSimulator components which came originally from the Diva distribution.  Hence their different organization compared to other features.
** ''OpenSim.Region.ClientStack.LindenUDP''.  The code necessary for communicating with clients using the Second Life protocol.
+
** '''OpenSim.Addons.Groups'''.  Core support for user groups.
** ''OpenSim.Region.Communications.Local''.  Code used for communicating with non-region services (user, asset, grid, etc.) when OpenSim is running in standalone mode.
+
** '''OpenSim.Addons.OfflineIM'''.  Core support for offline IM.
** ''OpenSim.Region.Communications.OGS1''.  Code for communicating with remote UGAI services using the Open Grid Services 1 protocol.
+
* '''OpenSim.ApplicationPlugins'''. Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region server. See [[RegionModules]] for more details or one of [http://justincc.wordpress.com/2008/05/08/opensim-tech-basics-my-god-its-full-of-modules/ justincc's blogposts] for a higher level overview (which should really be in this wiki).
** ''OpenSim.Region.Environment''.  A large package containing both the vast majority of the region modules (categorized under modules) and the central scene manipulation code (under Scenes, appropriately enough)Modules here do everything from handling region chat to loading and saving terrain.
+
* '''OpenSim.Capabilities'''.  Basic structures for [[capabilities]] support.
** ''OpenSim.Region.Modules''.  More region module code.  Actually, I'm not too sure why these aren't in the OpenSim.Region.Environment package.
+
* '''OpenSim.ConsoleClient'''.  [[Client|RestConsole]] for connecting remotely to the OpenSimulator console.  Not widely used as of yet - more commonly OpenSimulator is started in a [https://www.gnu.org/software/screen/ screen] instance in Linux and in separate terminals in Windows.
** ''OpenSim.Region.Physics''. General physics framework code and plugins for specific engines (BulletX, ODE, etc).  The ODE code is by far the most reliable and stable.
+
* '''OpenSim.Data'''. This is all the low level database manipulation stuff. In here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag).
** ''OpenSim.Region.ScriptEngine''. Script engine code that powers the scripts run within OpenSim. Currently there are two engines, the classic [[DotNetEngine]] and the relatively new [[XEngine]].
+
* '''OpenSim.Framework'''. General framework code used by the rest of OpenSimulator which is either not specific to regions or used in common with regions and other classes.
** ''[[Overview of How RegionsWork|OpenSim.Region.Framework.Scene.Scene]]'' is the "heart" of OpenSim functionalityIt has the "main loop" for a region, in the method <tt>Update()</tt>.
+
** '''OpenSim.Framework.AssetLoader.Filesystem'''.  Small DLL used to load library assets from the filesystem.
 +
** '''OpenSim.Framework.Communications'''.  Some comms related odds and ends.  Notably request limiting code in OpenSimulator master git branch post 0.7.6.
 +
** '''OpenSim.Framework.Configuration.HTTP'''.  Old code for loading configuration via HTTP.  May have decayed beyond usefulness.
 +
** '''OpenSim.Framework.Configuration.XML'''.  Code for loading XML config files (as opposed to INI).  May have decayed beyond usefulness.
 +
** '''OpenSim.Framework.Console'''.  Framework classes for the OpenSimulator console.
 +
** '''OpenSim.Framework.Monitoring'''.  Statistics gathering and monitoring infrastructure.
 +
** '''OpenSim.Framework.RegionLoader.FileSystem'''.  Code for loading region configuration from the filesystem (e.g. bin/Regions/Regions.ini).
 +
** '''OpenSim.Framework.RegionLoader.Web'''.  Code for loading region configuration remotely.
 +
** '''OpenSim.Framework.Serialization'''.  Infrastructure for OAR and IAR read and write.
 +
** '''OpenSim.Framework.Servers'''. Contains generic server code, such as the base OpenSimulator server and http servers.
 +
* '''OpenSim.Region'''. Contains most of the meaty code that it specifically concerned with regions/scenes.
 +
** '''OpenSim.Region.ClientStack'''. Small amount of infrastructure for making different UDP client stacks available (though currently only LLUDP exists).
 +
** '''OpenSim.Region.ClientStack.LindenCaps'''.  The code necessary for communicating with clients using the HTTP (capability) part of the Second Life protocol.  Glues data going to/from the client to region modules that handle specific aspects.
 +
** '''OpenSim.Region.ClientStack.LindenUDP'''.  The code necessary for communicating with clients using the UDP part of the Second Life protocol.  Glues data going to/from the client to scene and region module code.
 +
** '''OpenSim.Region.CoreModules'''.  The region modules considered core to OpenSimulator (and thus come bundled with it). This covers a vast range.  For example, the Avatar/Attachments/AttachmentsModule handles avatar attachments (though with many structures still in core region code).  The Framework/EntityTransfer/EntityTransferModule handles region crossings and teleports (though again in co-operation with large chunks of core region code).
 +
** '''OpenSim.Region.DataSnapshot'''.  Code for creating an XML file that summarises the contents of the region.  Could be exposed to an external search engine.
 +
** '''OpenSim.Region.Framework'''.  The framework for the region itself, including scene graph, scene presence (agent) classes, scene object classes, inventory, etcCould be considered the core of the simulator.
 +
** '''OpenSim.Region.OptionalModules'''.  Optional region modules that aren't considered core to the simulator. In principle, could be removed yet have a working Second Life compatible simulator, though this has not been well tested.
 +
** '''OpenSim.Region.Physics'''. General physics framework code and plugins for specific engines (BulletSim, ODE, etc).  Only the ODE and BulletSim modules are mature or complete enough for use.
 +
** '''OpenSim.Region.ScriptEngine'''. Script engine code that powers the scripts run within OpenSimulator. Currently there is only one engine bundled with OpenSimulator, [[XEngine]].
 +
** '''OpenSim.Region.UserStatistics'''.  [[Web Statistics Module]] for exposing some simulator information via a webpage.
 +
* '''OpenSim.Server'''.  Server infrastructure.
 +
** '''OpenSim.Server.Base'''. Common server infrastructure for the simulator and Robust grid servers.
 +
** '''OpenSim.Server.Handlers'''.  Classes for handling raw incoming data and translating these to service requests.  Used mainly in grid mode for ROBUST server instances.  For instance, Asset/AssetServerGetHandler translates internal asset fetch requests to the asset service and replies with the results.
 +
* '''OpenSim.Services'''Code which provides both service implementations (e.g. AssetService to read and write assets to persistent storage) interfaces and infrastructure (chiefly OpenSim.Services.Connectors which creates the actually HTTP requests to invoke services remotely).  In grid mode, these are instantiated within Robust.exe instances.  In standalone mode, these are instantiated directly in OpenSim.exe.
 +
* '''OpenSim.Tests.*'''.  Test code, both common infrastructure used in regression tests and some standalone client test code.  Actual regression tests are spread throughout the code base in separate *.Test packages (e.g. OpenSim.Region.Framework.Tests project).
 +
* '''OpenSim.Tools'''.  Some OpenSimulator associated tools.  Not used in actual OpenSimulator operation.
 +
* '''pCampbot'''.  A test tool which uses bots to simulate activity load on a simulator.  See [[pCampbot]] for more details.  Not used in actual OpenSimulator operation.
  
 +
== Coding restrictions ==
  
* '''OpenSim.Tests'''.  Test code.  Really, we don't actually have much of this (yet).
+
Because OpenSimulator has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that the minimum version of [http://en.wikipedia.org/wiki/Mono_(software) Mono] we require supports.

Revision as of 02:07, 6 April 2019

Contents

Introduction

OpenSimulator is a large and organically grown project, so its architecture is still subject to change. As such, there are very few formal internal interfaces - region modules operating by plugging in directly to the core internals.

Architecture

An diagram of the core parts of the simulator code organization.

This is a very high level diagram of the main chunks of the simulator code organization. Arrows indicate direction of request initiation.

A viewer (at the bottom) interacts with the simulator by sending messages through both UDP and HTTP.

Regions and region modules register for UDP message events (e.g. user command inputs such as move forward, inventory manipulation requests, etc). These events are invoked when the LLUDP client stack receives the relevant requests from the viewer. In turn, the regions and region modules will pass messages back to the LLUDP clientstack (e.g. movements of avatars and objects) for delivery back to the client.

LL capabilities are a similar message passing mechanism but with HTTP as the carrier rather than UDP. These tend to be used for messages which are not time critical and are often used for moving large amounts of data in newer viewers (e.g. texture and inventory request and delivery, though not other inventory commands such as item move). Again, these are handled by regions and region modules.

A simulator can pass messages back to the region via the event queue capability, which is essentially a poll by the viewer to the simulator to fetch messages.

Regions and region modules will in turn communicate with backend grid services to fetch required data (e.g. assets), manipulate inventory, etc. In OpenSimulator standalone mode, services will be in the same process as OpenSim.exe. In grid mode, they will be hosted in a separate Robust.exe process.

More exotic configurations of OpenSimulator would complicate this diagram. For instance, it's possible to configure OpenSimulator to fetch assets directly from capabilities implemented in the same process as the grid service rather than go through the simulator.

Packages

  • OpenSim. This contains the classes which launch the OpenSimulator region server and handle the console
    • Startup
      • The Main() for OpenSim.exe can be found in the file OpenSim/Region/Application/Application.cs. It does a bunch of configuration checks, but the meat of it is the instantiation of an OpenSim object, calling its Startup() method (which gets things rolling in separate threads), and then blocking the initial thread by prompting on the main console (using the Prompt() method MainConsole.Instance)
      • OpenSim.OpenSim.Startup() comes from an ancestor, OpenSim.Framework.Servers.BaseOpenSimServer; it calls StartupSpecific()
  • OpenSim.Addons. OpenSimulator components which came originally from the Diva distribution. Hence their different organization compared to other features.
    • OpenSim.Addons.Groups. Core support for user groups.
    • OpenSim.Addons.OfflineIM. Core support for offline IM.
  • OpenSim.ApplicationPlugins. Each package is an individual application plugin - an application plugin is a modular piece of code that has an effect on the entire region server. See RegionModules for more details or one of justincc's blogposts for a higher level overview (which should really be in this wiki).
  • OpenSim.Capabilities. Basic structures for capabilities support.
  • OpenSim.ConsoleClient. RestConsole for connecting remotely to the OpenSimulator console. Not widely used as of yet - more commonly OpenSimulator is started in a screen instance in Linux and in separate terminals in Windows.
  • OpenSim.Data. This is all the low level database manipulation stuff. In here, you'll see packages for SQLite, MySQL and MSSQL (which tends to lag).
  • OpenSim.Framework. General framework code used by the rest of OpenSimulator which is either not specific to regions or used in common with regions and other classes.
    • OpenSim.Framework.AssetLoader.Filesystem. Small DLL used to load library assets from the filesystem.
    • OpenSim.Framework.Communications. Some comms related odds and ends. Notably request limiting code in OpenSimulator master git branch post 0.7.6.
    • OpenSim.Framework.Configuration.HTTP. Old code for loading configuration via HTTP. May have decayed beyond usefulness.
    • OpenSim.Framework.Configuration.XML. Code for loading XML config files (as opposed to INI). May have decayed beyond usefulness.
    • OpenSim.Framework.Console. Framework classes for the OpenSimulator console.
    • OpenSim.Framework.Monitoring. Statistics gathering and monitoring infrastructure.
    • OpenSim.Framework.RegionLoader.FileSystem. Code for loading region configuration from the filesystem (e.g. bin/Regions/Regions.ini).
    • OpenSim.Framework.RegionLoader.Web. Code for loading region configuration remotely.
    • OpenSim.Framework.Serialization. Infrastructure for OAR and IAR read and write.
    • OpenSim.Framework.Servers. Contains generic server code, such as the base OpenSimulator server and http servers.
  • OpenSim.Region. Contains most of the meaty code that it specifically concerned with regions/scenes.
    • OpenSim.Region.ClientStack. Small amount of infrastructure for making different UDP client stacks available (though currently only LLUDP exists).
    • OpenSim.Region.ClientStack.LindenCaps. The code necessary for communicating with clients using the HTTP (capability) part of the Second Life protocol. Glues data going to/from the client to region modules that handle specific aspects.
    • OpenSim.Region.ClientStack.LindenUDP. The code necessary for communicating with clients using the UDP part of the Second Life protocol. Glues data going to/from the client to scene and region module code.
    • OpenSim.Region.CoreModules. The region modules considered core to OpenSimulator (and thus come bundled with it). This covers a vast range. For example, the Avatar/Attachments/AttachmentsModule handles avatar attachments (though with many structures still in core region code). The Framework/EntityTransfer/EntityTransferModule handles region crossings and teleports (though again in co-operation with large chunks of core region code).
    • OpenSim.Region.DataSnapshot. Code for creating an XML file that summarises the contents of the region. Could be exposed to an external search engine.
    • OpenSim.Region.Framework. The framework for the region itself, including scene graph, scene presence (agent) classes, scene object classes, inventory, etc. Could be considered the core of the simulator.
    • OpenSim.Region.OptionalModules. Optional region modules that aren't considered core to the simulator. In principle, could be removed yet have a working Second Life compatible simulator, though this has not been well tested.
    • OpenSim.Region.Physics. General physics framework code and plugins for specific engines (BulletSim, ODE, etc). Only the ODE and BulletSim modules are mature or complete enough for use.
    • OpenSim.Region.ScriptEngine. Script engine code that powers the scripts run within OpenSimulator. Currently there is only one engine bundled with OpenSimulator, XEngine.
    • OpenSim.Region.UserStatistics. Web Statistics Module for exposing some simulator information via a webpage.
  • OpenSim.Server. Server infrastructure.
    • OpenSim.Server.Base. Common server infrastructure for the simulator and Robust grid servers.
    • OpenSim.Server.Handlers. Classes for handling raw incoming data and translating these to service requests. Used mainly in grid mode for ROBUST server instances. For instance, Asset/AssetServerGetHandler translates internal asset fetch requests to the asset service and replies with the results.
  • OpenSim.Services. Code which provides both service implementations (e.g. AssetService to read and write assets to persistent storage) interfaces and infrastructure (chiefly OpenSim.Services.Connectors which creates the actually HTTP requests to invoke services remotely). In grid mode, these are instantiated within Robust.exe instances. In standalone mode, these are instantiated directly in OpenSim.exe.
  • OpenSim.Tests.*. Test code, both common infrastructure used in regression tests and some standalone client test code. Actual regression tests are spread throughout the code base in separate *.Test packages (e.g. OpenSim.Region.Framework.Tests project).
  • OpenSim.Tools. Some OpenSimulator associated tools. Not used in actual OpenSimulator operation.
  • pCampbot. A test tool which uses bots to simulate activity load on a simulator. See pCampbot for more details. Not used in actual OpenSimulator operation.

Coding restrictions

Because OpenSimulator has to run on both Windows and Mono (usually on Linux or Mac), we are limited to the parts of .net that the minimum version of Mono we require supports.

Personal tools
General
About This Wiki