OpenSim Services and Service Connectors/de - Revision history

Manni at 18:51, 21 February 2023

2023-02-21T18:51:06Z

Manni at 18:49, 21 February 2023

2023-02-21T18:49:57Z

Show changes

Manni at 18:40, 21 February 2023

2023-02-21T18:40:31Z

Manni: Created page with "{{Quicklinks|OpenSim Services and Service Connectors}} {{Warning|This documentation, though still useful, is rather out of date. Please see Connectors for more accurat..."

2023-02-21T18:33:39Z

Created page with "{{Quicklinks|OpenSim Services and Service Connectors}} {{Warning|This documentation, though still useful, is rather out of date. Please see Connectors for more accurat..."

New page

{{Quicklinks|OpenSim Services and Service Connectors}}

{{Warning|This documentation, though still useful, is rather out of date.

Please see [[Connectors]] for more accurate information.

Or see [[Services]] for more information about services themselves.}}

= Introduction to OpenSimulator Services and Service Connectors =

The OpenSimulator framework is becoming so many things to so many people that it has reached the point where the original architecture of the software itself, largely inspired by what the SL Client dictates and by the Linden Lab grid, is hampering progress on all those fronts. We need to rethink the architecture of the software in order to be able to support the variety of things that people want to do with OpenSimulator without having to go through lengthy design discussions and negotiations for it to do "the right thing", which obviously doesn't exist. For some, "the right thing" is a server infrastructure that is a 100% reproduction of Second Life; for others, "the right thing" is a 3D server infrastructure that is 100% compatible with the Web; for others, "the right thing" is a desktop 3D simulator; and for many others "the right thing" is many points in between.

This proposal comes in the sequence of an email thread on the -dev mailing list:
https://lists.berlios.de/pipermail/opensim-dev/2009-April/006325.html
The general ideas here are similar to those already in place for using Database and Physics engines. This proposal is about extending those general ideas to all Resource Services, which carries the additional difficulty of having to deal with service placement and service composition.

Proposers: Diva and Melanie

Closing date: 5/28/09

== Proposal Summary ==

[[Image:Connector_Architecture.jpg|600px]]

We propose a new '''software architecture''' that can easily accommodate all the '''system architectures''' that people want to build. The basic concepts of this new software architecture are (1) Service Interfaces; (2) Service Connectors; (3) Service Implementations; and (4) Server Shells. Service implementations are pieces of code that can be loaded into a server shell, along with their "in" service connectors that receive service requests from clients. Clients access services through service interfaces, and by loading the corresponding "out" service connectors that send requests to the service implementations. The figure above illustrates the general concept.

A Service is the collection of its implementation and its connectors, and obeys a certain service interface. The people that implement the service also implement its connectors. Therefore, the details of the protocol between the client-side and the server-side are invisible to the service clients, which only know about the service interface. As long as the interface is preserved, switching service implementations is a trivial matter of replacing the out connector in the client. The specification of all of these elements is done at initialisation time, by loading the specified DLLs and/or by using a generic plugin infrastructure.

== Benefits ==

The figure below shows 4 different system architectures that can be trivially configured if the proposed software architecture is in place.

[[Image:System_Architectures.jpg]]

Note that these are only 4 out of a variety of combinations.

The benefits of this software architecture are the following:
* We move away from the current boolean architectural choices given by the variable "gridmode=true|false", and into a model where simulators can be configured in a variety of hybrid modes.
* The dynamic loading of service connectors into the simulator allows a variety of protocols "on-the-wire" between the simulator and the services without having to change the simulator code. (As long as that service implementation can honor the established service interface for it)
* [Corollary of the previous] All discussions about simulator-service protocols (HTTP vs XMLRPC vs ...) become irrelevant to the OpenSimulator framework itself. Opensim will continue to provide a few service implementations and their connectors out-of-the-box, but anyone can replace those with their own without having to make any changes to the source code of the Simulator, by making the corresponding connector DLL available to the simulators.
* All OpenSimulator servers become server shells of the same type. That is, they are all configured through the .ini mechanism, and they can run any of the services and service connectors. Therefore, it also becomes trivially possible to have system architectures where some resource service uses another resource service directly. For example, the Inventory server may use the Asset server in order to provide a combined IInventoryService+IAssetService interface to viewers.

== Implementation of the Proposed Software Architecture ==

The implementation is illustrated using the Asset Service as an example. Here is the overview of the new organization. The boxes here denote, roughly, DLLs. Note that this is still under heavy construction, and that some of this organization and vocabulary may change.

OpenSim/

Region/

CoreModules/
ServiceConnectors/
(place to put all OUT service connectors from the simulator)
Asset/
HGAssetBroker.cs
LocalssetServiceConnector.cs
RemoteAssetServiceConnector.cs
Grid/
...
Interregion/
...
Inventory/
...
Messaging/
...
User/
...

Server/

ServerMain.cs (the server shell, produces OpemSim.Services.exe)

Base/
(Base and basic classes for the HTTP server, one DLL)
HttpServerBase.cs
ServerUtils.cs
ServicesServerBase.cs

Handlers/
(place to put all IN connectors and network service handlers)
Asset/
AssetServerConnector.cs
AssetServerDeleteHandler.cs
AssetServerGetHandler.cs
AssetServerPostHandler.cs
...
Base/
(Base and basic classes for IN connectors)
ServerConnector.cs

Services/
(service implementations, each on its own dll)

AssetService/

Base/

GridService/

InventoryService/

MessagingService/

UserService/

(meat of the OUT connectors, one DLL)
Connectors/
...

(service interfaces, one DLL)
Interfaces/
...

=== Service Interfaces ===

'''OpenSim.Services.Interfaces''': This DLL contains all the service interfaces known to OpenSimulator.

Here is how IAssetService looks like:

<source lang="csharp">
public interface IAssetService
{
// Three different ways to retrieve an asset
//
AssetBase Get(string id);
AssetMetadata GetMetadata(string id);
byte[] GetData(string id);

bool Get(string id, Object sender, AssetRetrieved handler);

// Creates a new asset
// Returns a random ID if none is passed into it
//
string Store(AssetBase asset);

// Attachments and bare scripts need this!!
//
bool UpdateContent(string id, byte[] data);

// Kill an asset
//
bool Delete(string id);
}

</source>

=== Connectors ===

The direction of a connector is determined by the direction of the connection being made, not by the direction of the data flow. Therefore, an OUT connector is always a client, while an IN connector is always a server. Conversely, an OUT connector will provide an interface, while an IN connector will consume an interface. Interfaces are consistent across the entire chain of connectors, therefore IN connectors can be directly connected to OUT connectors, which would create a proxy server.

==== Out Connectors ====

'''OpenSim.Region.CoreModules.ServiceConnectors''': this is where all concrete OUT connectors used by the simulator hang out.

'''OpenSim.Services.Connectors''': this is where all the base classes of the OUT connectors hang out. These ones can be reused by servers other than the simulator.

An OUT connector primarily exposes its interfaces, it may have a secondary interface to be able to load as a region module. Here is a snippet of the RemoteAssetServiceConnector, which, as the name says, connects to a remote asset server:

<source lang="csharp">
public class RemoteAssetServicesConnector :
AssetServicesConnector, ISharedRegionModule, IAssetService
{

private bool m_Enabled = false;
private IImprovedAssetCache m_Cache;

public string Name
{
get { return "RemoteAssetServicesConnector"; }
}

public override void Initialise(IConfigSource source)
{
IConfig moduleConfig = source.Configs["Modules"];
if (moduleConfig != null)
{
string name = moduleConfig.GetString("AssetServices", "");
if (name == Name)
{
IConfig assetConfig = source.Configs["AssetService"];
if (assetConfig == null)
{
m_log.Error("[ASSET CONNECTOR]: AssetService missing from OpenSim.ini");
return;
}
m_Enabled = true;
base.Initialise(source);
m_log.Info("[ASSET CONNECTOR]: Remote assets enabled");
}
}
}

public void AddRegion(Scene scene)
{
if (!m_Enabled)
return;
scene.RegisterModuleInterface<IAssetService>(this);
}
...
}
</source>

The above code is just the region module, which is enabled or not depending on configuration variables (described further down). Two things are noteworthy:

* This module registers with Scene as the simulator's IAssetService. In other words, this modules is the direct provider of that interface within the simulator. All asset operations go through it first.
* This class extends AssetServicesConnector, which is the class that has the 'meat' of the service. A small part of it is presented below:

<source lang="csharp">

public class AssetServicesConnector : IAssetService
{
private string m_ServerURI = String.Empty;
private IImprovedAssetCache m_Cache = null;

public AssetServicesConnector(string serverURI)
{
m_ServerURI = serverURI;
}

public AssetServicesConnector(IConfigSource source)
{
Initialise(source);
}

public virtual void Initialise(IConfigSource source)
{
IConfig assetConfig = source.Configs["AssetService"];
if (assetConfig == null)
{
m_log.Error("[ASSET CONNECTOR]: AssetService missing from OpenSim.ini");
throw new Exception("Asset connector init error");
}

string serviceURI = assetConfig.GetString("AssetServerURI",
String.Empty);

if (serviceURI == String.Empty)
{
m_log.Error("[ASSET CONNECTOR]: No Server URI named in section AssetService");
throw new Exception("Asset connector init error");
}
m_ServerURI = serviceURI;
}

public AssetBase Get(string id)
{
string uri = m_ServerURI + "/assets/" + id;

AssetBase asset = null;
if (m_Cache != null)
asset = m_Cache.Get(id);

if (asset == null)
{
asset = SynchronousRestObjectRequester.
MakeRequest<int, AssetBase>("GET", uri, 0);

if (m_Cache != null)
m_Cache.Cache(asset);
}
return asset;
}
...
}

</source>

The class above is the one that actually holds the reference to the remote server, as an URL, and that performs the remote calls to it.

Besides this connector, there are currently 2 other ones that can be used: LocalAssetServiceConnector and HGAssetBroker. The former is meant to be used when the service runs within the simulator's process; the latter is meant to be used for Hypergrided simulators.

==== In Connectors ====

'''OpenSim.Server.Handlers''': this DLL is where all IN connectors hang out.

IN connectors listen for requests and decode them for processing. They are typically bound to a HTTP server, although other transports are possible. An IN connector will acquire an instance of a class exposing the target interface and send all received and decoded data to that class, which may be a consumer (database, etc.), a forwarder/broker (HGBroker) or an OUT connector, which would create a proxy server.
IN connectors can share the same HTTP server, so all services can be contained in a single server process, or split among several processes or hosts.
The proxy configuration is of particular interest, since it allows load balancing and write request routing. For instance, in SQL clusters it is possible to route write requests to different hosts than read requests.

Here is how the IN connector for an asset server looks like:

<source lang="csharp">
public class AssetServiceConnector : ServiceConnector
{
private IAssetService m_AssetService;

public AssetServiceConnector(IConfigSource config, IHttpServer server) :
base(config, server)
{
IConfig serverConfig = config.Configs["AssetService"];
if (serverConfig == null)
throw new Exception("No section 'Server' in config file");

string assetService = serverConfig.GetString("LocalServiceModule",
String.Empty);

if (assetService == String.Empty)
throw new Exception("No AssetService in config file");

Object[] args = { config };
m_AssetService =
ServerUtils.LoadPlugin<IAssetService>(assetService, args);

server.AddStreamHandler(new AssetServerGetHandler(m_AssetService));
server.AddStreamHandler(new AssetServerPostHandler(m_AssetService));
server.AddStreamHandler(new AssetServerDeleteHandler(m_AssetService));
}
}

</source>

Two noteworthy things about this connector:
* It establishes the service handlers on the server
* It loads a DLL containing the service implementation (finally!).

=== Service Implementations ===

'''OpenSim.Services.AssetService''': the default OpenSimulator implementation of the asset service. Here is a snippet:

<source lang="csharp">
public class AssetService : AssetServiceBase, IAssetService
{
public AssetService(IConfigSource config) : base(config)
{
if (m_AssetLoader != null)
{
IConfig assetConfig = config.Configs["AssetService"];
if (assetConfig == null)
throw new Exception("No AssetService configuration");

string loaderArgs = assetConfig.GetString("AssetLoaderArgs",
String.Empty);

m_log.InfoFormat("[ASSET]: Loading default asset set from {0}", loaderArgs);
m_AssetLoader.ForEachDefaultXmlAsset(loaderArgs,
delegate(AssetBase a)
{
Store(a);
});

m_log.Info("[ASSET CONNECTOR]: Local asset service enabled");
}
}

public AssetBase Get(string id)
{
m_log.DebugFormat("[ASSET SERVICE]: Get asset {0}", id);
UUID assetID;

if (!UUID.TryParse(id, out assetID))
return null;

return m_Database.FetchAsset(assetID);
}
...
}
</source>

The above class is loaded by the IN connector shown before.

=== Server Shells ===

'''OpenSim.Services.exe'''

There is now a generic server shell that can load any IN service connectors, along with their specified service implementations. This server shell is configured with a .ini file, using exactly the same configuration variables that pertain to the asset service configuration in OpenSin.ini.

Here is the new server shell:

<source lang="csharp">

public class OpenSimServer
{
protected static HttpServerBase m_Server = null;

protected static List<IServiceConnector> m_ServiceConnectors =
new List<IServiceConnector>();

static int Main(string[] args)
{
m_Server = new HttpServerBase("Asset", args);

IConfig serverConfig = m_Server.Config.Configs["Startup"];
if (serverConfig == null)
{
System.Console.WriteLine("Startup config section missing in .ini file");
throw new Exception("Configuration error");
}

string connList = serverConfig.GetString("ServiceConnectors", String.Empty);
string[] conns = connList.Split(new char[] {',', ' '});

foreach (string conn in conns)
{
if (conn == String.Empty)
continue;

string[] parts = conn.Split(new char[] {':'});
string friendlyName = parts[0];
if (parts.Length > 1)
friendlyName = parts[1];

m_log.InfoFormat("[SERVER]: Loading {0}", friendlyName);

Object[] modargs = { m_Server.Config, m_Server.HttpServer };
IServiceConnector connector =
ServerUtils.LoadPlugin<IServiceConnector>(conn,
modargs);

if (connector != null)
{
m_ServiceConnectors.Add(connector);
m_log.InfoFormat("[SERVER]: {0} loaded successfully", friendlyName);
}
else
{
m_log.InfoFormat("[SERVER]: Failed to load {0}", conn);
}
}
return m_Server.Run();
}
}
</source>

Essentially, this generic server loads all DLLs for the specificed ServiceConnectors configuration variable. Therefore it's trivial to combine several services in a single server shell, or to have several server shells each one running a service.

== Configuration ==

This architecture splits all services and service connectors into individual region modules and DLLs that are then specified in the server's configuration files. Each server -- OpenSim.exe and all OpenSim.Services.exe -- has its own .ini file. There are two ways of configuring the servers, described next.

=== Direct Configuration ===

The familiar .ini file has now several new configuration variable for every service connector. See [[Services and Service Connectors Configuration]] for the extra variables pertaining to configuring the asset service in OpenSim.ini.

As for configuring a resource server, here is an example of an asset server in the new style of server shells (these variables are stored in OpenSim.Services.ini)

[Startup]
; These are also available as command line options

; console = "local" ; Use "basic" to use this on a pipe
; inifile = "OpenSim.Servers.AssetServer.ini"
; logfile = "AssetServer.log" ; Also read from application config file

; Connectors, comma separated
ServiceConnectors = "OpenSim.Server.Handlers.dll:AssetServiceConnector"

[Network]
port = 8003

[AssetService]
LocalServiceModule = "OpenSim.Services.AssetService.dll:AssetService"
StorageProvider = "OpenSim.Data.MySQL.dll"
ConnectionString = "Data Source=localhost;Database=opensim;User ID=opensim;Password=opensim;"
DefaultAssetLoader = "OpenSim.Framework.AssetLoader.Filesystem.dll"
AssetLoaderArgs = "assets/AssetSets.xml"

=== Prepackaged System Architectures ===

While the split in small design elements provides all the flexibility one can get from the framework, the number of configuration variables increases considerably, and can quickly become overwhelming. To address this problem we are working on configuration "includes", which will allow us to provide pre-packaged system architectures, having all the new variables properly set up so that they can produce architectures such as the ones in the picture above.

There will be three prepackaged system architectures: StandaloneGrid, StandardGrid and HyperGrid. Users of the framework can define additional ones.

= See Also =

* [[OpenSim:Introduction_and_Definitions#UGAIM_Services|What are UGAIM Services?]]
* [[OpenSim Services and Service Connectors|Services and Service Connectors Main Page]]
* [[Services and Service Connectors Configuration|Services and Service Connectors Configuration Page]]

= Change Log =

* Diva, 5/22/09, added suggestions from Justincc @ -dev.

[[Category:Users]]
[[Category:Support]]
[[Category:Tech Reference]]
[[Category:Help]]
[[Category:Configuration]]
[[Category:Getting Started]]
[[Category:Development]]

← Older revision		Revision as of 18:51, 21 February 2023
Line 497:		Line 497:
	[[Category:Getting Started]]		[[Category:Getting Started]]
	[[Category:Development]]		[[Category:Development]]
		+
		+	[[Category:German Translations]]

@@ Line 1: / Line 1: @@
 {{Quicklinks|OpenSim Services and Service Connectors}}
-{{Warning|This documentation, though still useful, is rather out of date.
+{{Warning|Diese Dokumentation ist zwar immer noch nützlich, aber ziemlich veraltet.
-Please see [[Connectors]] for more accurate information.
+Genauere Informationen finden Sie unter [[Connectors]].
-Or see [[Services]] for more information about services themselves.}}
+Oder siehe [[Services]]  für weitere Informationen über die Dienste selbst.}}
-The OpenSimulator framework is becoming so many things to so many people that it has reached the point where the original architecture of the software itself, largely inspired by what the SL Client dictates and by the Linden Lab grid, is hampering progress on all those fronts. We need to rethink the architecture of the software in order to be able to support the variety of things that people want to do with OpenSimulator without having to go through lengthy design discussions and negotiations for it to do "the right thing", which obviously doesn't exist. For some, "the right thing" is a server infrastructure that is a 100% reproduction of Second Life; for others, "the right thing" is a 3D server infrastructure that is 100% compatible with the Web; for others, "the right thing" is a desktop 3D simulator; and for many others "the right thing" is many points in between.
+= Einführung in OpenSimulator Services und Service Connectors =
-This proposal comes in the sequence of an email thread on the -dev mailing list:
+Das OpenSimulator-Framework wird für so viele Menschen zu so vielen Dingen, dass es den Punkt erreicht hat, an dem die ursprüngliche Architektur der Software selbst, die weitgehend von den Vorgaben des SL-Clients und dem Linden Lab-Grid inspiriert ist, den Fortschritt an all diesen Fronten behindert. Wir müssen die Architektur der Software überdenken, um die Vielfalt der Dinge unterstützen zu können, die Menschen mit OpenSimulator tun möchten, ohne langwierige Designdiskussionen und Verhandlungen führen zu müssen, damit sie "das Richtige" tut, was offensichtlich der Fall ist nicht vorhanden. Für manche ist „das Richtige“ eine Server-Infrastruktur, die eine 100-prozentige Reproduktion von Second Life ist; für andere ist „das Richtige“ eine 3D-Server-Infrastruktur, die zu 100 % mit dem Web kompatibel ist; für andere ist „das Richtige“ ein Desktop-3D-Simulator;
 https://lists.berlios.de/pipermail/opensim-dev/2009-April/006325.html
-The general ideas here are similar to those already in place for using Database and Physics engines. This proposal is about extending those general ideas to all Resource Services, which carries the additional difficulty of having to deal with service placement and service composition.
+Die allgemeinen Ideen hier ähneln denen bereits darin Ort für die Verwendung von Datenbank- und Physik-Engines. Bei diesem Vorschlag geht es darum, diese allgemeinen Ideen auf alle Ressourcendienste auszudehnen, was die zusätzliche Schwierigkeit mit sich bringt, sich mit Dienstplatzierung und Dienstzusammensetzung befassen zu müssen.
-Proposers: Diva and Melanie
+Vorschlagende: Diva und Melanie
 Closing date: 5/28/09
-== Proposal Summary ==
+== Zusammenfassung ==
 [[Image:Connector_Architecture.jpg|600px]]
-We propose a new '''software architecture''' that can easily accommodate all the '''system architectures''' that people want to build. The basic concepts of this new software architecture are (1) Service Interfaces; (2) Service Connectors; (3) Service Implementations; and (4) Server Shells. Service implementations are pieces of code that can be loaded into a server shell, along with their "in" service connectors that receive service requests from clients. Clients access services through service interfaces, and by loading the corresponding "out" service connectors that send requests to the service implementations. The figure above illustrates the general concept.
+Wir schlagen eine neue Softwarearchitektur vor , die problemlos alle Systemarchitekturen aufnehmen kann , die Menschen aufbauen möchten. Die Grundkonzepte dieser neuen Softwarearchitektur sind (1) Dienstschnittstellen; (2) Serviceanschlüsse; (3) Dienstimplementierungen; und (4) Server-Shells. Dienstimplementierungen sind Codeteile, die in eine Server-Shell geladen werden können, zusammen mit ihren „in“-Dienstkonnektoren, die Dienstanforderungen von Clients empfangen. Clients greifen über Dienstschnittstellen auf Dienste zu und indem sie die entsprechenden "Ausgangs"-Dienstkonnektoren laden, die Anforderungen an die Dienstimplementierungen senden. Die obige Abbildung veranschaulicht das allgemeine Konzept.
-A Service is the collection of its implementation and its connectors, and obeys a certain service interface. The people that implement the service also implement its connectors. Therefore, the details of the protocol between the client-side and the server-side are invisible to the service clients, which only know about the service interface. As long as the interface is preserved, switching service implementations is a trivial matter of replacing the out connector in the client. The specification of all of these elements is done at initialisation time, by loading the specified DLLs and/or by using a generic plugin infrastructure.
+Ein Dienst ist die Sammlung seiner Implementierung und seiner Konnektoren und gehorcht einer bestimmten Dienstschnittstelle. Die Personen, die den Dienst implementieren, implementieren auch seine Konnektoren. Daher sind die Details des Protokolls zwischen der Clientseite und der Serverseite für die Dienstclients unsichtbar, die nur von der Dienstschnittstelle wissen. Solange die Schnittstelle erhalten bleibt, ist das Wechseln von Dienstimplementierungen eine triviale Angelegenheit, da der Out-Connector im Client ersetzt wird. Die Spezifikation all dieser Elemente erfolgt zum Zeitpunkt der Initialisierung, indem die angegebenen DLLs geladen und/oder eine generische Plugin-Infrastruktur verwendet werden.
-== Benefits ==
+== Vorteile ==
-The figure below shows 4 different system architectures that can be trivially configured if the proposed software architecture is in place.
+Die folgende Abbildung zeigt 4 verschiedene Systemarchitekturen, die einfach konfiguriert werden können, wenn die vorgeschlagene Softwarearchitektur vorhanden ist.
 [[Image:System_Architectures.jpg]]
-Note that these are only 4 out of a variety of combinations.
+Beachten Sie, dass dies nur 4 aus einer Vielzahl von Kombinationen sind.
 == Implementation of the Proposed Software Architecture ==