Home

 

Download

 

News

 

Support

 

Admin

 

Dev

 

Screen Shots

 

Grid List

 

Bugs

Languages:

English    Deutsch

Einführung in OpenSimulator Services und Service Connectors

Das OpenSimulator-Framework wird für so viele Menschen zu so vielen Dingen, dass es den Punkt erreicht hat, an dem die ursprüngliche Architektur der Software selbst, die weitgehend von den Vorgaben des SL-Clients und dem Linden Lab-Grid inspiriert ist, den Fortschritt an all diesen Fronten behindert. Wir müssen die Architektur der Software überdenken, um die Vielfalt der Dinge unterstützen zu können, die Menschen mit OpenSimulator tun möchten, ohne langwierige Designdiskussionen und Verhandlungen führen zu müssen, damit sie "das Richtige" tut, was offensichtlich der Fall ist nicht vorhanden. Für manche ist „das Richtige“ eine Server-Infrastruktur, die eine 100-prozentige Reproduktion von Second Life ist; für andere ist „das Richtige“ eine 3D-Server-Infrastruktur, die zu 100 % mit dem Web kompatibel ist; für andere ist „das Richtige“ ein Desktop-3D-Simulator;

Dieser Vorschlag kommt in der Folge eines E-Mail-Threads auf der -dev-Mailingliste: https://lists.berlios.de/pipermail/opensim-dev/2009-April/006325.html Die allgemeinen Ideen hier ähneln denen bereits darin Ort für die Verwendung von Datenbank- und Physik-Engines. Bei diesem Vorschlag geht es darum, diese allgemeinen Ideen auf alle Ressourcendienste auszudehnen, was die zusätzliche Schwierigkeit mit sich bringt, sich mit Dienstplatzierung und Dienstzusammensetzung befassen zu müssen.

Vorschlagende: Diva und Melanie

Closing date: 5/28/09

Zusammenfassung

Wir schlagen eine neue Softwarearchitektur vor , die problemlos alle Systemarchitekturen aufnehmen kann , die Menschen aufbauen möchten. Die Grundkonzepte dieser neuen Softwarearchitektur sind (1) Dienstschnittstellen; (2) Serviceanschlüsse; (3) Dienstimplementierungen; und (4) Server-Shells. Dienstimplementierungen sind Codeteile, die in eine Server-Shell geladen werden können, zusammen mit ihren „in“-Dienstkonnektoren, die Dienstanforderungen von Clients empfangen. Clients greifen über Dienstschnittstellen auf Dienste zu und indem sie die entsprechenden "Ausgangs"-Dienstkonnektoren laden, die Anforderungen an die Dienstimplementierungen senden. Die obige Abbildung veranschaulicht das allgemeine Konzept.

Ein Dienst ist die Sammlung seiner Implementierung und seiner Konnektoren und gehorcht einer bestimmten Dienstschnittstelle. Die Personen, die den Dienst implementieren, implementieren auch seine Konnektoren. Daher sind die Details des Protokolls zwischen der Clientseite und der Serverseite für die Dienstclients unsichtbar, die nur von der Dienstschnittstelle wissen. Solange die Schnittstelle erhalten bleibt, ist das Wechseln von Dienstimplementierungen eine triviale Angelegenheit, da der Out-Connector im Client ersetzt wird. Die Spezifikation all dieser Elemente erfolgt zum Zeitpunkt der Initialisierung, indem die angegebenen DLLs geladen und/oder eine generische Plugin-Infrastruktur verwendet werden.

Vorteile

Die folgende Abbildung zeigt 4 verschiedene Systemarchitekturen, die einfach konfiguriert werden können, wenn die vorgeschlagene Softwarearchitektur vorhanden ist.

Beachten Sie, dass dies nur 4 aus einer Vielzahl von Kombinationen sind.

Die Vorteile dieser Softwarearchitektur sind die folgenden:

* Wir entfernen uns von den aktuellen booleschen Architekturoptionen, die durch die Variable "gridmode=true|false" gegeben sind, und hin zu einem Modell, bei dem Simulatoren in einer Vielzahl von Hybridmodi konfiguriert werden können.
* Das dynamische Laden von Dienstkonnektoren in den Simulator ermöglicht eine Vielzahl von Protokollen „on-the-wire“ zwischen dem Simulator und den Diensten, ohne dass der Simulatorcode geändert werden muss. (Solange diese Dienstimplementierung die etablierte Dienstschnittstelle dafür berücksichtigen kann)
   [Ergänzung des vorherigen] Alle Diskussionen über Simulator-Service-Protokolle (HTTP vs. XMLRPC vs. ...) werden für das OpenSimulator-Framework selbst irrelevant. Opensim wird weiterhin einige Dienstimplementierungen und ihre Konnektoren out-of-the-box bereitstellen, aber jeder kann diese durch seine eigenen ersetzen, ohne Änderungen am Quellcode des Simulators vornehmen zu müssen, indem er die entsprechende Konnektor-DLL zur Verfügung stellt die Simulatoren.
* Alle OpenSimulator-Server werden zu Server-Shells des gleichen Typs. Das heißt, sie werden alle über den .ini-Mechanismus konfiguriert und können alle Dienste und Dienstconnectors ausführen. Daher wird es auch trivialerweise möglich, Systemarchitekturen zu haben, bei denen ein Ressourcendienst einen anderen Ressourcendienst direkt verwendet. Beispielsweise kann der Inventarserver den Asset-Server verwenden, um Betrachtern eine kombinierte IInventoryService+IAssetService-Schnittstelle bereitzustellen.

Implementation of the Proposed Software Architecture

The implementation is illustrated using the Asset Service as an example. Here is the overview of the new organization. The boxes here denote, roughly, DLLs. Note that this is still under heavy construction, and that some of this organization and vocabulary may change.

OpenSim/

  Region/

    CoreModules/
      ServiceConnectors/
       (place to put all OUT service connectors from the simulator)
        Asset/
          HGAssetBroker.cs
          LocalssetServiceConnector.cs
          RemoteAssetServiceConnector.cs
        Grid/
          ...
        Interregion/
          ...
        Inventory/
          ...
        Messaging/
          ...
        User/ 
          ...

    Server/

      ServerMain.cs (the server shell, produces OpemSim.Services.exe)

      Base/
        (Base and basic classes for the HTTP server, one DLL)
        HttpServerBase.cs
        ServerUtils.cs
        ServicesServerBase.cs

      Handlers/
         (place to put all IN connectors and network service handlers)
         Asset/
           AssetServerConnector.cs
           AssetServerDeleteHandler.cs
           AssetServerGetHandler.cs
           AssetServerPostHandler.cs
         ...
         Base/
           (Base and basic classes for IN connectors)
           ServerConnector.cs

    Services/
      (service implementations, each on its own dll)

       AssetService/

       Base/

       GridService/

       InventoryService/

       MessagingService/

       UserService/

       (meat of the OUT connectors, one DLL)
       Connectors/
         ...

       (service interfaces, one DLL)
       Interfaces/
         ...

Service Interfaces

OpenSim.Services.Interfaces: This DLL contains all the service interfaces known to OpenSimulator.

Here is how IAssetService looks like:

public interface IAssetService
    {
        // Three different ways to retrieve an asset
        //
        AssetBase Get(string id);
        AssetMetadata GetMetadata(string id);
        byte[] GetData(string id);
 
        bool Get(string id, Object sender, AssetRetrieved handler);
 
        // Creates a new asset
        // Returns a random ID if none is passed into it
        //
        string Store(AssetBase asset);
 
        // Attachments and bare scripts need this!!
        //
        bool UpdateContent(string id, byte[] data);
 
        // Kill an asset
        //
        bool Delete(string id);
    }

Connectors

The direction of a connector is determined by the direction of the connection being made, not by the direction of the data flow. Therefore, an OUT connector is always a client, while an IN connector is always a server. Conversely, an OUT connector will provide an interface, while an IN connector will consume an interface. Interfaces are consistent across the entire chain of connectors, therefore IN connectors can be directly connected to OUT connectors, which would create a proxy server.

Out Connectors

OpenSim.Region.CoreModules.ServiceConnectors: this is where all concrete OUT connectors used by the simulator hang out.

OpenSim.Services.Connectors: this is where all the base classes of the OUT connectors hang out. These ones can be reused by servers other than the simulator.

An OUT connector primarily exposes its interfaces, it may have a secondary interface to be able to load as a region module. Here is a snippet of the RemoteAssetServiceConnector, which, as the name says, connects to a remote asset server:

public class RemoteAssetServicesConnector :
            AssetServicesConnector, ISharedRegionModule, IAssetService
    {
 
        private bool m_Enabled = false;
        private IImprovedAssetCache m_Cache;
 
        public string Name
        {
            get { return "RemoteAssetServicesConnector"; }
        }
 
        public override void Initialise(IConfigSource source)
        {
            IConfig moduleConfig = source.Configs["Modules"];
            if (moduleConfig != null)
            {
                string name = moduleConfig.GetString("AssetServices", "");
                if (name == Name)
                {
                    IConfig assetConfig = source.Configs["AssetService"];
                    if (assetConfig == null)
                    {
                        m_log.Error("[ASSET CONNECTOR]: AssetService missing from OpenSim.ini");
                        return;
                    }
                    m_Enabled = true;
                    base.Initialise(source);
                    m_log.Info("[ASSET CONNECTOR]: Remote assets enabled");
                }
            }
        }
 
        public void AddRegion(Scene scene)
        {
            if (!m_Enabled)
                return;
            scene.RegisterModuleInterface<IAssetService>(this);
        }
        ...
     }

The above code is just the region module, which is enabled or not depending on configuration variables (described further down). Two things are noteworthy:

• This module registers with Scene as the simulator's IAssetService. In other words, this modules is the direct provider of that interface within the simulator. All asset operations go through it first.
• This class extends AssetServicesConnector, which is the class that has the 'meat' of the service. A small part of it is presented below:

public class AssetServicesConnector : IAssetService
    {
        private string m_ServerURI = String.Empty;
        private IImprovedAssetCache m_Cache = null;
 
        public AssetServicesConnector(string serverURI)
        {
            m_ServerURI = serverURI;
        }
 
        public AssetServicesConnector(IConfigSource source)
        {
            Initialise(source);
        }
 
        public virtual void Initialise(IConfigSource source)
        {
            IConfig assetConfig = source.Configs["AssetService"];
            if (assetConfig == null)
            {
                m_log.Error("[ASSET CONNECTOR]: AssetService missing from OpenSim.ini");
                throw new Exception("Asset connector init error");
            }
 
            string serviceURI = assetConfig.GetString("AssetServerURI",
                    String.Empty);
 
            if (serviceURI == String.Empty)
            {
                m_log.Error("[ASSET CONNECTOR]: No Server URI named in section AssetService");
                throw new Exception("Asset connector init error");
            }
            m_ServerURI = serviceURI;
        }
 
        public AssetBase Get(string id)
        {
            string uri = m_ServerURI + "/assets/" + id;
 
            AssetBase asset = null;
            if (m_Cache != null)
                asset = m_Cache.Get(id);
 
            if (asset == null)
            {
                asset = SynchronousRestObjectRequester.
                        MakeRequest<int, AssetBase>("GET", uri, 0);
 
                if (m_Cache != null)
                    m_Cache.Cache(asset);
            }
            return asset;
        }
        ...
    }

The class above is the one that actually holds the reference to the remote server, as an URL, and that performs the remote calls to it.

Besides this connector, there are currently 2 other ones that can be used: LocalAssetServiceConnector and HGAssetBroker. The former is meant to be used when the service runs within the simulator's process; the latter is meant to be used for Hypergrided simulators.

In Connectors

OpenSim.Server.Handlers: this DLL is where all IN connectors hang out.

IN connectors listen for requests and decode them for processing. They are typically bound to a HTTP server, although other transports are possible. An IN connector will acquire an instance of a class exposing the target interface and send all received and decoded data to that class, which may be a consumer (database, etc.), a forwarder/broker (HGBroker) or an OUT connector, which would create a proxy server. IN connectors can share the same HTTP server, so all services can be contained in a single server process, or split among several processes or hosts. The proxy configuration is of particular interest, since it allows load balancing and write request routing. For instance, in SQL clusters it is possible to route write requests to different hosts than read requests.

Here is how the IN connector for an asset server looks like:

public class AssetServiceConnector : ServiceConnector
    {
        private IAssetService m_AssetService;
 
        public AssetServiceConnector(IConfigSource config, IHttpServer server) :
                base(config, server)
        {
            IConfig serverConfig = config.Configs["AssetService"];
            if (serverConfig == null)
                throw new Exception("No section 'Server' in config file");
 
            string assetService = serverConfig.GetString("LocalServiceModule",
                    String.Empty);
 
            if (assetService == String.Empty)
                throw new Exception("No AssetService in config file");
 
            Object[] args = { config };
            m_AssetService =
                    ServerUtils.LoadPlugin<IAssetService>(assetService, args);
 
            server.AddStreamHandler(new AssetServerGetHandler(m_AssetService));
            server.AddStreamHandler(new AssetServerPostHandler(m_AssetService));
            server.AddStreamHandler(new AssetServerDeleteHandler(m_AssetService));
        }
    }

Two noteworthy things about this connector:

• It establishes the service handlers on the server
• It loads a DLL containing the service implementation (finally!).

Service Implementations

OpenSim.Services.AssetService: the default OpenSimulator implementation of the asset service. Here is a snippet:

public class AssetService : AssetServiceBase, IAssetService
    {
        public AssetService(IConfigSource config) : base(config)
        {
            if (m_AssetLoader != null)
            {
                IConfig assetConfig = config.Configs["AssetService"];
                if (assetConfig == null)
                    throw new Exception("No AssetService configuration");
 
                string loaderArgs = assetConfig.GetString("AssetLoaderArgs",
                        String.Empty);
 
                m_log.InfoFormat("[ASSET]: Loading default asset set from {0}", loaderArgs);
                m_AssetLoader.ForEachDefaultXmlAsset(loaderArgs,
                        delegate(AssetBase a)
                        {
                            Store(a);
                        });
 
                m_log.Info("[ASSET CONNECTOR]: Local asset service enabled");
            }
        }
 
        public AssetBase Get(string id)
        {
            m_log.DebugFormat("[ASSET SERVICE]: Get asset {0}", id);
            UUID assetID;
 
            if (!UUID.TryParse(id, out assetID))
                return null;
 
            return m_Database.FetchAsset(assetID);
        }
        ...
     }

The above class is loaded by the IN connector shown before.

Server Shells

OpenSim.Services.exe

There is now a generic server shell that can load any IN service connectors, along with their specified service implementations. This server shell is configured with a .ini file, using exactly the same configuration variables that pertain to the asset service configuration in OpenSin.ini.

Here is the new server shell:

public class OpenSimServer
    {
        protected static HttpServerBase m_Server = null;
 
        protected static List<IServiceConnector> m_ServiceConnectors =
                new List<IServiceConnector>();
 
        static int Main(string[] args)
        {
            m_Server = new HttpServerBase("Asset", args);
 
            IConfig serverConfig = m_Server.Config.Configs["Startup"];
            if (serverConfig == null)
            {
                System.Console.WriteLine("Startup config section missing in .ini file");
                throw new Exception("Configuration error");
            }
 
            string connList = serverConfig.GetString("ServiceConnectors", String.Empty);
            string[] conns = connList.Split(new char[] {',', ' '});
 
            foreach (string conn in conns)
            {
                if (conn == String.Empty)
                    continue;
 
                string[] parts = conn.Split(new char[] {':'});
                string friendlyName = parts[0];
                if (parts.Length > 1)
                    friendlyName = parts[1];
 
                m_log.InfoFormat("[SERVER]: Loading {0}", friendlyName);
 
                Object[] modargs = { m_Server.Config, m_Server.HttpServer };
                IServiceConnector connector =
                        ServerUtils.LoadPlugin<IServiceConnector>(conn,
                        modargs);
 
                if (connector != null)
                {
                    m_ServiceConnectors.Add(connector);
                    m_log.InfoFormat("[SERVER]: {0} loaded successfully", friendlyName);
                }
                else
                {
                    m_log.InfoFormat("[SERVER]: Failed to load {0}", conn);
                }
            }
            return m_Server.Run();
        }
    }

Essentially, this generic server loads all DLLs for the specificed ServiceConnectors configuration variable. Therefore it's trivial to combine several services in a single server shell, or to have several server shells each one running a service.

Configuration

This architecture splits all services and service connectors into individual region modules and DLLs that are then specified in the server's configuration files. Each server -- OpenSim.exe and all OpenSim.Services.exe -- has its own .ini file. There are two ways of configuring the servers, described next.

Direct Configuration

The familiar .ini file has now several new configuration variable for every service connector. See Services and Service Connectors Configuration for the extra variables pertaining to configuring the asset service in OpenSim.ini.

As for configuring a resource server, here is an example of an asset server in the new style of server shells (these variables are stored in OpenSim.Services.ini)

[Startup]
; These are also available as command line options

; console = "local" ; Use "basic" to use this on a pipe
; inifile = "OpenSim.Servers.AssetServer.ini"
; logfile = "AssetServer.log" ; Also read from application config file

; Connectors, comma separated
ServiceConnectors = "OpenSim.Server.Handlers.dll:AssetServiceConnector"

[Network]
port = 8003

[AssetService]
LocalServiceModule = "OpenSim.Services.AssetService.dll:AssetService"
StorageProvider = "OpenSim.Data.MySQL.dll"
ConnectionString = "Data Source=localhost;Database=opensim;User ID=opensim;Password=opensim;"
DefaultAssetLoader = "OpenSim.Framework.AssetLoader.Filesystem.dll"
AssetLoaderArgs = "assets/AssetSets.xml"

Prepackaged System Architectures

While the split in small design elements provides all the flexibility one can get from the framework, the number of configuration variables increases considerably, and can quickly become overwhelming. To address this problem we are working on configuration "includes", which will allow us to provide pre-packaged system architectures, having all the new variables properly set up so that they can produce architectures such as the ones in the picture above.

There will be three prepackaged system architectures: StandaloneGrid, StandardGrid and HyperGrid. Users of the framework can define additional ones.

See Also

• What are UGAIM Services?
• Services and Service Connectors Main Page
• Services and Service Connectors Configuration Page

Change Log

• Diva, 5/22/09, added suggestions from Justincc @ -dev.