REST

Revision as of 20:31, 9 October 2008

The following is taken from Alan M Webb's OpenSim REST Support Draft document dated 8/18/08. The information here and specification are works in progress but provide the basis for useful OpenSim integrations including content input and retrieval without direct DB access. --MoHax 00:10, 10 October 2008 (UTC)

There are a number of REST interfaces1 implemented within OpenSim. This document describes the work done to support specific interactions required in support of external application integration. XML is used to encode the payloads associated with these requests. The specific XML used is documented as an appendix, this is an interim specification.

What is REST?

The notion of Representational State Transfer (REST) was introduced by Roy Fielding in his doctoral dissertation in 2000. It is essentially an architectural concept rather than a protocol, and this has led to a variety of implementations, all of which might claim to be compliant REST implementations.

A REST interface is built upon the HTTP protocol, making use of the four principle methods defined therein: GET, PUT, POST, and DELETE. A fifth method HEAD is also supported, but that is really just a semantic variation of GET. The TRACE and OPTIONS methods have a less fundamental role. No other protocol is involved.

Note that REST does not specify the nature of the payload either sent or received. The interface associated with this document uses XML meta-data to manage an arbitrary payload. The need for a schema document to give form to this XML will be dealt with as a separate discussion.

Of particular significance is the understood meaning of the methods themselves, which is (and must be) as defined by the HTTP specification:

• GET retrieves the information identified by the associated URI and returns it to the requester. GET is idempotent. If the URI identifies a process, then it is the output of that process which is returned, and not the source text of that process. GET is conditional when the headers include any of the If headers. GET is partial if a range related header is included. The results of GET may be able to be cached.

• HEAD is functionally equivalent to GET except that the payload is not included in the response. The result of a HEAD request may be able to be cached.

• POST requests that the server accept the payload as a new entity, subordinate to the existing entity identified by the URI. POST places requirements upon session capabilities in the server insofar as it requires persistent connection and flow control capabilities.

• PUT indicates that the payload should be stored on the server under1 the resource named by the URI. If the named resource already exists, then the payload is considered to be a modified version of that resource. If the named resource does not exist, then it should be created by the server.

• DELETE requests that the server delete the resource identified by the URI.

Other methods (TRACE, OPTIONS, CONNECT) may be supported, but are not fundamental to the principles of REST. Use of these methods also implies specific requirements in terms of how the response codes should be used.

REST Inventory Support

The REST support in OpenSim is separated into a number of functional components, all of which live in the OpenSim/ApplicationPlugins/Rest hierarchy.

1. Plug-in management. E.g. RestPlugin.cs and Inventory/RestHandler.cs
2. Service management: Inventory/RestHandler.cs
3. REST implementation: RequestData.cs, Rest.cs
4. Service implementations: Inventory/RestInventoryServices.cs, Inventory/RestAssetServices.cs, etc.

This separation serves a number of purposes:

• it limits the sensitivity of each component to implementation changes in the others
• it simplifies the task of service implementation by “hiding” infrastructure
• it helps make services consistent by sharing a single implementation of REST related tasks

The design intention is that each subdirectory in the REST hierarchy, I.e. Inventory, corresponds to a distinct REST handler assembly consisting of service management and one or more service implementations. The Inventory directory is one example of such an assembly.

Configuration Parameters

In order for the REST handlers to be loaded and effective, statements must be added to the OpenSim.ini file. OpenSim reads the initialization file during start-up. This file is partitioned into various functional domains, e.g.

[Startup]
gridmode = false
physics = basicphysics

[RestPlugins]
enabled = false

[RestHandler]
enabled = false

The RestPlugins partition affects REST handlers in general and supports the following entries:

[RestPlugins]
enabled = true|false || false
god_key = <string> || “”
prefix  = <string>   || “/admin”

Only enabled=true is necessary to enable REST processing.

Additionally, REST inventory processing currently requires that the following partition be defined:

[RestHandler]
enabled = true|false || false
authenticate = true|false || true
extended-escape = true|false || true
realm = <string> || “OpenSim REST”
dump-asset = true|false || false
dump-line-size = int%8 || 32
path-fill = true|false || false

The partition name corresponds to the ConfigName value specified in the plug-in subclass, I.e. RestHandler.cs.

Again, only the enabled=true entry must be provided to enable the inventory handler; all other defaults are acceptable for normal operation. The meanings of these values will become apparent in the rest of the document.

When the server starts, the REST handler will publish several messages indicating which configuration values are in effect.

Effective URI

From a REST perspective, the path component of the URI is viewed as a resource identification, the precise significance of which depends upon the HTTP method being used. There are subtleties here, in some cases a prefix of the path may be viewed as a functional qualification of the authority, with the remainder of the URI being a resource identification within the domain of that authority. In this document, the notion of a service corresponds to the functional qualification mechanism just described.

The prefix value in the configuration file is used to construct the server's base prefix for REST services, as illustrated in the description of the inventory services prefix shown below:

http://<host>:<port>/admin/inventory
                     ^^^^^

In this URI /admin is the base prefix, and /inventory is a prefix extension that identifies the inventory service handler.

All requests that have an /admin/inventory prefix will be passed to the inventory handler. The prefix must be followed by the name of the avatar that owns the inventory in question. For example:

http://localhost:9000/admin/inventory/Alan Webb

Assuming the GET method, then this will return Alan Webb's inventory catalog, as XML data in the response payload, for example:

<Inventory>
<Folder name = “My Inventory” uuid=”11111111-...-111111111111” ....
        <Folder name=”Objects” uuid=....
        ....
        </Folder>
    </Folder>
</Inventory>

In this case /admin/inventory identiifes the service, and /Alan Webb is the resource specification component of the path.

Implementation Overview

The REST implementation being described is manifest in a subset of the files contained in the ApplicationPlugins/Rest file hierarchy. It is logically partitioned into three domains:

• Plugin and configuration management
• REST implementation services
• REST service providers

The objective here being to separate functionality so that the task of the service provider is made as straightforward as possible while ensuring that all services implemented using the interface are consistent in both approach and behavior.

Illustration 1: Overall Data Flow Model

As shown in Illustration 1: Overall Data Flow Model all requests originate, and are eventually returned to, the OpenSim HTTP server. The handler is isolated from this implementation by the IHttpAgentHandler interface and the OSHTTP data-types. After matching, the request is passed to the appropriate handler; this relationship is defined by the IRest interface and the RequestData data-type.

Changes to BaseHttpServer

The existing OpenSim HTTP server class was modified to support plug-in modules that conform to the IHttpAgentHandler interface. Three new subroutines were added: AddHttpAgentHandler, RemoveHttpAgentHandler, and TryHttpAgentHandler; this is consistent with the way in which existing handlers are managed.

Additionally, the HandleRequest method was extended to include support for the IHttpAgentHandler interface.

The processing of existing handlers is not affected by the new support.

Plugin and Configuration Management

IHttpAgentHandler describes the interface between OpenSim's base HTTP server and the REST implementation, a plug-in must implement this interface. Any class that exports this interface, and that registers its handlers using the appropriate (not defined by an interface1) methods in the base HTTP server can act as a handler. Note that while the usage described here is intended to support REST, the interface itself is agnostic and may be used as the basis for any HTTP request handler.

RestPlugin.cs provides a basic agent-handling superclass. An implementation of this class represents a handler from the point of view of the base HTTP server. This class implements the IHttpAgentHandler interface. A given implementation class need only be differentiated from the others in terms of the agent name it uses when it registers. At present, a duplicate agent (based upon supplied agent name) will not be registered; how such a failure is handled depends upon the implementation class. Aside from the name used, a given handler should be differentiated in terms of the set of request patterns that it will match; overlap is undesirable as the first match will win, and at present the order in which OpenSim loads modules is non-deterministic. The RestHandler class in the Inventory sub-directory is an example of an implementation class for RestPlugin.cs.

Inventory/RestHandler.cs implements the plug-in interface described above, and also implements a local model of integrating a set of services within a single handler instance. It does this by scanning the classes included in the assembly for those that implement the IRest interface; each such class is then initialized. This provides implementation independence from a development standpoint (adding a new service is as simple as adding a single class) whilst avoiding unnecessary overt modularization in OpenSim. This class also takes care of global configuration and initialization of the basic REST environment, i.e. It sets basic references for the OpenSim communications manager, user services, inventory services, and asset management.

For performance reasons, and as an obligation to the IHttpAgentHandler definition, this class also implements the Match function, using pattern information registered by the individual services when they were initialized1.

This class contains no REST specific, or service specific functionality beyond some basic initialization. This class can be used in multiple assemblies without change.

Rest.cs provides both state and functionality in support of the REST implementation. Most of the state is static and constant, a small set of static values are initialized by the RestHandler when the plug-in is loaded. This class will only change as the REST implementation evolves and becomes more complete. It must not contain any service sensitive elements. This class may be sub-classed by a given assembly to provide global additions required by a specific set of services.

RequestData.cs serves as a carrier for the state associated with a request. It may be sub-classed by a given service provider in order to add service specific state and functionality, for example the inventory services currently utilize an XML extension to handle inventory specific payload interpretation and generation.

The remaining classes in the Inventory directory provide different classes of service.

Adding A New REST Service To An Existing Handler

The process for adding a new service is straightforward. The file RestTestServices.cs serves as an example1 of what a new services class must minimally include. Key points to note are:

1. Each service class should have an associated value for qPrefix. This should be unique for each service, as it represents the relative extension of the URI that will be matched by this service. The value supplied here may be absolute, but this is discouraged. The constructor will automatically update a relative value to reflect the absolute prefix set globally during initialization.
2. In the case of test services the constructor loads a set of classes from a sub-directory, each of which represents a specific service implementation. For example, these may be method-related, or based upon other values contained in the supplied URI.
3. The services constructor must register itself with the RestHandler. It provides three values at registration time: a reference to the service method, the URI prefix that this service will match, and a reference to an RequestData allocation method that will return an appropriately sub-classed instance of the RequestData class.
4. Implementation of the service and allocation methods described above must be provided.

Other methods definitions are simply part of the overall infrastructure and would be unique to a given service implementation.

Note that a given services class may implement zero or more services, according to its intended function. A class providing shared functionality to other services may be loaded but not register any actual handling methods. Similarly, a single services class could register multiple handling methods.

REST Implementation Classes

This section provides additional documentation of the state and services provided by Rest.cs and RequestData.cs files.

Rest.cs

The eponymous class provides a collection of request independent services and state definitions. Any functions directly dependent upon the HTTP request and response elements are provided in RequestData.cs.

State

The Log declaration provides local access to the log4net facility. Though apparently complex it simply resolves to a static, read-only reference that can be used to pass messages to the logging facility.

The value of DEBUG reflects whether or not debug messages have been enabled in the OpenSim log. If they have, then DEBUG will be enabled, if not, then they will not. Being a static declaration it is anticipated that the JIT compiler will be able to make efficient code generation assumptions. This is only of real significance where some substantial operation is performed solely in support of debugging.

The Plugin reference is set by RestHandler during initialization. It eliminates the need for a services class to have direct awareness of how it was activated. For example, this reference is used to locate the AddPathHandler method needed to register its interface.

The main reference is set by the RestHandler during initialization. Its value is a reference to the simulator's base class. It is provided to the plug-in interface and is stored for that reason. It is not currently used except by RestHandler to locate the communications manager during initialization.

Comms provides a globally accessible reference to the simulator's communications manager. It is initialized by RestHandler during initialization and is currently only used by RestHandler to locate other, commonly needed, simulator services.

InventoryServices, UserServices, and AssetServices are references to the corresponding simulator service interfaces. This list may grow as common service requirements expand.

Prefix is the common URI prefix associated with all handlers within a given assembly. In the case of inventory services it is obtained from the global REST configuration. It is currently /admin. This is a configurable attribute. Typical this should be set to whatever is considered to be the standard prefix for REST services in a given URI, it will be combined with scheme and authority information to construct a valid URL.

Config is a saved reference to the configuration file used by this assembly. The configuration partition is identified by the ConfigName string supplied by the RestHandler implementation. This is currently only used by the RestHandler, but it could be used by a service implementation to access attributes unique to the service.

GodKey is a string obtained from the configuration file which contains the password for the God user. This user has unfettered access to all administrative functions. This is a temporary mechanism and will eventually be replaced by something more secure. This is a configurable attribute.

Authenticate indicates whether or not user authentication is required by this handler. This is a configurable attribute, true by default.

Secure indicates whether or not the HTTP session should be secured. Secure sessions are currently not available to OpenSim. This is a configurable attribute, true by default.

ExtendedEscape controls whether or not the URI is processed for parsing extensions such as “+” in place of spaces. This is a configurable attribute, true by default.

DumpAsset controls whether or not the content of an incoming asset should be dumped to the trace file after decoding has completed. This is only intended to be used when there is some indication of a problem with the decoding process. It may produce a significant amount of trace output depending upon the quantity of asset data being uploaded. This is configurable and is normally false.

Fill controls whether or not the server will automatically generate intermediate folder names when handling a POST request; it is not currently considered appropriate for PUT. This is not presently implemented.

FlushEnabled indicates that the server will completely flush an inbound data stream before responding. This accommodates clients that can not handle the case where a socket error is generated as a result of an early termination of the inbound data stream (due to an authentication failure for example). This mechanism is vulnerable to DOS attacks and can be selectively enabled/disabled.

Realm is a string that may be used to distinguish the authentication realm associated with the handler, in place of the URI prefix. This corresponds directly to the realm token included in the WWW-Authenticate response header. This is a configurable option, and is set to 'OpenSim REST' by default. Clients may use this value to determine if additional authentication is necessary, or to select appropriate cached credentials.

DumpLineSize indicates the line length to use for any data dumps (e.g. DumpAsset) during runtime. The default is 132. This is a configurable option.

statusHead contains the standard HTML header used for reporting request status. Similarly, statusTail is used to provide a standard suffix. These are both used by the standard status reporting mechanisms.

HttpStatusDesc is an array which, when indexed by an appropriate HTTP status code, will return a corresponding status description.

CreationDate (property) provides a single consistent reference for creation data value needed when introducing new items into the inventory.

MsgId (property) provides a consistent message prefix for log messages generated by the handler. Although static, the value is thread specific, so it is actually unique per request.

RequestId (property) provides a unique, monotonically incrementing request counter for processed requests.

Encoding provides a standard encoding reference (UTF8) for all data handled by the handler.

Version and Name are intended to uniquely identify the REST implementation represented by this particular class. Version information should be updated if any significant changes to the REST interface are implemented. This is not currently exploited, but should be. We should have an extended header that indicates the REST version that the client is expecting, and the Match process should be sensitive to this. In this way we can provide concurrent support for multiple REST versions.

The extensive list of constant values are all held to be self-explanatory and should be used wheresoever the value they represent is required in a service implementation. This ensures consistency between the REST implementation and the REST service implementations.

Functions