Developer Documentation

This documentation gives an overview of the distributed asset server inner workings. The asset server uses ExtensionLoader, which implements the Inversion of Control design methodology. The result is a series of distinct components referred to in the documentation as interfaces or extensions. Each interface exposes a series of functions and that may be implemented in very different ways by different extensions. Generally, each interface is assigned a single extension through a configuration file*. Most functions return a StorageResponse value, a simple enumeration of the possible success or failure values of a request. Many asset functions use the common Metadata class which is explained below, and inventory functions use the common InventoryFolder and InventoryItem classes.

• An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching.

Code Diagram

The following diagram shows the interaction between the different interfaces, frontends, and external resources.

Metadata

All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:

public class Metadata
{
    public UUID ID;
    public string Name;
    public string Description;
    public DateTime CreationDate;
    public string ContentType;
    public byte[] SHA1;
    public bool Temporary;
    public Dictionary<string, Uri> Methods;
    public OSDMap ExtraData;
}

Content-Type

Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:

public static string SLAssetTypeToExtraData(int assetType)
{
    switch (assetType)
    {
        case  0: return "image/jp2";
        case  1: return "application/ogg";
        case  2: return "application/x-metaverse-callingcard";
        case  3: return "application/x-metaverse-landmark";
        case  5: return "application/x-metaverse-clothing";
        case  6: return "application/x-metaverse-primitive";
        case  7: return "application/x-metaverse-notecard";
        case 10: return "application/x-metaverse-lsl";
        case 11: return "application/x-metaverse-lso";
        case 12: return "image/tga";
        case 13: return "application/x-metaverse-bodypart";
        case 17: return "audio/x-wav";
        case 19: return "image/jpeg";
        case 20: return "application/x-metaverse-animation";
        case 21: return "application/x-metaverse-gesture";
        case 22: return "application/x-metaverse-simstate";
        default: return "application/octet-stream";
    }
}

Methods

A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.

ExtraData

The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.

Storage

Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.

public interface IStorageProvider
{
    StorageResponse TryFetchMetadata(UUID assetID, UUID authToken, out Metadata metadata);
    StorageResponse TryFetchData(UUID assetID, UUID authToken, out byte[] assetData);
    StorageResponse TryFetchDataMetadata(UUID assetID, UUID authToken, out Metadata metadata, out byte[] assetData);
    StorageResponse TryCreateAsset(Metadata metadata, byte[] assetData, UUID authToken, out UUID assetID);
    StorageResponse TryCreateAsset(Metadata metadata, byte[] assetData, UUID authToken);
    int ForEach(Action<Metadata> action, int start, int count, UUID authToken);
}

The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.

Inventory

Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.

public interface IInventoryProvider
{
    StorageResponse TryFetchItem(Uri owner, UUID itemID, UUID authToken, out InventoryItem item);
    StorageResponse TryFetchFolder(Uri owner, UUID folderID, UUID authToken, out InventoryFolder folder);
    StorageResponse TryFetchFolderContents(Uri owner, UUID folderID, UUID authToken, out InventoryCollection contents);
    StorageResponse TryFetchFolderList(Uri owner, UUID authToken, out List<InventoryFolder> folders);
    StorageResponse TryFetchInventory(Uri owner, UUID authToken, out InventoryCollection inventory);

    StorageResponse TryFetchActiveGestures(Uri owner, UUID authToken, out List<InventoryItem> gestures);

    StorageResponse TryCreateItem(Uri owner, InventoryItem item, UUID authToken);
    StorageResponse TryCreateFolder(Uri owner, InventoryFolder folder, UUID authToken);
    StorageResponse TryCreateInventory(Uri owner, InventoryFolder rootFolder, UUID authToken);

    StorageResponse TryDeleteItem(Uri owner, UUID itemID, UUID authToken);
    StorageResponse TryDeleteFolder(Uri owner, UUID folderID, UUID authToken);
    StorageResponse TryPurgeFolder(Uri owner, UUID folderID, UUID authToken);
}

The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.

Frontends

Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.

Authentication

Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.

public interface IAuthenticationProvider
{
    void AddIdentifier(UUID authToken, Uri identifier);
    bool RemoveIdentifier(UUID authToken);
    bool TryGetIdentifier(UUID authToken, out Uri identifier);
}

Authentication providers should expire UUID to URI mappings after some amount of inactivity.

Authorization

Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.

public interface IAuthorizationProvider
{
    bool IsMetadataAuthorized(UUID authToken, UUID assetID);
    bool IsDataAuthorized(UUID authToken, UUID assetID);
    bool IsCreateAuthorized(UUID authToken);

    bool IsInventoryReadAuthorized(UUID authToken, Uri owner);
    bool IsInventoryWriteAuthorized(UUID authToken, Uri owner);
}