AssetServer/DeveloperDocs
From OpenSimulator
(→See Also: Updated "See Also" heading with links.) |
m (Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
| − | {{ | + | {{Quicklinks}} |
| − | + | ||
| − | + | ||
{{proposal}} | {{proposal}} | ||
{{content}} | {{content}} | ||
| − | =Developer Documentation= | + | = Developer Documentation = |
This documentation gives an overview of the distributed asset server inner workings. The asset server uses ExtensionLoader, which implements the [http://en.wikipedia.org/wiki/Inversion_of_control Inversion of Control] design methodology. The result is a series of distinct components referred to in the documentation as interfaces or extensions. Each interface exposes a series of functions and that may be implemented in very different ways by different extensions. Generally, each interface is assigned a single extension through a configuration file*. Most functions return a StorageResponse value, a simple enumeration of the possible success or failure values of a request. Many asset functions use the common Metadata class which is explained below, and inventory functions use the common InventoryFolder and InventoryItem classes. | This documentation gives an overview of the distributed asset server inner workings. The asset server uses ExtensionLoader, which implements the [http://en.wikipedia.org/wiki/Inversion_of_control Inversion of Control] design methodology. The result is a series of distinct components referred to in the documentation as interfaces or extensions. Each interface exposes a series of functions and that may be implemented in very different ways by different extensions. Generally, each interface is assigned a single extension through a configuration file*. Most functions return a StorageResponse value, a simple enumeration of the possible success or failure values of a request. Many asset functions use the common Metadata class which is explained below, and inventory functions use the common InventoryFolder and InventoryItem classes. | ||
| Line 12: | Line 10: | ||
* An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching. | * An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching. | ||
| − | ==Code Diagram== | + | == Code Diagram == |
The following diagram shows the interaction between the different interfaces, frontends, and external resources. | The following diagram shows the interaction between the different interfaces, frontends, and external resources. | ||
| Line 18: | Line 16: | ||
'''WARNING:''' The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes. | '''WARNING:''' The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes. | ||
| − | [[ | + | [[File:DistributedAssetServer-UML.png]] |
| − | ==Metadata== | + | == Metadata == |
All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below: | All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below: | ||
| Line 37: | Line 35: | ||
}</pre> | }</pre> | ||
| − | ===Content-Type=== | + | === Content-Type === |
Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion: | Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion: | ||
| Line 65: | Line 63: | ||
}</pre> | }</pre> | ||
| − | ===Methods=== | + | === Methods === |
A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata. | A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata. | ||
| − | ===ExtraData=== | + | === ExtraData === |
The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure. | The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure. | ||
| − | ==Storage== | + | == Storage == |
Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface. | Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface. | ||
| Line 89: | Line 87: | ||
The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure. | The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure. | ||
| − | ==Inventory== | + | == Inventory == |
Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders. | Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders. | ||
| Line 114: | Line 112: | ||
The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release. | The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release. | ||
| − | ==Frontends== | + | == Frontends == |
| − | Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an | + | Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSimulator asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSimulator inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol. |
| − | ==Authentication== | + | == Authentication == |
Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check. | Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check. | ||
| Line 131: | Line 129: | ||
Authentication providers should expire UUID to URI mappings after some amount of inactivity. | Authentication providers should expire UUID to URI mappings after some amount of inactivity. | ||
| − | ==Authorization== | + | == Authorization == |
Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways. | Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways. | ||
| Line 149: | Line 147: | ||
* [[:Category:Getting Started|Getting Started]] | * [[:Category:Getting Started|Getting Started]] | ||
* [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]] | * [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]] | ||
| − | * [[UserServer|User Server ]] | + | * [[UserServer|User Server]] |
* [[UserServer/ClientDocs|UserServer Client Documentation]] | * [[UserServer/ClientDocs|UserServer Client Documentation]] | ||
* [[UserServer/DeveloperDocs|UserServer Developer Documentation]] | * [[UserServer/DeveloperDocs|UserServer Developer Documentation]] | ||
| − | * [[GridServer|Grid Server ]] | + | * [[GridServer|Grid Server]] |
* [[GridServer/ClientDocs|GridServer Client Documentation]] | * [[GridServer/ClientDocs|GridServer Client Documentation]] | ||
* [[GridServer/DeveloperDocs|GridServer Developer Documentation]] | * [[GridServer/DeveloperDocs|GridServer Developer Documentation]] | ||
| Line 158: | Line 156: | ||
* [[AssetServer/ClientDocs|AssetServer Client Documentation]] | * [[AssetServer/ClientDocs|AssetServer Client Documentation]] | ||
* [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]] | * [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]] | ||
| − | * [[InventoryServer|Inventory Server ]] | + | * [[InventoryServer|Inventory Server]] |
* [[InventoryServer/ClientDocs|InventoryServer Client Documentation]] | * [[InventoryServer/ClientDocs|InventoryServer Client Documentation]] | ||
* [[InventoryServer/DeveloperDocs|InventoryServer Developer Documentation]] | * [[InventoryServer/DeveloperDocs|InventoryServer Developer Documentation]] | ||
| − | * [[MessagingServer|Messaging Server ]] | + | * [[MessagingServer|Messaging Server]] |
* [[MessagingServer/ClientDocs|MessagingServer Client Documentation]] | * [[MessagingServer/ClientDocs|MessagingServer Client Documentation]] | ||
* [[MessagingServer/DeveloperDocs|MessagingServer Developer Documentation]] | * [[MessagingServer/DeveloperDocs|MessagingServer Developer Documentation]] | ||
* [[:Category:Tech Reference|Technical Reference Pages]] | * [[:Category:Tech Reference|Technical Reference Pages]] | ||
| − | |||
[[Category:Support]] | [[Category:Support]] | ||
| − | [[Category:Tech Reference]] | + | [[Category:Tech Reference]] |
[[Category:Help]] | [[Category:Help]] | ||
[[Category:Configuration]] | [[Category:Configuration]] | ||
| − | [[Category: | + | [[Category:Getting Started]] |
[[Category:Development]] | [[Category:Development]] | ||
[[Category:Todo]] | [[Category:Todo]] | ||
[[Category:Proposal]] | [[Category:Proposal]] | ||
Latest revision as of 22:13, 3 March 2012
 |
Languages: |
|
|
|
This article or section is a Proposal It does not represent the current state of OpenSim, but is an idea for future work in OpenSim. Please feel free to update this page as part of the proposal discussion. |
|
|
This article or section contains incomplete information. Please help us by completing the content on this page. |
[edit] Developer Documentation
This documentation gives an overview of the distributed asset server inner workings. The asset server uses ExtensionLoader, which implements the Inversion of Control design methodology. The result is a series of distinct components referred to in the documentation as interfaces or extensions. Each interface exposes a series of functions and that may be implemented in very different ways by different extensions. Generally, each interface is assigned a single extension through a configuration file*. Most functions return a StorageResponse value, a simple enumeration of the possible success or failure values of a request. Many asset functions use the common Metadata class which is explained below, and inventory functions use the common InventoryFolder and InventoryItem classes.
- An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching.
[edit] Code Diagram
The following diagram shows the interaction between the different interfaces, frontends, and external resources.
WARNING: The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes.
[edit] Metadata
All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:
public class Metadata
{
public UUID ID;
public string Name;
public string Description;
public DateTime CreationDate;
public string ContentType;
public byte[] SHA1;
public bool Temporary;
public Dictionary<string, Uri> Methods;
public OSDMap ExtraData;
}
[edit] Content-Type
Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:
public static string SLAssetTypeToExtraData(int assetType)
{
switch (assetType)
{
case 0: return "image/jp2";
case 1: return "application/ogg";
case 2: return "application/x-metaverse-callingcard";
case 3: return "application/x-metaverse-landmark";
case 5: return "application/x-metaverse-clothing";
case 6: return "application/x-metaverse-primitive";
case 7: return "application/x-metaverse-notecard";
case 10: return "application/x-metaverse-lsl";
case 11: return "application/x-metaverse-lso";
case 12: return "image/tga";
case 13: return "application/x-metaverse-bodypart";
case 17: return "audio/x-wav";
case 19: return "image/jpeg";
case 20: return "application/x-metaverse-animation";
case 21: return "application/x-metaverse-gesture";
case 22: return "application/x-metaverse-simstate";
default: return "application/octet-stream";
}
}
[edit] Methods
A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.
[edit] ExtraData
The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.
[edit] Storage
Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.
public interface IStorageProvider
{
BackendResponse TryFetchMetadata(UUID assetID, out Metadata metadata);
BackendResponse TryFetchData(UUID assetID, out byte[] assetData);
BackendResponse TryFetchDataMetadata(UUID assetID, out Metadata metadata, out byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData, out UUID assetID);
int ForEach(Action<Metadata> action, int start, int count);
}
The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.
[edit] Inventory
Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.
public interface IInventoryProvider
{
BackendResponse TryFetchItem(Uri owner, UUID itemID, out InventoryItem item);
BackendResponse TryFetchFolder(Uri owner, UUID folderID, out InventoryFolder folder);
BackendResponse TryFetchFolderContents(Uri owner, UUID folderID, out InventoryCollection contents);
BackendResponse TryFetchFolderList(Uri owner, out List<InventoryFolder> folders);
BackendResponse TryFetchInventory(Uri owner, out InventoryCollection inventory);
BackendResponse TryFetchActiveGestures(Uri owner, out List<InventoryItem> gestures);
BackendResponse TryCreateItem(Uri owner, InventoryItem item);
BackendResponse TryCreateFolder(Uri owner, InventoryFolder folder);
BackendResponse TryCreateInventory(Uri owner, InventoryFolder rootFolder);
BackendResponse TryDeleteItem(Uri owner, UUID itemID);
BackendResponse TryDeleteFolder(Uri owner, UUID folderID);
BackendResponse TryPurgeFolder(Uri owner, UUID folderID);
}
The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.
[edit] Frontends
Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSimulator asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSimulator inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
[edit] Authentication
Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.
public interface IAuthenticationProvider
{
void AddIdentifier(UUID authToken, Uri identifier);
bool RemoveIdentifier(UUID authToken);
bool TryGetIdentifier(UUID authToken, out Uri identifier);
}
Authentication providers should expire UUID to URI mappings after some amount of inactivity.
[edit] Authorization
Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.
public interface IAuthorizationProvider
{
bool IsMetadataAuthorized(UUID authToken, UUID assetID);
bool IsDataAuthorized(UUID authToken, UUID assetID);
bool IsCreateAuthorized(UUID authToken);
bool IsInventoryReadAuthorized(UUID authToken, Uri owner);
bool IsInventoryWriteAuthorized(UUID authToken, Uri owner);
}
[edit] See Also
- Getting Started
- What are UGAIM Services?
- User Server
- UserServer Client Documentation
- UserServer Developer Documentation
- Grid Server
- GridServer Client Documentation
- GridServer Developer Documentation
- Asset Server
- AssetServer Client Documentation
- AssetServer Developer Documentation
- Inventory Server
- InventoryServer Client Documentation
- InventoryServer Developer Documentation
- Messaging Server
- MessagingServer Client Documentation
- MessagingServer Developer Documentation
- Technical Reference Pages
