GridServer/DeveloperDocs

From OpenSimulator

(Difference between revisions)
Jump to: navigation, search
(See Also: Fixed "Technical Reference" pages link, and updated description text. - Please ignore this page, it's a work in progress. Thank-you!)
m (Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
{{Template:Quicklinks}}
+
{{Quicklinks}}
 
+
[[GridServer/DeveloperDocs|GridServer Developer Documentation]]
[[Technical Reference | Technical Reference]] -> [[Technical Reference/terms | Terms]] -> [[GridServer | GridServer]] -> [[GridServer/DeveloperDocs | GridServer Developer Documentation]]
+
 
{{proposal}}
 
{{proposal}}
 
{{content}}
 
{{content}}
  
=Developer Documentation=
+
= Developer Documentation =
  
This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah. Enter some text here.
+
This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah. Enter some text here.
  
 
* Put some text here, blah, blah, blah.
 
* Put some text here, blah, blah, blah.
Line 19: Line 18:
  
 
----------------------------------------------------------------------------
 
----------------------------------------------------------------------------
****************   MODIFY ALL THE TEXT BELOW THIS AREA ********************
+
**************** MODIFY ALL THE TEXT BELOW THIS AREA ********************
 
----------------------------------------------------------------------------
 
----------------------------------------------------------------------------
==Code Diagram==
+
== Code Diagram ==
  
 
The following diagram shows the interaction between the different interfaces, frontends, and external resources.
 
The following diagram shows the interaction between the different interfaces, frontends, and external resources.
Line 29: Line 28:
 
[[Image:DistributedAssetServer-UML.png]]
 
[[Image:DistributedAssetServer-UML.png]]
  
==Metadata==
+
== Metadata ==
  
 
All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:
 
All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:
Line 46: Line 45:
 
}</pre>
 
}</pre>
  
===Content-Type===
+
=== Content-Type ===
  
 
Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:
 
Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:
Line 74: Line 73:
 
}</pre>
 
}</pre>
  
===Methods===
+
=== Methods ===
  
 
A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.
 
A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.
  
===ExtraData===
+
=== ExtraData ===
  
 
The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.
 
The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.
  
==Storage==
+
== Storage ==
  
 
Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.
 
Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.
Line 98: Line 97:
 
The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.
 
The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.
  
==Inventory==
+
== Inventory ==
  
 
Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.
 
Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.
Line 123: Line 122:
 
The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.
 
The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.
  
==Frontends==
+
== Frontends ==
  
Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
+
Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSimulator asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSimulator inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
  
==Authentication==
+
== Authentication ==
  
 
Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.
 
Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.
Line 140: Line 139:
 
Authentication providers should expire UUID to URI mappings after some amount of inactivity.
 
Authentication providers should expire UUID to URI mappings after some amount of inactivity.
  
==Authorization==
+
== Authorization ==
  
 
Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.
 
Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.
Line 156: Line 155:
  
 
----------------------------------------------------------------------------
 
----------------------------------------------------------------------------
****************   MODIFY ALL THE TEXT ABOVE THIS AREA ********************
+
**************** MODIFY ALL THE TEXT ABOVE THIS AREA ********************
 
----------------------------------------------------------------------------
 
----------------------------------------------------------------------------
  
 
= See Also =
 
= See Also =
  
 +
* [[:Category:Getting Started|Getting Started]]
 +
* [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]]
 +
* [[UserServer|User Server]]
 +
* [[UserServer/ClientDocs|UserServer Client Documentation]]
 +
* [[UserServer/DeveloperDocs|UserServer Developer Documentation]]
 +
* [[GridServer|Grid Server]]
 +
* [[GridServer/ClientDocs|GridServer Client Documentation]]
 +
* [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
 
* [[AssetServer|Asset Server]]
 
* [[AssetServer|Asset Server]]
 
* [[AssetServer/ClientDocs|AssetServer Client Documentation]]
 
* [[AssetServer/ClientDocs|AssetServer Client Documentation]]
 
* [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]]
 
* [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]]
* [[GridServer|Grid Server ]]
+
* [[InventoryServer|Inventory Server]]  
* [[GridServer/ClientDocs|GridServer Client Documentation]]
+
* [[InventoryServer/ClientDocs|InventoryServer Client Documentation]]
* [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
+
* [[InventoryServer/DeveloperDocs|InventoryServer Developer Documentation]]
 +
* [[MessagingServer|Messaging Server]]
 +
* [[MessagingServer/ClientDocs|MessagingServer Client Documentation]]
 +
* [[MessagingServer/DeveloperDocs|MessagingServer Developer Documentation]]
 +
 
 
* [[:Category:Tech Reference|Technical Reference Pages]]
 
* [[:Category:Tech Reference|Technical Reference Pages]]
  
 
[[Category:Support]]
 
[[Category:Support]]
[[Category:Tech Reference]]  
+
[[Category:Tech Reference]]
 
[[Category:Help]]
 
[[Category:Help]]
 
[[Category:Configuration]]
 
[[Category:Configuration]]
[[Category:Getting_Started]]
+
[[Category:Getting Started]]
 
[[Category:Development]]
 
[[Category:Development]]
 
[[Category:Todo]]
 
[[Category:Todo]]
 
[[Category:Proposal]]
 
[[Category:Proposal]]

Latest revision as of 22:33, 3 March 2012

GridServer Developer Documentation


[edit] Developer Documentation

This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah. Enter some text here.

  • Put some text here, blah, blah, blah.
<Edit the previous text to make sense, this is just placeholder text, no real importance or real value here>

Begin adding meaningful content and editing here...

<edit all the text below this mark (this is leftover text from the AssetServer/ClientDocs page), please change/adapt and format the context of this page for use for the Grid Server/ClientDocs page>


                                • MODIFY ALL THE TEXT BELOW THIS AREA ********************

[edit] Code Diagram

The following diagram shows the interaction between the different interfaces, frontends, and external resources.

WARNING: The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes.

DistributedAssetServer-UML.png

[edit] Metadata

All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:

public class Metadata
{
    public UUID ID;
    public string Name;
    public string Description;
    public DateTime CreationDate;
    public string ContentType;
    public byte[] SHA1;
    public bool Temporary;
    public Dictionary<string, Uri> Methods;
    public OSDMap ExtraData;
}

[edit] Content-Type

Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:

public static string SLAssetTypeToExtraData(int assetType)
{
    switch (assetType)
    {
        case  0: return "image/jp2";
        case  1: return "application/ogg";
        case  2: return "application/x-metaverse-callingcard";
        case  3: return "application/x-metaverse-landmark";
        case  5: return "application/x-metaverse-clothing";
        case  6: return "application/x-metaverse-primitive";
        case  7: return "application/x-metaverse-notecard";
        case 10: return "application/x-metaverse-lsl";
        case 11: return "application/x-metaverse-lso";
        case 12: return "image/tga";
        case 13: return "application/x-metaverse-bodypart";
        case 17: return "audio/x-wav";
        case 19: return "image/jpeg";
        case 20: return "application/x-metaverse-animation";
        case 21: return "application/x-metaverse-gesture";
        case 22: return "application/x-metaverse-simstate";
        default: return "application/octet-stream";
    }
}

[edit] Methods

A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.

[edit] ExtraData

The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.

[edit] Storage

Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.

public interface IStorageProvider
{
    BackendResponse TryFetchMetadata(UUID assetID, out Metadata metadata);
    BackendResponse TryFetchData(UUID assetID, out byte[] assetData);
    BackendResponse TryFetchDataMetadata(UUID assetID, out Metadata metadata, out byte[] assetData);
    BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData);
    BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData, out UUID assetID);
    int ForEach(Action<Metadata> action, int start, int count);
}

The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.

[edit] Inventory

Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.

public interface IInventoryProvider
{
    BackendResponse TryFetchItem(Uri owner, UUID itemID, out InventoryItem item);
    BackendResponse TryFetchFolder(Uri owner, UUID folderID, out InventoryFolder folder);
    BackendResponse TryFetchFolderContents(Uri owner, UUID folderID, out InventoryCollection contents);
    BackendResponse TryFetchFolderList(Uri owner, out List<InventoryFolder> folders);
    BackendResponse TryFetchInventory(Uri owner, out InventoryCollection inventory);

    BackendResponse TryFetchActiveGestures(Uri owner, out List<InventoryItem> gestures);

    BackendResponse TryCreateItem(Uri owner, InventoryItem item);
    BackendResponse TryCreateFolder(Uri owner, InventoryFolder folder);
    BackendResponse TryCreateInventory(Uri owner, InventoryFolder rootFolder);

    BackendResponse TryDeleteItem(Uri owner, UUID itemID);
    BackendResponse TryDeleteFolder(Uri owner, UUID folderID);
    BackendResponse TryPurgeFolder(Uri owner, UUID folderID);
}

The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.

[edit] Frontends

Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSimulator asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSimulator inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.

[edit] Authentication

Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.

public interface IAuthenticationProvider
{
    void AddIdentifier(UUID authToken, Uri identifier);
    bool RemoveIdentifier(UUID authToken);
    bool TryGetIdentifier(UUID authToken, out Uri identifier);
}

Authentication providers should expire UUID to URI mappings after some amount of inactivity.

[edit] Authorization

Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.

public interface IAuthorizationProvider
{
    bool IsMetadataAuthorized(UUID authToken, UUID assetID);
    bool IsDataAuthorized(UUID authToken, UUID assetID);
    bool IsCreateAuthorized(UUID authToken);

    bool IsInventoryReadAuthorized(UUID authToken, Uri owner);
    bool IsInventoryWriteAuthorized(UUID authToken, Uri owner);
}



                                • MODIFY ALL THE TEXT ABOVE THIS AREA ********************

[edit] See Also

Personal tools
General
About This Wiki