GridServer/DeveloperDocs - Revision history

MakoBot: Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name

2012-03-04T06:33:07Z

Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name

MakoBot: Robot: Cosmetic changes

2012-03-04T03:06:20Z

Robot: Cosmetic changes

MakoBot: Removed 'Template:' prefix from template includings and/or changed external-link into internal-link

2011-06-10T14:27:53Z

Removed 'Template:' prefix from template includings and/or changed external-link into internal-link

Makopoppo at 09:01, 4 June 2011

2011-06-04T09:01:28Z

Mark.malewski: /* See Also */ Updated "See Also" heading with links.

2009-05-21T20:15:44Z

‎See Also: Updated "See Also" heading with links.

Mark.malewski: /* See Also */ Fixed "Technical Reference" pages link, and updated description text. - Please ignore this page, it's a work in progress. Thank-you!

2009-05-21T19:33:28Z

‎See Also: Fixed "Technical Reference" pages link, and updated description text. - Please ignore this page, it's a work in progress. Thank-you!

Mark.malewski: /* See Also */ Added link for "Technical Reference" pages to the "See Also" heading.

2009-05-21T19:32:06Z

‎See Also: Added link for "Technical Reference" pages to the "See Also" heading.

Mark.malewski: /* See Also */ Added links for "GridServer" pages, and "AssetServer" pages to the "See Also" heading. - This page is a work in progress. Please temporarily ignore this page!

2009-05-21T19:30:33Z

‎See Also: Added links for "GridServer" pages, and "AssetServer" pages to the "See Also" heading. - This page is a work in progress. Please temporarily ignore this page!

Mark.malewski: Added "Modify all the text" above and below this area tags. This page needs work! Please edit the content of this page, and make it something meaningful.

2009-05-21T19:24:54Z

Added "Modify all the text" above and below this area tags. This page needs work! Please edit the content of this page, and make it something meaningful.

Mark.malewski: Created GridServer/DeveloperDocs page - This page is under construction and is a work in progress - Please ignore this page!

2009-05-21T19:18:06Z

Created GridServer/DeveloperDocs page - This page is under construction and is a work in progress - Please ignore this page!

New page

__NOTOC__
{{Template:Quicklinks}}

[[Technical Reference | Technical Reference]] -> [[Technical Reference/terms | Terms]] -> [[GridServer | GridServer]] -> [[GridServer/DeveloperDocs | GridServer Developer Documentation]]
{{proposal}}
{{content}}

=Developer Documentation=

This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah. Enter some text here.

* An exception to this rule is the MemcacheStorage provider, which stacks on top of another storage provider to provide scalable caching.

==Code Diagram==

The following diagram shows the interaction between the different interfaces, frontends, and external resources.

'''WARNING:''' The model has changed to require frontends to contact the authorization provider instead of passing authorization tokens to the backend. This allows insecure frontends such as the OpenSimFrontend and OpenSimInventoryFrontend to function alongside secured frontends. It also allows partial security such as the BrowseFrontend, where asset IDs are shown but no additional metadata. This diagram needs to be updated to reflect the changes.

[[Image:DistributedAssetServer-UML.png]]

==Metadata==

All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:

<pre>public class Metadata
{
public UUID ID;
public string Name;
public string Description;
public DateTime CreationDate;
public string ContentType;
public byte[] SHA1;
public bool Temporary;
public Dictionary<string, Uri> Methods;
public OSDMap ExtraData;
}</pre>

===Content-Type===

Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:

<pre>public static string SLAssetTypeToExtraData(int assetType)
{
switch (assetType)
{
case 0: return "image/jp2";
case 1: return "application/ogg";
case 2: return "application/x-metaverse-callingcard";
case 3: return "application/x-metaverse-landmark";
case 5: return "application/x-metaverse-clothing";
case 6: return "application/x-metaverse-primitive";
case 7: return "application/x-metaverse-notecard";
case 10: return "application/x-metaverse-lsl";
case 11: return "application/x-metaverse-lso";
case 12: return "image/tga";
case 13: return "application/x-metaverse-bodypart";
case 17: return "audio/x-wav";
case 19: return "image/jpeg";
case 20: return "application/x-metaverse-animation";
case 21: return "application/x-metaverse-gesture";
case 22: return "application/x-metaverse-simstate";
default: return "application/octet-stream";
}
}</pre>

===Methods===

A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.

===ExtraData===

The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.

==Storage==

Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.

<pre>public interface IStorageProvider
{
BackendResponse TryFetchMetadata(UUID assetID, out Metadata metadata);
BackendResponse TryFetchData(UUID assetID, out byte[] assetData);
BackendResponse TryFetchDataMetadata(UUID assetID, out Metadata metadata, out byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData);
BackendResponse TryCreateAsset(Metadata metadata, byte[] assetData, out UUID assetID);
int ForEach(Action<Metadata> action, int start, int count);
}</pre>

The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.

==Inventory==

Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.

<pre>public interface IInventoryProvider
{
BackendResponse TryFetchItem(Uri owner, UUID itemID, out InventoryItem item);
BackendResponse TryFetchFolder(Uri owner, UUID folderID, out InventoryFolder folder);
BackendResponse TryFetchFolderContents(Uri owner, UUID folderID, out InventoryCollection contents);
BackendResponse TryFetchFolderList(Uri owner, out List<InventoryFolder> folders);
BackendResponse TryFetchInventory(Uri owner, out InventoryCollection inventory);

BackendResponse TryFetchActiveGestures(Uri owner, out List<InventoryItem> gestures);

BackendResponse TryCreateItem(Uri owner, InventoryItem item);
BackendResponse TryCreateFolder(Uri owner, InventoryFolder folder);
BackendResponse TryCreateInventory(Uri owner, InventoryFolder rootFolder);

BackendResponse TryDeleteItem(Uri owner, UUID itemID);
BackendResponse TryDeleteFolder(Uri owner, UUID folderID);
BackendResponse TryPurgeFolder(Uri owner, UUID folderID);
}</pre>

The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.

==Frontends==

Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.

==Authentication==

Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.

<pre>public interface IAuthenticationProvider
{
void AddIdentifier(UUID authToken, Uri identifier);
bool RemoveIdentifier(UUID authToken);
bool TryGetIdentifier(UUID authToken, out Uri identifier);
}</pre>

Authentication providers should expire UUID to URI mappings after some amount of inactivity.

==Authorization==

Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.

<pre>public interface IAuthorizationProvider
{
bool IsMetadataAuthorized(UUID authToken, UUID assetID);
bool IsDataAuthorized(UUID authToken, UUID assetID);
bool IsCreateAuthorized(UUID authToken);

bool IsInventoryReadAuthorized(UUID authToken, Uri owner);
bool IsInventoryWriteAuthorized(UUID authToken, Uri owner);
}</pre>

= See Also =

* [[GridServer|Grid Server ]]
* [[GridServer/ClientDocs|Client Documentation]]

[[Category:Support]]
[[Category:Tech Reference]]
[[Category:Help]]
[[Category:Configuration]]
[[Category:Getting_Started]]
[[Category:Development]]
[[Category:Todo]]
[[Category:Proposal]]

@@ Line 124: / Line 124: @@
 == Frontends ==
-Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
+Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSimulator asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSimulator inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
 == Authentication ==

@@ Line 161: / Line 161: @@
 = See Also =
-* [[AssetServer|Asset Server]]
+* [[:Category:Getting Started|Getting Started]]
-* [[AssetServer/ClientDocs|AssetServer Client Documentation]]
+* [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]]
-* [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]]
+* [[UserServer|User Server ]]
 * [[GridServer|Grid Server ]]
 * [[GridServer/ClientDocs|GridServer Client Documentation]]
 * [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
 * [[:Category:Tech Reference|Technical Reference Pages]]
 [[Category:Support]]

@@ Line 167: / Line 167: @@
 * [[GridServer/ClientDocs|GridServer Client Documentation]]
 * [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
-* [[Category:Tech Reference|Technical Reference Pages]]
+* [[:Category:Tech Reference|Technical Reference Pages]]
 [[Category:Support]]

@@ Line 167: / Line 167: @@
 * [[GridServer/ClientDocs|GridServer Client Documentation]]
 * [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
+* [[Category:Tech Reference|Technical Reference Pages]]
 [[Category:Support]]

@@ Line 1: / Line 1: @@
 __NOTOC__
 {{Quicklinks}}
-[[GridServer/DeveloperDocs | GridServer Developer Documentation]]
+[[GridServer/DeveloperDocs|GridServer Developer Documentation]]
 {{proposal}}
 {{content}}
-=Developer Documentation=
+= Developer Documentation =
-This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah.  Enter some text here.
+This documentation gives an overview of the distributed grid server inner workings. The grid server uses <<type something here>>, blah, blah, blah. Enter some text here.
 * Put some text here, blah, blah, blah.
@@ Line 18: / Line 18: @@
 ----------------------------------------------------------------------------
-****************   MODIFY ALL THE TEXT BELOW THIS AREA  ********************
+**************** MODIFY ALL THE TEXT BELOW THIS AREA ********************
 ----------------------------------------------------------------------------
-==Code Diagram==
+== Code Diagram ==
 The following diagram shows the interaction between the different interfaces, frontends, and external resources.
@@ Line 28: / Line 28: @@
 [[Image:DistributedAssetServer-UML.png]]
-==Metadata==
+== Metadata ==
 All assets share a basic set of common metadata. Storage backends should make a best attempt to support the entire common metadata set, and use default values for unsupported fields. The common metadata structure is listed below:
@@ Line 45: / Line 45: @@
 }</pre>
-===Content-Type===
+=== Content-Type ===
 Asset type is distinguished with an Internet Media Type (MIME type). For backward compatibility with OpenSim's Second Life(tm) implementation, a set of nonstandard content types map to SL asset types. For reference, here is the function that does the conversion:
@@ Line 73: / Line 73: @@
 }</pre>
-===Methods===
+=== Methods ===
 A mapping of strings to URIs holds all of the available methods for an asset. The only common method between all assets is "data", although access level restrictions may prevent users from seeing any methods at all. The method URIs do not necessarily map to resources on the same server that hosts the metadata.
-===ExtraData===
+=== ExtraData ===
 The SimpleStorage field is used to store extended values that can be serialized by a frontend interface without any additional knowledge of the data contained in it. The SimpleStorage backend defines two additional metadata fields for JPEG2000 textures: components and layer_ends. components is an integer type holding the number of components (red, green, blue, alpha, etc.) for a texture. layer_ends is an array of integers denoting byte positions in the file where each quality layer ends. The data is stored as an Open Structured Data (OSD) map, allowing frontends to enumerate over the collection of values and serialize to a wire format without prior knowledge of the structure.
-==Storage==
+== Storage ==
 Storage providers are responsible for accessing the metadata and file data for assets. Example storage providers include local file storage (OpenSim), using the Amazon Simple Storage Service to host data and associated metadata (AmazonS3Storage), an existing AssetServer.Grid.OpenSim.exe MySQL database (OpenSimMySQLStorage), or a caching storage provider (MemcacheStorage) that works in conjunction with another storage provider. Storage providers must implement the IStorageProvider interface.
@@ Line 97: / Line 97: @@
 The StorageResponse enum has the possible values Success, NotFound, AuthNeeded, and Failure.
-==Inventory==
+== Inventory ==
 Inventory providers are similar to storage providers but implement a set of interface functions for dealing with inventory items. Inventory is a hierarchical (folder-based) storage system for small inventory item files that contain additional metadata and link to assets. Currently the only inventory backend is SimpleInventory, using file-system based storage for inventory items and folders.
@@ Line 122: / Line 122: @@
 The TryFetchActiveGestures() interface function is currently required by OpenSim, but may be removed in a future release.
-==Frontends==
+== Frontends ==
 Frontends provide the HTTP interface(s) that users, simulators, and client applications communicate with. Frontends do not implement any interface because no other code depends on them. Unlike the interfaces, many frontends can run at the same time, serving up the same backend content through several different protocols. Source code files implementing frontends typically end with "...Frontend.cs". The current frontends include a browsing interface for paging through the assets with a web browser, an OpenSim asset frontend to replace OpenSim.Grid.AssetServer.exe, an OpenSim inventory frontend to replace OpenSim.Grid.InventoryServer.exe, and a reference frontend that uses the new JSON-based wire protocol.
-==Authentication==
+== Authentication ==
 Authentication extensions provide a mapping from authentication tokens to identifiers (URIs). The authentication provider may provide a frontend for authenticating with the asset server (such as the OpenIdAuth extension), but the function interfaces are typically only called by authorization providers to convert authentication tokens to identifiers before performing an authorization check.
@@ Line 139: / Line 139: @@
 Authentication providers should expire UUID to URI mappings after some amount of inactivity.
-==Authorization==
+== Authorization ==
 Authorization providers determine what actions are allowed for assets and inventory trees. Given an authentication token, an authorization provider will contact the authentication provider and retrieve an identifier. Based on the identifier, the requested resource, and a set of rules in the authorization provider, access is either denied or granted. These interface functions are called from storage providers where a failed authorization translates into a return code of AuthNeeded. Frontend providers can deliver this message with an HTML message, an HTTP response code, or any number of different ways.
@@ Line 155: / Line 155: @@
 ----------------------------------------------------------------------------
-****************   MODIFY ALL THE TEXT ABOVE THIS AREA  ********************
+**************** MODIFY ALL THE TEXT ABOVE THIS AREA ********************
 ----------------------------------------------------------------------------
@@ Line 162: / Line 162: @@
 * [[:Category:Getting Started|Getting Started]]
 * [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]]
-* [[UserServer|User Server ]]
+* [[UserServer|User Server]]
 * [[UserServer/ClientDocs|UserServer Client Documentation]]
 * [[UserServer/DeveloperDocs|UserServer Developer Documentation]]
-* [[GridServer|Grid Server ]]
+* [[GridServer|Grid Server]]
 * [[GridServer/ClientDocs|GridServer Client Documentation]]
 * [[GridServer/DeveloperDocs|GridServer Developer Documentation]]
@@ Line 171: / Line 171: @@
 * [[AssetServer/ClientDocs|AssetServer Client Documentation]]
 * [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]]
-* [[InventoryServer|Inventory Server ]]
+* [[InventoryServer|Inventory Server]]
 * [[InventoryServer/ClientDocs|InventoryServer Client Documentation]]
 * [[InventoryServer/DeveloperDocs|InventoryServer Developer Documentation]]
-* [[MessagingServer|Messaging Server ]]
+* [[MessagingServer|Messaging Server]]
 * [[MessagingServer/ClientDocs|MessagingServer Client Documentation]]
 * [[MessagingServer/DeveloperDocs|MessagingServer Developer Documentation]]
 * [[:Category:Tech Reference|Technical Reference Pages]]
 [[Category:Support]]
 [[Category:Tech Reference]]
 [[Category:Help]]
 [[Category:Configuration]]
-[[Category:Getting_Started]]
+[[Category:Getting Started]]
 [[Category:Development]]
 [[Category:Todo]]
 [[Category:Proposal]]