AssetServer
From OpenSimulator
(New terms defined, removed "capability" reference, clarified authentication section) |
m (Robot: Replacing 'OpenSim' to 'OpenSimulator', which is the precise name) |
||
| (23 intermediate revisions by 7 users not shown) | |||
| Line 1: | Line 1: | ||
| + | __NOTOC__ | ||
| + | {{Quicklinks}} | ||
{{proposal}} | {{proposal}} | ||
| + | {{content}} | ||
| − | =Asset Server | + | = What is an Asset Server? = |
| − | == | + | == Introduction == |
Many virtual worlds are being developed independently with a wide variety of different architectures and protocols. However, one common component across these worlds is an asset service. An asset service is the means of storing and serving static content that populates the world; anything from textures and 3D meshes to clothing definitions can be stored. The current generation of virtual world asset services are implemented as proprietary, behind-the-simulator services that are not directly available to a user. | Many virtual worlds are being developed independently with a wide variety of different architectures and protocols. However, one common component across these worlds is an asset service. An asset service is the means of storing and serving static content that populates the world; anything from textures and 3D meshes to clothing definitions can be stored. The current generation of virtual world asset services are implemented as proprietary, behind-the-simulator services that are not directly available to a user. | ||
| Line 14: | Line 17: | ||
# An asset server closely tied to a grid limits the content creator and owner's ability to manage asset lifecycle. An asset owner may wish to have their assets stored, backed up, and tracked from their own servers rather than handing over control to a grid. | # An asset server closely tied to a grid limits the content creator and owner's ability to manage asset lifecycle. An asset owner may wish to have their assets stored, backed up, and tracked from their own servers rather than handing over control to a grid. | ||
| − | ==Vision== | + | == Vision == |
We envision virtual world grids modeled after the current World Wide Web, with millions of independent administrative domains. Content will be spread across millions of small, independent domains as well as aggregated on large domains supporting millions of users. A rich community of value added services and the free and open exchange of content will weave the network together, much as the Web 2.0 movement is tying the web together today. Every organization can choose to host an asset service or connect with any number of third party asset providers. Additionally, the content rights decisions are placed in the hands of the content hosts. With the proper authentication users are free to move assets to wherever they roam. | We envision virtual world grids modeled after the current World Wide Web, with millions of independent administrative domains. Content will be spread across millions of small, independent domains as well as aggregated on large domains supporting millions of users. A rich community of value added services and the free and open exchange of content will weave the network together, much as the Web 2.0 movement is tying the web together today. Every organization can choose to host an asset service or connect with any number of third party asset providers. Additionally, the content rights decisions are placed in the hands of the content hosts. With the proper authentication users are free to move assets to wherever they roam. | ||
| − | Data services will become | + | Data services will become as important as data hosting itself. Just as search engines and content portals have changed how we use the web, services that can plug into a common interface in asset hosting will change how we use virtual worlds. Auditing services can provide an approach to rights management and traffic analytics. Existing caching techniques and services that have been built for today's web content can be leveraged for delivery of rich virtual world content. |
| − | ==Approach== | + | == Approach == |
Most implementations, including Second Life® and its open source derivatives, have opaque asset hosting that is a part of the world itself. The opaque asset hosting model has a choke point at the simulator level. Asset downloads make up the bulk of virtual world bandwidth requirements, and in the current Second Life model all assets must be proxied through the simulator. | Most implementations, including Second Life® and its open source derivatives, have opaque asset hosting that is a part of the world itself. The opaque asset hosting model has a choke point at the simulator level. Asset downloads make up the bulk of virtual world bandwidth requirements, and in the current Second Life model all assets must be proxied through the simulator. | ||
| Line 26: | Line 29: | ||
In the proposed architecture, control of assets is moved from grid owners to content owners to allow cross-world assets with content authorization management. Protocol enhancements could include better support for content rights, asset hosting as an independent service, asset migration between worlds, and scalable performance by allowing centralized servers to be removed from the equation. | In the proposed architecture, control of assets is moved from grid owners to content owners to allow cross-world assets with content authorization management. Protocol enhancements could include better support for content rights, asset hosting as an independent service, asset migration between worlds, and scalable performance by allowing centralized servers to be removed from the equation. | ||
| − | Our approach is to start simple by proposing common interfaces and paths for extensibility. By working with the community to refine these interfaces and explore use cases, we can outline specifications and minimum requirements documents. The short-term goals are to consolidate on a common metadata format with core features for version one, and build a reference asset service that plugs into the | + | Our approach is to start simple by proposing common interfaces and paths for extensibility. By working with the community to refine these interfaces and explore use cases, we can outline specifications and minimum requirements documents. The short-term goals are to consolidate on a common metadata format with core features for version one, and build a reference asset service that plugs into the OpenSimulator UGAIM model. |
| − | ===Asset Hosting as An Independent Service=== | + | === Asset Hosting as An Independent Service === |
The current model of asset hosting approaches trust and permissions from the viewpoint of a grid. Users authenticate with a grid login service, upload assets to grid asset storage, and set permissions that apply to other users of that grid. By moving authorization to the asset storage, an asset service no longer needs to be tied to a specific grid and can serve anyone who is properly authenticated. This opens up the possibility of web service access to assets or asset metadata, or other use cases that do not directly involve logging in to a virtual world. | The current model of asset hosting approaches trust and permissions from the viewpoint of a grid. Users authenticate with a grid login service, upload assets to grid asset storage, and set permissions that apply to other users of that grid. By moving authorization to the asset storage, an asset service no longer needs to be tied to a specific grid and can serve anyone who is properly authenticated. This opens up the possibility of web service access to assets or asset metadata, or other use cases that do not directly involve logging in to a virtual world. | ||
| − | ===Leveraging Existing Infrastructure=== | + | === Leveraging Existing Infrastructure === |
With asset serving separated from grid-specific code, the solution looks similar to existing solutions. Proven infrastructure for asset lookups, authentication, serving metadata, and content distribution can be used as building blocks. | With asset serving separated from grid-specific code, the solution looks similar to existing solutions. Proven infrastructure for asset lookups, authentication, serving metadata, and content distribution can be used as building blocks. | ||
| − | ===Scalability=== | + | === Scalability === |
The difficult research in scalable content distribution has already been done and implemented across the web in various different implementations. The goal is to reuse that existing knowledge for virtual world assets, adding optional features such as identity-based permission levels or new caching strategies. | The difficult research in scalable content distribution has already been done and implemented across the web in various different implementations. The goal is to reuse that existing knowledge for virtual world assets, adding optional features such as identity-based permission levels or new caching strategies. | ||
| − | ==Terminology== | + | == Terminology == |
* Client: From the asset server's point of view, the networked program that is fetching or storing an asset. | * Client: From the asset server's point of view, the networked program that is fetching or storing an asset. | ||
| Line 50: | Line 53: | ||
* Grid: For the purposes of this document, a grid is defined as the set of services where a particular Asset ID has contextual meaning. | * Grid: For the purposes of this document, a grid is defined as the set of services where a particular Asset ID has contextual meaning. | ||
| − | ==Protocol Overview== | + | == Protocol Overview == |
# The end user retrieves an authorization token to connect to an asset service. This can be returned by directly connecting to an asset service and supplying OpenID credentials, or an out of band method such as logging into a virtual world grid server. | # The end user retrieves an authorization token to connect to an asset service. This can be returned by directly connecting to an asset service and supplying OpenID credentials, or an out of band method such as logging into a virtual world grid server. | ||
| Line 59: | Line 62: | ||
All user-facing services must be available over standard HTTP 1.1, and must adopt at least one common metadata format. | All user-facing services must be available over standard HTTP 1.1, and must adopt at least one common metadata format. | ||
| − | ==Asset Metadata Request== | + | == Asset Metadata Request == |
An end user must discover the asset service through some means. The specific means of discovery is out of scope of this document but could for example be built into a client application, provided during grid login, or published on a webpage. To request asset metadata, an HTTP 1.1 GET is made to the asset service with the asset ID and metadata operation specified. For example, a request might look like: | An end user must discover the asset service through some means. The specific means of discovery is out of scope of this document but could for example be built into a client application, provided during grid login, or published on a webpage. To request asset metadata, an HTTP 1.1 GET is made to the asset service with the asset ID and metadata operation specified. For example, a request might look like: | ||
| Line 68: | Line 71: | ||
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94</pre> | Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94</pre> | ||
| − | ==Authentication== | + | If the asset is a protected resource the Authorization header is required. If the resource is publicly accessible, no Authorization header is needed and a normal HTTP GET (such as one generated by a web browser) can be used. |
| + | |||
| + | == Authentication == | ||
Access to the asset service methods is controlled with a authentication token, a UUID. This UUID is a non-guessable, temporary token assigned to an end user at the end of a successful authentication process. This token could be granted by authentication through a virtual world login server (where the login server is responsible for transmitting the authentication token to the asset server), or through a direct login to the asset service itself using OpenID. Temporary tokens must only be given out to authenticated end users, and should be expired similar to HTTP cookies. | Access to the asset service methods is controlled with a authentication token, a UUID. This UUID is a non-guessable, temporary token assigned to an end user at the end of a successful authentication process. This token could be granted by authentication through a virtual world login server (where the login server is responsible for transmitting the authentication token to the asset server), or through a direct login to the asset service itself using OpenID. Temporary tokens must only be given out to authenticated end users, and should be expired similar to HTTP cookies. | ||
| − | If the asset service supports direct logins, it | + | If the asset service supports direct logins, it may respond to unauthorized requests with a response that includes information about authenticating with OpenID. |
| − | Once an authenticated request for an asset has been made, the asset service determines if the authenticated user has authorization for the requested content. If a request is unauthorized, a 403 Forbidden response must be sent back to the end user, optionally including | + | Once an authenticated request for an asset has been made, the asset service determines if the authenticated user has authorization for the requested content. If a request is unauthorized, a 403 Forbidden response must be sent back to the end user, optionally including information about authenticating with OpenID so the user can choose to authenticate with another identity. |
| − | ==Lookup== | + | == Lookup == |
| − | Once a request has been authorized, a lookup is performed. This could be as simple as a map<UUID,metadata> or a scalable solution such as a distributed hash table. The lookup resolves the asset metadata and asset location, and together with the authorization can determine the list of supported actions that are returned in metadata. | + | Once a request has been authorized, a lookup is performed. This could be as simple as a map<UUID,metadata> or a more scalable solution such as a distributed hash table. The lookup resolves the asset metadata and asset location, and together with the authorization can determine the list of supported actions that are returned in metadata. |
| − | ==Metadata== | + | == Metadata == |
| − | The metadata provides information on supported methods for the resource and information about the resource itself (such as the size | + | The metadata provides information on supported methods for the resource and information about the resource itself (such as the size and dimensions of a texture, or byte boundaries of quality layers in a JPEG2000 file). One of the goals of this research is to work with the community to develop a common metadata format for requests and responses. The current metadata reference implementation uses JSON for the wire protocol. See [[AssetServer/ClientDocs|Asset Server Client Documentation]] for more information. |
| − | + | The metadata is broken up into three parts. Required fields, methods, and extra fields. | |
| − | The | + | The required fields are basic attributes that all assets share. These are: |
| − | + | * id - UUID of the asset | |
| − | + | * name - Name of the asset | |
| − | + | * description - Description of the asset | |
| − | + | * creation_date - Timestamp when the asset was created | |
| + | * type - HTTP Content-Type of the asset | ||
| + | * sha1 - The SHA-1 hash of the asset data | ||
| + | * temporary - Boolean, whether the asset is temporary or not | ||
| − | + | The methods section is a collection of allowed methods for the asset. The only required method is data, however this method may not be present in the list of returned methods if the client has permission to access an assets metadata but not the data. Each method is a string (such as "data") that maps to a URL. The URL may exist on the same server as the metadata, but this is not required. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | The extra data section is a free-form version of the required fields section. Any fields can be defined, and is most commonly used to add extended data for specific types of assets. | |
| − | + | == Data == | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | Once the metadata has been retrieved, if a data method exists the asset data can be fetched with a simple HTTP GET on the asset data URL. This allows the actual data to optionally be stored on a simple web server, a content distribution network, a third party storage provider, etc. | |
| − | + | ||
| − | + | = See Also = | |
| − | + | ||
| − | + | * [[:Category:Getting Started|Getting Started]] | |
| + | * [[OpenSim:Introduction_and_Definitions|What are UGAIM Services?]] | ||
| + | * [[UserServer|User Server]] | ||
| + | * [[UserServer/ClientDocs|UserServer Client Documentation]] | ||
| + | * [[UserServer/DeveloperDocs|UserServer Developer Documentation]] | ||
| + | * [[GridServer|Grid Server]] | ||
| + | * [[GridServer/ClientDocs|GridServer Client Documentation]] | ||
| + | * [[GridServer/DeveloperDocs|GridServer Developer Documentation]] | ||
| + | * [[AssetServer|Asset Server]] | ||
| + | * [[AssetServer/ClientDocs|AssetServer Client Documentation]] | ||
| + | * [[AssetServer/DeveloperDocs|AssetServer Developer Documentation]] | ||
| + | * [[InventoryServer|Inventory Server]] | ||
| + | * [[InventoryServer/ClientDocs|InventoryServer Client Documentation]] | ||
| + | * [[InventoryServer/DeveloperDocs|InventoryServer Developer Documentation]] | ||
| + | * [[MessagingServer|Messaging Server]] | ||
| + | * [[MessagingServer/ClientDocs|MessagingServer Client Documentation]] | ||
| + | * [[MessagingServer/DeveloperDocs|MessagingServer Developer Documentation]] | ||
| + | * [[:Category:Tech Reference|Technical Reference Pages]] | ||
| − | + | [[Category:Support]] | |
| + | [[Category:Tech Reference]] | ||
| + | [[Category:Help]] | ||
| + | [[Category:Configuration]] | ||
| + | [[Category:Getting Started]] | ||
| + | [[Category:Development]] | ||
| + | [[Category:Todo]] | ||
| + | [[Category:Proposal]] | ||
Latest revision as of 22:12, 3 March 2012
 |
Languages: |
|
|
|
This article or section is a Proposal It does not represent the current state of OpenSim, but is an idea for future work in OpenSim. Please feel free to update this page as part of the proposal discussion. |
|
|
This article or section contains incomplete information. Please help us by completing the content on this page. |
[edit] What is an Asset Server?
[edit] Introduction
Many virtual worlds are being developed independently with a wide variety of different architectures and protocols. However, one common component across these worlds is an asset service. An asset service is the means of storing and serving static content that populates the world; anything from textures and 3D meshes to clothing definitions can be stored. The current generation of virtual world asset services are implemented as proprietary, behind-the-simulator services that are not directly available to a user.
We believe there are shortcomings with this approach. Specifically:
- Assets are tied to a single grid. Wide availability of content is critical to the growth of virtual worlds, and the inability to share content between grids limits the ability of small providers. Having an asset server closely tied to a grid limits the ability of many small grids to reach critical mass.
- The current vendor-specific nature of asset storage limits the ability of third-party value added services. Offline content creation, asset archiving, specialized caching, and web service integration are all services that could be added with a more open approach.
- With assets "locked up" in proprietary asset servers, the ability to combine and re-purpose content is limited. For instance, specialized asset processing tuned to mobile applications.
- An asset server closely tied to a grid limits the content creator and owner's ability to manage asset lifecycle. An asset owner may wish to have their assets stored, backed up, and tracked from their own servers rather than handing over control to a grid.
[edit] Vision
We envision virtual world grids modeled after the current World Wide Web, with millions of independent administrative domains. Content will be spread across millions of small, independent domains as well as aggregated on large domains supporting millions of users. A rich community of value added services and the free and open exchange of content will weave the network together, much as the Web 2.0 movement is tying the web together today. Every organization can choose to host an asset service or connect with any number of third party asset providers. Additionally, the content rights decisions are placed in the hands of the content hosts. With the proper authentication users are free to move assets to wherever they roam.
Data services will become as important as data hosting itself. Just as search engines and content portals have changed how we use the web, services that can plug into a common interface in asset hosting will change how we use virtual worlds. Auditing services can provide an approach to rights management and traffic analytics. Existing caching techniques and services that have been built for today's web content can be leveraged for delivery of rich virtual world content.
[edit] Approach
Most implementations, including Second Life® and its open source derivatives, have opaque asset hosting that is a part of the world itself. The opaque asset hosting model has a choke point at the simulator level. Asset downloads make up the bulk of virtual world bandwidth requirements, and in the current Second Life model all assets must be proxied through the simulator.
In the proposed architecture, control of assets is moved from grid owners to content owners to allow cross-world assets with content authorization management. Protocol enhancements could include better support for content rights, asset hosting as an independent service, asset migration between worlds, and scalable performance by allowing centralized servers to be removed from the equation.
Our approach is to start simple by proposing common interfaces and paths for extensibility. By working with the community to refine these interfaces and explore use cases, we can outline specifications and minimum requirements documents. The short-term goals are to consolidate on a common metadata format with core features for version one, and build a reference asset service that plugs into the OpenSimulator UGAIM model.
[edit] Asset Hosting as An Independent Service
The current model of asset hosting approaches trust and permissions from the viewpoint of a grid. Users authenticate with a grid login service, upload assets to grid asset storage, and set permissions that apply to other users of that grid. By moving authorization to the asset storage, an asset service no longer needs to be tied to a specific grid and can serve anyone who is properly authenticated. This opens up the possibility of web service access to assets or asset metadata, or other use cases that do not directly involve logging in to a virtual world.
[edit] Leveraging Existing Infrastructure
With asset serving separated from grid-specific code, the solution looks similar to existing solutions. Proven infrastructure for asset lookups, authentication, serving metadata, and content distribution can be used as building blocks.
[edit] Scalability
The difficult research in scalable content distribution has already been done and implemented across the web in various different implementations. The goal is to reuse that existing knowledge for virtual world assets, adding optional features such as identity-based permission levels or new caching strategies.
[edit] Terminology
- Client: From the asset server's point of view, the networked program that is fetching or storing an asset.
- Asset Server: The network service that presents an interface allowing fetching and storing of asset items.
- Asset: A typed blob of data. The actual definition of asset content is left to the asset creator.
- Asset Metadata: Information stored along with the asset blob which includes type-specific information, ownership information, creation information (creation data), license, and anything else useful in identifying and controlling the asset data.
- Asset ID: A 128-bit identifier for content in a virtual world. These identifiers may or may not follow RFC 4122; the specific implementation is out of scope of the asset service. Asset IDs only have context in the grid they were created in.
- Authentication Token: A session cookie that is temporarily stored on the Asset Server and the Client that proves authentication and may grant some level of authorization. These cookies take the same UUID form as Asset IDs.
- Grid: For the purposes of this document, a grid is defined as the set of services where a particular Asset ID has contextual meaning.
[edit] Protocol Overview
- The end user retrieves an authorization token to connect to an asset service. This can be returned by directly connecting to an asset service and supplying OpenID credentials, or an out of band method such as logging into a virtual world grid server.
- The end user does a GET operation, supplying the credentials from step 1, and requests the metadata for an asset ID.
- The asset service performs a lookup step which resolves the ID to metadata which includes the location of the content, a list of supported REST operations on the content, and additional information, such as the dimensions of a texture.
- The end user has information on different valid operations on the asset, and in the most common scenario will choose to use the fetch operation to download the asset.
All user-facing services must be available over standard HTTP 1.1, and must adopt at least one common metadata format.
[edit] Asset Metadata Request
An end user must discover the asset service through some means. The specific means of discovery is out of scope of this document but could for example be built into a client application, provided during grid login, or published on a webpage. To request asset metadata, an HTTP 1.1 GET is made to the asset service with the asset ID and metadata operation specified. For example, a request might look like:
GET /89556747-24cb-43ed-920b-47caed15465f/metadata HTTP/1.1 Host: assets.virtualworld.com Date: Mon, 22 Sep 2008 12:00:00 GMT Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94
If the asset is a protected resource the Authorization header is required. If the resource is publicly accessible, no Authorization header is needed and a normal HTTP GET (such as one generated by a web browser) can be used.
[edit] Authentication
Access to the asset service methods is controlled with a authentication token, a UUID. This UUID is a non-guessable, temporary token assigned to an end user at the end of a successful authentication process. This token could be granted by authentication through a virtual world login server (where the login server is responsible for transmitting the authentication token to the asset server), or through a direct login to the asset service itself using OpenID. Temporary tokens must only be given out to authenticated end users, and should be expired similar to HTTP cookies.
If the asset service supports direct logins, it may respond to unauthorized requests with a response that includes information about authenticating with OpenID.
Once an authenticated request for an asset has been made, the asset service determines if the authenticated user has authorization for the requested content. If a request is unauthorized, a 403 Forbidden response must be sent back to the end user, optionally including information about authenticating with OpenID so the user can choose to authenticate with another identity.
[edit] Lookup
Once a request has been authorized, a lookup is performed. This could be as simple as a map<UUID,metadata> or a more scalable solution such as a distributed hash table. The lookup resolves the asset metadata and asset location, and together with the authorization can determine the list of supported actions that are returned in metadata.
[edit] Metadata
The metadata provides information on supported methods for the resource and information about the resource itself (such as the size and dimensions of a texture, or byte boundaries of quality layers in a JPEG2000 file). One of the goals of this research is to work with the community to develop a common metadata format for requests and responses. The current metadata reference implementation uses JSON for the wire protocol. See Asset Server Client Documentation for more information.
The metadata is broken up into three parts. Required fields, methods, and extra fields.
The required fields are basic attributes that all assets share. These are:
* id - UUID of the asset * name - Name of the asset * description - Description of the asset * creation_date - Timestamp when the asset was created * type - HTTP Content-Type of the asset * sha1 - The SHA-1 hash of the asset data * temporary - Boolean, whether the asset is temporary or not
The methods section is a collection of allowed methods for the asset. The only required method is data, however this method may not be present in the list of returned methods if the client has permission to access an assets metadata but not the data. Each method is a string (such as "data") that maps to a URL. The URL may exist on the same server as the metadata, but this is not required.
The extra data section is a free-form version of the required fields section. Any fields can be defined, and is most commonly used to add extended data for specific types of assets.
[edit] Data
Once the metadata has been retrieved, if a data method exists the asset data can be fetched with a simple HTTP GET on the asset data URL. This allows the actual data to optionally be stored on a simple web server, a content distribution network, a third party storage provider, etc.
[edit] See Also
- Getting Started
- What are UGAIM Services?
- User Server
- UserServer Client Documentation
- UserServer Developer Documentation
- Grid Server
- GridServer Client Documentation
- GridServer Developer Documentation
- Asset Server
- AssetServer Client Documentation
- AssetServer Developer Documentation
- Inventory Server
- InventoryServer Client Documentation
- InventoryServer Developer Documentation
- Messaging Server
- MessagingServer Client Documentation
- MessagingServer Developer Documentation
- Technical Reference Pages
