Asset Server Proposal

Motivation

Many virtual worlds are being developed independently with a wide variety of different architectures and protocols. However, one common component across these worlds is an asset service. An asset service is the means of storing and serving static content that populates the world; anything from textures and 3D meshes to clothing definitions can be stored. The current generation of virtual world asset services are implemented as proprietary, behind-the-simulator services that are not directly available to a user.

We believe there are shortcomings with this approach. Specifically:

1. Assets are tied to a single grid. Wide availability of content is critical to the growth of virtual worlds, and the inability to share content between grids limits the ability of small providers. Having an asset server closely tied to a grid limits the ability of many small grids to reach critical mass.
2. The current vendor-specific nature of asset storage limits the ability of third-party value added services. Offline content creation, asset archiving, specialized caching, and web service integration are all services that could be added with a more open approach.
3. With assets "locked up" in proprietary asset servers, the ability to combine and re-purpose content is limited. For instance, specialized asset processing tuned to mobile applications.
4. An asset server closely tied to a grid limits the content creator and owner's ability to manage asset lifecycle. An asset owner may wish to have their assets stored, backed up, and tracked from their own servers rather than handing over control to a grid.

Vision

We envision virtual world grids modeled after the current World Wide Web, with millions of independent administrative domains. Content will be spread across millions of small, independent domains as well as aggregated on large domains supporting millions of users. A rich community of value added services and the free and open exchange of content will weave the network together, much as the Web 2.0 movement is tying the web together today. Every organization can choose to host an asset service or connect with any number of third party asset providers. Additionally, the content rights decisions are placed in the hands of the content hosts. With the proper authentication users are free to move assets to wherever they roam.

Data services will become as important as data hosting itself. Just as search engines and content portals have changed how we use the web, services that can plug into a common interface in asset hosting will change how we use virtual worlds. Auditing services can provide an approach to rights management and traffic analytics. Existing caching techniques and services that have been built for today's web content can be leveraged for delivery of rich virtual world content.

Approach

Most implementations, including Second Life® and its open source derivatives, have opaque asset hosting that is a part of the world itself. The opaque asset hosting model has a choke point at the simulator level. Asset downloads make up the bulk of virtual world bandwidth requirements, and in the current Second Life model all assets must be proxied through the simulator.

In the proposed architecture, control of assets is moved from grid owners to content owners to allow cross-world assets with content authorization management. Protocol enhancements could include better support for content rights, asset hosting as an independent service, asset migration between worlds, and scalable performance by allowing centralized servers to be removed from the equation.

Our approach is to start simple by proposing common interfaces and paths for extensibility. By working with the community to refine these interfaces and explore use cases, we can outline specifications and minimum requirements documents. The short-term goals are to consolidate on a common metadata format with core features for version one, and build a reference asset service that plugs into the OpenSim UGAIM model.

Asset Hosting as An Independent Service

The current model of asset hosting approaches trust and permissions from the viewpoint of a grid. Users authenticate with a grid login service, upload assets to grid asset storage, and set permissions that apply to other users of that grid. By moving authorization to the asset storage, an asset service no longer needs to be tied to a specific grid and can serve anyone who is properly authenticated. This opens up the possibility of web service access to assets or asset metadata, or other use cases that do not directly involve logging in to a virtual world.

Leveraging Existing Infrastructure

With asset serving separated from grid-specific code, the solution looks similar to existing solutions. Proven infrastructure for asset lookups, authentication, serving metadata, and content distribution can be used as building blocks.

Scalability

The difficult research in scalable content distribution has already been done and implemented across the web in various different implementations. The goal is to reuse that existing knowledge for virtual world assets, adding optional features such as identity-based permission levels or new caching strategies.

Terminology

• Client: From the asset server's point of view, the networked program that is fetching or storing an asset.
• Asset Server: The network service that presents an interface allowing fetching and storing of asset items.
• Asset: A typed blob of data. The actual definition of asset content is left to the asset creator.
• Asset Metadata: Information stored along with the asset blob which includes type-specific information, ownership information, creation information (creation data), license, and anything else useful in identifying and controlling the asset data.
• Asset ID: A 128-bit identifier for content in a virtual world. These identifiers may or may not follow RFC 4122; the specific implementation is out of scope of the asset service. Asset IDs only have context in the grid they were created in.
• Authentication Token: A session cookie that is temporarily stored on the Asset Server and the Client that proves authentication and may grant some level of authorization. These cookies take the same UUID form as Asset IDs.
• Grid: For the purposes of this document, a grid is defined as the set of services where a particular Asset ID has contextual meaning.

Protocol Overview

1. The end user retrieves an authorization token to connect to an asset service. This can be returned by directly connecting to an asset service and supplying OpenID credentials, or an out of band method such as logging into a virtual world grid server.
2. The end user does a GET operation, supplying the credentials from step 1, and requests the metadata for an asset ID.
3. The asset service performs a lookup step which resolves the ID to metadata which includes the location of the content, a list of supported REST operations on the content, and additional information, such as the dimensions of a texture.
4. The end user has information on different valid operations on the asset, and in the most common scenario will choose to use the fetch operation to download the asset.

All user-facing services must be available over standard HTTP 1.1, and must adopt at least one common metadata format.

Asset Metadata Request

An end user must discover the asset service through some means. The specific means of discovery is out of scope of this document but could for example be built into a client application, provided during grid login, or published on a webpage. To request asset metadata, an HTTP 1.1 GET is made to the asset service with the asset ID and metadata operation specified. For example, a request might look like:

GET /89556747-24cb-43ed-920b-47caed15465f/metadata HTTP/1.1
Host: assets.virtualworld.com
Date: Mon, 22 Sep  2008 12:00:00 GMT
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94

Authentication

Access to the asset service methods is controlled with a authentication token, a UUID. This UUID is a non-guessable, temporary token assigned to an end user at the end of a successful authentication process. This token could be granted by authentication through a virtual world login server (where the login server is responsible for transmitting the authentication token to the asset server), or through a direct login to the asset service itself using OpenID. Temporary tokens must only be given out to authenticated end users, and should be expired similar to HTTP cookies.

If the asset service supports direct logins, it should respond to unauthorized requests with a response that includes an OpenID Endpoint URL. [METADATA TODO]

Once an authenticated request for an asset has been made, the asset service determines if the authenticated user has authorization for the requested content. If a request is unauthorized, a 403 Forbidden response must be sent back to the end user, optionally including an OpenID Endpoint URL so the user can choose to authenticate with another identity.

Lookup

Once a request has been authorized, a lookup is performed. This could be as simple as a map<UUID,metadata> or a scalable solution such as a distributed hash table. The lookup resolves the asset metadata and asset location, and together with the authorization can determine the list of supported actions that are returned in metadata.

Metadata

The metadata provides information on supported methods for the resource and information about the resource itself (such as the size, dimensions, and average color of a texture). One of the goals of this research is to work with the community to develop a common metadata format for requests and responses. [METADATA TODO]

Examples

The following example shows a request for the lowest quality layer of a texture asset.

GET /535f9839-64a9-4146-9e14-a2e2e7896cf1/metadata HTTP/1.1
Host: assets.virtualworld.com
Date Mon, 22 Sep  2008 12:00:00 GM
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e944

HTTP 1.1 200 OK
Date: Mon, 22 Sep  2008 12:00:00 GMT
Last-Modified: Sun, 1 Jan 2006 12:00:00 GMT
Content-Type: application/xml
Content-Length: 223

<?xml version="1.0" encoding="UTF-8"?>
<properties>
 <layer value="0" start="0" end="334"/>
 <layer value="1" start="334" end="98281"/>
 <width value="512"/><height value="256"/>
</properties>
<methods>
 <method value="data"/>
</methods>

GET /89556747-24cb-43ed-920b-47caed15465f/data HTTP/1.1
Host: assets.virtualworld.com
Date: Mon, 22 Sep  2008 12:00:00 GMT
Authorization: OpenGrid ce9a6995-a737-4206-bae8-910cae310e94
Range: bytes=0-334

HTTP 1.1 200 OK
Date: Mon, 22 Sep  2008 12:00:00 GMT
Last-Modified: Sun, 1 Jan 2006 12:00:00 GMT
Content-Type: image/jp2
Content-Length: 334

...