[Opensim-dev] Technical assessment of Cable Beach asset server
Justin Clark-Casey
jjustincc at googlemail.com
Thu Jan 15 14:47:22 UTC 2009
Mike Mazur wrote:
> Hello,
>
> I'd like to present my analysis of the Cable Beach[1] design & code,
> with the aim to eventually use it as the official OpenSim asset and
> inventory server.
Hey Mike. Thanks very much for doing this summary. Here are some comments. Sorry these are grouped together but I
think this is better than having them interspersed in a long e-mail.
1) As a general point, is Cable Beach development going to continue externally to OpenSim? Point 3 in
http://opensimulator.org/wiki/AssetServerProposal talks about this being an alternative to proprietary asset servers.
If development of it continues in OpenSim, doesn't it then just remain our own flavour of asset service (albeit with a
few more features and more modularity).
2) I agree with you about the plugin concerns. It would really not be nice to see another plugin infrastructure come
into the OpenSim tree (in addition to Mono plugins and the region module plugin subsystem). My impression was that we
had agreed to an eventual migration to Mono plugins. My understanding was that even if it's technically inferior to
home grown solutions right now, the fact that Mono addins has an independent community (being used in various other
projects such as MonoDevelop) makes it a better long term alternative both from a maintenance and documentation point of
view (we don't have to take on that burden ourselves) and for future functionality (independent teams do nice things
like making modules browsable/changeable via a generic web interface).
So I would argue that we should look to convert Cable Beach to use mono addins if we are going to take it into core.
However, this may make it more difficult to share plugins with non OpenSim Cable Beach instances (if any such exist).
3) HttpServer. Do we really want to be using another http server in addition to HttpServer.dll? This doesn't actually
bother me too much since it's hidden to the user and http is very well known, but it might be an issue.
4) Config files. I seem to remember MW saying that the XML files for grid config are actually legacy. So going to
something that uses Nini (as I believe Cable Beach does) would actually be a good thing, though we would really want to
move all the other grid configs to Nini sooner rather than leaving everything more confusingly inconsistent.
>
> In this email I'll give a brief summary describing why this project was
> developed, describe how it works and highlight a few concerns.
>
> For more information, please refer to the Asset Server Proposal[2],
> Client Documentation[3] and Developer Documentation[4] prepared by John
> Hurliman on the OpenSim wiki.
>
>
> Short Summary of Motivation
> ===========================
>
> Currently assets can only be accessed by logging into a region on the
> particular grid the assets exist on. The main motivation for this new
> asset server is to empower the user to access their assets in any way.
>
> With the built-in authentication capabilities in Cable Beach, regions in
> other grids, or HyperGrid regions, could access assets and inventory
> without the assets or inventory being openly available to anyone on the
> Internet. External applications, such as texture or script editors,
> capable of pushing to the asset server directly become feasible.
>
> Future viewer clients can access assets directly from the asset server.
> Removing asset traffic from the region server results in improved
> region server performance.
>
>
> How It Works
> ============
>
> The main asset server executable[5]:
>
> 1. defines a set of interfaces,
> 2. reads a config file,
> 3. loads plugins, specified in the config file, which implement the
> interfaces,
> 4. starts an HTTP server, and
> 5. starts loaded plugins.
>
> Interfaces
> ----------
>
> The interfaces[6] are:
>
> - IStorageProvider
> - IInventoryProvider
> - IAuthenticationProvider
> - IAuthorizationProvider
> - IMetricsProvider
>
> IStorageProvider declares methods for accessing and creating
> (modifying?) asset data and metadata. How the data is stored depends on
> the storage provider. Current storage provider plugins include:
>
> - SimpleStorage[7], stores assets to disk
> - OpenSimMySQLStorage[8], compatible with the current core OpenSim
> MySQL asset storage
> - AmazonS3Storage[9], stores assets in Amazon's S3 cloud solution
>
> IInventoryProvider declares methods for interacting with inventory.
> Current inventory plugins include:
>
> - SimpleInventory[10], stores inventory information to disk
> - OpenSimMySQLInventory[11], compatible with the current core OpenSim
> MySQL inventory storage
>
> IAuthenticationProvider declares methods for managing authenticated
> users/sessions in the asset server.
>
> IAuthorizationProvider declares methods for determining access to
> assets and inventory. Currently only the AuthorizeAll[12] plugin exists,
> which allows all access.
>
> IMetricsProvider declares methods to collect access statistics.
>
> Config File
> -----------
>
> The asset server uses Nini to read an INI config file[13]. The config
> file allows control over basic things like the port the asset server
> should listen on or the DB connection string for the OpenSimMySQL*
> plugins. Plugins implementing the various interfaces are listed as well.
>
> Loading Plugins
> ---------------
>
> The asset server uses ExtensionLoader[14] for loading the plugins at
> runtime. Currently the plugins to load are specified in the config
> file. If any of the interfaces are not implemented, an exception is
> thrown and the server does not start.
>
> Starting HTTP Server and Plugins
> --------------------------------
>
> The HTTP server in the LibOMV codebase is used[15]. After the HTTP
> server has started, each of the plugins is started.
>
> The frontend plugins will register handlers with the HTTP server
> according to the services they provide. For example, the
> OpenSimFrontend[16] registers a GET and POST handler at /assets/, which
> provide asset retrieval and creation, respectively.
>
> The backend plugins also perform setup actions. The SimpleStorage[7]
> plugin, for instance, will load asset metadata from disk into memory, or
> create the storage directory if it doesn't already exist. The MySQL
> plugins will first check that a valid connection to the DB can be
> established.
>
> On start, the plugins are passed a reference to the main asset server
> object through which they communicate. For example, when a request for
> an asset arrives at the /assets/<UUID>?texture handler of the
> HttpServer, the OpenSimFrontend[16] will access the asset through the
> StoragerProvider, whichever it may be, as follows:
>
> Metadata metadata;
> byte[] assetData;
> server.StorageProvider.TryFetchDataMetadata(assetID,
> out metadata,
> out assetData);
>
>
> Concerns and Thoughts
> =====================
>
> After going through the code and testing the asset server in my
> (admittedly non-production) development environment, I like where this
> thing is going. I would like to see the concepts included in OpenSim.
>
> Not everything currently in the Cable Beach SVN needs to be brought in.
> For starters, the main asset server along with the OpenSim-specific
> plugins and dummy authentication/authorization plugins are all that's
> required. Additional plugins can be available on Forge. This subset of
> the Cable Beach codebase shouldn't duplicate any existing code that
> isn't meant to be replaced.
>
> Storage Backends
> ----------------
>
> Currently only MySQL is supported. Additional storage plugins will have
> to be written to support SQLite, MSSQL, NHibernate and Null storage
> backends. Another alternative could be writing one plugin that uses
> OpenSim's IAssetProviderPlugin interface to support the various
> backends.
>
> ExtensionLoader and Mono Addins
> -------------------------------
>
> ExtensionLoader, used for loading plugins at runtime, duplicates Mono
> Addins functionality and adds another dependency. The path of least
> resistance is probably to have the asset server use Mono Addins
> instead, though I'm not sure what the workload would be like to achieve
> this.
>
> Alternatively, if ExtensionLoader is superior to Mono Addins, it may be
> beneficial to have both in the tree for the time being, and slowly
> convert the plugin architecture to standardize on ExtensionLoader.
> Further investigation is necessary here.
>
> Config Files
> ------------
>
> The asset server configuration will change from the familiar
> AssetServer_Config.xml file identical in format to all the other UGAI
> server config files to an INI file. This can increase support requests
> by users which are unaware that there is a new asset server.
>
> Support issues can be minimized with good documentation that's widely
> publicized and with a long transition period where both asset servers
> are available in the tree. Once most users are comfortable with the new
> asset server, the old one can be removed.
>
>
> Next Steps
> ==========
>
> At this stage, let's discuss. Please ask your questions, raise
> concerns, and we'll work through this.
>
> Thank you for reading,
> Mike
>
>
> References
> ==========
>
> [1] http://forge.opensimulator.org/gf/project/assetserver/
> [2] http://opensimulator.org/wiki/AssetServerProposal
> [3] http://opensimulator.org/wiki/AssetServerProposal/ClientDocs
> [4] http://opensimulator.org/wiki/AssetServerProposal/DeveloperDocs
> [5]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FAssetServer.cs&view=markup
> [6]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FInterfaces.cs&view=markup
> [7]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FSimple%2FSimpleStorage.cs&view=markup
> [8]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FOpenSimMySQL%2FOpenSimMySQLStorage.cs&view=markup
> [9]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FAmazonS3%2FAmazonS3Storage.cs&view=markup
> [10]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FSimple%2FSimpleInventory.cs&view=markup
> [11]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FOpenSimMySQL%2FOpenSimMySQLInventory.cs&view=markup
> [12]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FAuthorizeAll.cs&view=markup
> [13] An example INI file, well documented:
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2Fbin%2FAssetServer.ini&view=markup
> [14] http://code.google.com/p/extensionloader/
> [15] http://openmetaverse.org/svn/misc/HttpServer/
> [16]
> http://forge.opensimulator.org/gf/project/assetserver/scmsvn/?action=browse&path=%2Ftrunk%2FAssetServer%2FExtensions%2FOpenSimFrontend.cs&view=markup
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
--
justincc
Justin Clark-Casey
http://justincc.wordpress.com
More information about the Opensim-dev
mailing list