[Opensim-dev] Decoupling Was: [Opensim-users] Blender Exporter for OpenSim

Fri Dec 5 17:59:47 UTC 2008

Hi Rhian,

The inner workings of the distributed asset server code are explained at http://opensimulator.org/wiki/AssetServerProposal/DeveloperDocs

The more important takeaway is the high level architecture. Some of the main goals of the new asset server model:

* Offload bandwidth and processing from the simulator, and leverage existing content distribution methods. Currently, 50-70% of the simulator bandwidth is texture data. No matter how well the protocol is optimized, you hit a concurrency limit much faster if the simulator has to worry about pushing static content as well as state information. The content distribution problem has already been solved on the web with the use of server farms, squid proxy, memcached, third party CDNs such as CoDeeN/Akamai/CloudFront, etc. By changing the asset interfaces to a true REST interface with cacheable requests, you can take advantage of the existing models to solve the problem of static content distribution.

* Remove the tight coupling between one asset server and one grid.

* Explore new usages of virtual world content. If assets change from UUIDs accessible through a simulator to addressable REST interfaces on the Internet, you have a direct path to Blender/Maya authoring plugins that publish direct to a grid, content spidering search engines for grids that allow open access to that metadata, or auditing services that check for duplicate or similar content. Additionally, content creators can now own and distribute their own content. A service like SLExchange can host assets directly and handle sales and authorization directly without brokering through a specific simulation.

* Move the point of authorization to the home of the assets. In the current model, the authority controlling the simulation infrastructure also administers the access controls to content. This works fine for a single grid model where all of the content and all of the simulation is administered by the same authority. It breaks down when you have one asset server or a clustering of asset servers that provide content for multiple grids (especially the SLExchange example given above). Given the information bits of who a user is, what [grid/agent domain/user server/trust circle/etc] the user is authenticated with, and what operation the user wants to do on what content, the asset server becomes the final decider on whether that access is allowed or not. An small example of the benefit from this: LSL scripts can be protected at the asset server instead of relying on the trust relationship between the asset server and any simulators that communicate with it (which, with Hypergrid now becomes a lot of untrusted simulators).

To answer your question, the distributed asset server proposal doesn't explain where the asset services are supposed to live, how many you need to run, etc. It's a way of breaking apart some of the existing assumptions so different configurations can be experimented with and tested.

John

> -----Original Message-----
> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev- 
> bounces at lists.berlios.de] On Behalf Of Rhian
> Sent: Friday, December 05, 2008 12:21 AM
> To: opensim-dev at lists.berlios.de
> Subject: Re: [Opensim-dev] Decoupling Was: [Opensim-users] Blender 
> Exporter for OpenSim
> 
> Hi John, and everyone else,
> 
> I have looked at it with great interest, but I don't quite understand
> the whole issue fully.
> The proposal as I see it, seems to unload the traffic from the
> simulator. I think that it is not that the simulator can't handle the
> traffic/requests, but the asset server/asset storage. After all when
> 40 agents are retrieving the data of a region in the current
> situation, the region hands out the cached data before retrieving the
> rest from the asset server. In the proposed situation I can only see
> the traffic lessening on the region server, but increasing on the
> asset server (unless it caches for hundreds of regions which is
> unrealistic). I think this will be increasing a bottleneck where we don't want one.
> 
> At first I thought that a distributed asset server would put the
> assets of an instance on an asset server which is coupled with that instance.
> (Is that what you mean by the FrontEnd extension? It is the first time
> I read about that, if so I'd like to know more.) Sort of like the
> cache but more permanent, which unloads the network of the current
> bandwidth needed to retrieve data from the asset server. In that case
> the only traffic going to the asset server is the data for the avatar
> visiting that region. Or when the agent rezzes something from
> inventory. The local asset instance will be functioning initially as a
> pass-through to retrieve the data, then stores it until the object has been removed.
> 
> When a new asset is generated the UUID could be retrieved from the
> asset server, then stored in both local and server instance. I'm not
> knowledgeable enough of the inner workings and I don't know enough C#
> to understand the code (I did try though), so I could be saying
> something impossible. (On a side note, any recommendations to start
> learning C#?)
> 
> Again apologies in case I ask what you're already saying in a way, but
> I'm trying to understand and think along.
> 
> I just noticed that Stefan asked a similar question about the
> OpenSimFrontend...
> 
> Rhian
> 
> On Thu, Dec 4, 2008 at 11:52 PM, Hurliman, John
> <john.hurliman at intel.com> wrote:
>> Take a look at the flow diagrams for the distributed asset server at
>> http://opensimulator.org/wiki/AssetServerProposal/ClientDocs
>> 
>> The only thing left out of that diagram is that the distributed
>> asset
> server is also compatible with the OpenSim protocol today by running
> with the OpensimFrontend extension, acting as a replacement for the
> current asset server. But instead of a single monolithic service, you
> can run as many as you want (in your diagram the region asset cache
> could be an asset server running local to the region) and the data can
> be sourced from anywhere (such as the AmazonS3+CloudFront extension in
> SVN).
>> 
>> John
>> 
>>> -----Original Message-----
>>> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>>> bounces at lists.berlios.de] On Behalf Of Stefan Andersson
>>> Sent: Thursday, December 04, 2008 12:57 PM
>>> To: opensim-dev at lists.berlios.de
>>> Subject: [Opensim-dev] Decoupling Was: [Opensim-users] Blender
>>> Exporter for OpenSim
>>> 
>>> Ok, so this has been rotting in my draft folder for so long, I'm just
>>> going to send it. Consider it just kicking mental box-walls.
>>> 
>>> One of the biggest (but not necessarily hardest) changes to opensim
>>> that would really get some balls rolling, would be to change how we
>>> think of assets from server-centric to region-centric. And, with
>>> that, introduce urls as the 'real' assetId.
>>> 
>>> The assetId of today is, at its core, a thing between the region
>>> and its connected clients.
>>> 
>>> Today, the region already fetches the assets with an http call.
>>> 
>>> So, if we had a model
>>> 
>>> client <-- assetId --> Region Asset Cache <-- assetUrl --> binary
>>> asset http resource
>>> 
>>> we could still use the 'old' assetService by just fabricating a
>>> standard url, just like we do today
>>> 
>>> BUT
>>> 
>>> we would also be able to refer to asset resources by url... web style.
>>> In practice, killing off the need of the monolithic asset server as it
>>> stands today (being all synchronous and stuff)
>>> 
>>> this would not be any problem with libomv, as the protocol would
>>> still be the same; the difference would be that some entity, quite
>>> possibly even the clientview/clientmanager would have to keep track
>>> of what asset url is connected to what assetId.
>>> 
>>> It's SUCH an low hanging fruit, it would be really cool if somebody
>>> would just play around with that asset/url substitution and see where
>>> the actual problems show up.
>>> 
>>> Best regards,
>>> Stefan Andersson
>>> Tribal Media AB
>>> 
>>> Join the 3d web revolution : http://tribalnet.se/
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ________________________________
>>> 
>>> 
>>> 
>>>> Date: Mon, 22 Sep 2008 11:42:36 +0530
>>>> From: shreekumar at hp.com
>>>> To: opensim-users at lists.berlios.de
>>>> Subject: Re: [Opensim-users] Blender Exporter for OpenSim
>>>> 
>>>> Diva Canto wrote:
>>>>> If possible, it would be better if the objects would be exported to
>>>>> an external representation, instead of the MySql DB. Ideally, the
>>>>> external representation would refer to the textures by URL
>>>>> (including the local file system), and the right thing would happen
>>>>> when OpenSim would parse that. So no cryptic info.
>>>>> 
>>>>> 
>>>> +1. That's a downside of my exporter : it writes directly to the
>>>> database. Not pretty.
>>>> Issue is that I need to restart my opensim instance every time I
>>>> export the scene.
>>>> 
>>>> The "load-xml2" command doesn't seem to fit these requirements too.
>>>> 
>>>> -- Shree
>>>> _______________________________________________
>>>> Opensim-users mailing list
>>>> Opensim-users at lists.berlios.de
>>>> https://lists.berlios.de/mailman/listinfo/opensim-users
>>> 
>>> 
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev