[Opensim-dev] Decoupling Was: [Opensim-users] Blender Exporter for OpenSim

Fri Dec 5 17:39:54 UTC 2008

There are a number of potential benefits for the asset server. (I'm
being careful to say "potential" because a big part of the reason for
building the asset server is to test these assumptions.)

1) move client asset retrieval from the simulator

there are a couple good things that come from this... first, there are
distinct differences between communication, computation and storage
requirements for asset delivery than for scene updates. and there has
already been a lot of relevant work done on building highly scalable
content delivery systems (think Akamai, Coral or CoDeeN for scaling
the front end and S3 or any replicated DB to scale the backend). a
second big win is that assets can be made available to multiple grids.
john hasn't talked much about the inventory extensions that go with
the asset server, but the combination means that i have one inventory
that can be accessed/managed across multiple grids.

2) move authorization from the region servers back to the asset server

right now, the asset server returns whatever it has stored to whomever
asks for it. all authorization for access to an asset is done in the
region servers. that assumes a pretty good trust model between region
servers and asset servers. note that as we move to more
decentralization in our grids (like what Hypergrid enables) this will
become more and more of an issue.

3) open up a bunch of value-add services on assets

in addition to the potential performance gains for the simulator
(which we're still testing)... the real value is in what Stefan
started to describe... the ability for third party services to add
value... such as a blender plugin that publishes directly to the asset
server. we're also looking at how the architecture could be used to
create the equivalent of caching proxy servers in the web... where a
local cache could be managed. in that architecture it becomes possible
to pre-fetch content. for example, a region might provide a torrent
that contains a single package with all the assets in the region that
could be pre-loaded into the local cache. that's more or less what
happens when you buy a game and install the DVD... it puts a bunch of
content on your local disk so that the network can be used just for
scene updates.

anyway... as i said... this is a "research" vehicle for us. well...
research in that we hope to learn what works & what doesn't work, but
real enough that it could be used as a drop in replacement for the
existing opensim asset servers.

--mic

C. Mic Bowman, PhD
Principal Engineer, Intel Corp

On Fri, Dec 5, 2008 at 12:21 AM, Rhian <dutchy.rhian at gmail.com> wrote:
> Hi John, and everyone else,
>
> I have looked at it with great interest, but I don't quite understand
> the whole issue fully.
> The proposal as I see it, seems to unload the traffic from the
> simulator. I think that it is not that the simulator can't handle the
> traffic/requests, but the asset server/asset storage. After all when
> 40 agents are retrieving the data of a region in the current
> situation, the region hands out the cached data before retrieving the
> rest from the asset server. In the proposed situation I can only see
> the traffic lessening on the region server, but increasing on the
> asset server (unless it caches for hundreds of regions which is
> unrealistic). I think this will be increasing a bottleneck where we
> don't want one.
>
> At first I thought that a distributed asset server would put the
> assets of an instance on an asset server which is coupled with that
> instance. (Is that what you mean by the FrontEnd extension? It is the
> first time I read about that, if so I'd like to know more.) Sort of
> like the cache but more permanent, which unloads the network of the
> current bandwidth needed to retrieve data from the asset server. In
> that case the only traffic going to the asset server is the data for
> the avatar visiting that region. Or when the agent rezzes something
> from inventory. The local asset instance will be functioning initially
> as a pass-through to retrieve the data, then stores it until the
> object has been removed.
>
> When a new asset is generated the UUID could be retrieved from the
> asset server, then stored in both local and server instance. I'm not
> knowledgeable enough of the inner workings and I don't know enough C#
> to understand the code (I did try though), so I could be saying
> something impossible. (On a side note, any recommendations to start
> learning C#?)
>
> Again apologies in case I ask what you're already saying in a way, but
> I'm trying to understand and think along.
>
> I just noticed that Stefan asked a similar question about the OpenSimFrontend...
>
> Rhian
>
> On Thu, Dec 4, 2008 at 11:52 PM, Hurliman, John <john.hurliman at intel.com> wrote:
>> Take a look at the flow diagrams for the distributed asset server at http://opensimulator.org/wiki/AssetServerProposal/ClientDocs
>>
>> The only thing left out of that diagram is that the distributed asset server is also compatible with the OpenSim protocol today by running with the OpensimFrontend extension, acting as a replacement for the current asset server. But instead of a single monolithic service, you can run as many as you want (in your diagram the region asset cache could be an asset server running local to the region) and the data can be sourced from anywhere (such as the AmazonS3+CloudFront extension in SVN).
>>
>> John
>>
>>> -----Original Message-----
>>> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>>> bounces at lists.berlios.de] On Behalf Of Stefan Andersson
>>> Sent: Thursday, December 04, 2008 12:57 PM
>>> To: opensim-dev at lists.berlios.de
>>> Subject: [Opensim-dev] Decoupling Was: [Opensim-users] Blender
>>> Exporter for OpenSim
>>>
>>> Ok, so this has been rotting in my draft folder for so long, I'm just
>>> going to send it. Consider it just kicking mental box-walls.
>>>
>>> One of the biggest (but not necessarily hardest) changes to opensim
>>> that would really get some balls rolling, would be to change how we
>>> think of assets from server-centric to region-centric. And, with that,
>>> introduce urls as the 'real' assetId.
>>>
>>> The assetId of today is, at its core, a thing between the region and
>>> its connected clients.
>>>
>>> Today, the region already fetches the assets with an http call.
>>>
>>> So, if we had a model
>>>
>>> client <-- assetId --> Region Asset Cache <-- assetUrl --> binary
>>> asset http resource
>>>
>>> we could still use the 'old' assetService by just fabricating a
>>> standard url, just like we do today
>>>
>>> BUT
>>>
>>> we would also be able to refer to asset resources by url... web style.
>>> In practice, killing off the need of the monolithic asset server as it
>>> stands today (being all synchronous and stuff)
>>>
>>> this would not be any problem with libomv, as the protocol would still
>>> be the same; the difference would be that some entity, quite possibly
>>> even the clientview/clientmanager would have to keep track of what
>>> asset url is connected to what assetId.
>>>
>>> It's SUCH an low hanging fruit, it would be really cool if somebody
>>> would just play around with that asset/url substitution and see where
>>> the actual problems show up.
>>>
>>> Best regards,
>>> Stefan Andersson
>>> Tribal Media AB
>>>
>>> Join the 3d web revolution : http://tribalnet.se/
>>>
>>>
>>>
>>>
>>>
>>>
>>> ________________________________
>>>
>>>
>>>
>>>> Date: Mon, 22 Sep 2008 11:42:36 +0530
>>>> From: shreekumar at hp.com
>>>> To: opensim-users at lists.berlios.de
>>>> Subject: Re: [Opensim-users] Blender Exporter for OpenSim
>>>>
>>>> Diva Canto wrote:
>>>>> If possible, it would be better if the objects would be exported to
>>>>> an external representation, instead of the MySql DB. Ideally, the
>>>>> external representation would refer to the textures by URL
>>>>> (including the local file system), and the right thing would happen
>>>>> when OpenSim would parse that. So no cryptic info.
>>>>>
>>>>>
>>>> +1. That's a downside of my exporter : it writes directly to the
>>>> database. Not pretty.
>>>> Issue is that I need to restart my opensim instance every time I
>>>> export the scene.
>>>>
>>>> The "load-xml2" command doesn't seem to fit these requirements too.
>>>>
>>>> -- Shree
>>>> _______________________________________________
>>>> Opensim-users mailing list
>>>> Opensim-users at lists.berlios.de
>>>> https://lists.berlios.de/mailman/listinfo/opensim-users
>>>
>>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>