[Opensim-dev] [REX] RE: Distributed grid services / LDAP login status

[Ryan McDougall]

John is really touching a lot of issues I have had a personal interest
in myself. Heroic work I think.

I would like to use all of this work in reX -- ideally by taking it
from upstream OpenSim....

Cheers,

On Tue, Dec 16, 2008 at 8:24 PM, Hurliman, John <john.hurliman at intel.com> wrote:
>
>
>
>> -----Original Message-----
>> From: opensim-dev-bounces at lists.berlios.de [mailto:opensim-dev-
>> bounces at lists.berlios.de] On Behalf Of Stefan Andersson
>> Sent: Tuesday, December 16, 2008 5:21 AM
>> To: opensim-dev at lists.berlios.de; realxtend at googlegroups.com
>> Subject: Re: [Opensim-dev] Distributed grid services / LDAP login
>> status
>>
>> John,
>>
>>> * The distributed asset server project
>> (http://forge.opensimulator.org/gf/project/assetserver/) has turned
>> into the distributed grid services project. The asset server is almost
>> at the same functionality as OpenSim.Grid.AssetServer.exe
>>
>> Excellent work, really cool that you're working on this!
>>
>>
>>> (it cannot create the MySQL database from scratch, but everything
>> else seems to be in order),
>>
>> I would argue that this is probably another reason to try to expand on
>> the existing backend infrastructure - try to better what's already in
>> the data layer so we don't end up with yet another set of DB services.
>>
>
> I had a very difficult time pulling the database interaction layer cleanly out of OpenSim and using it as a standalone module. Maybe I was just doing something wrong, but the OpenSim codebase seems to have a lot of dependencies going both up and down the stack. The primary goal of the project is to research virtual world simulation and discover a good path to scalability (both in the sense of grid services scaling, and the entire infrastructure scaling to support millions of independent administrative domains). While the distributed grid services (I think) will provide great replacements for the existing OpenSim services, it needs to be trivial to write new implementations of them in a couple hundred lines of Python, or with a fresh C# project. The first team to adopt the asset service outside of Intel was actually another virtual world research project completely unrelated to Second Life.
>
> Starting with a new codebase also provides the ability to break away from pain points that have been discovered over the last year+ of OpenSim development. For example, I developed ExtensionLoader (http://code.google.com/p/extensionloader/, to improve upon Mono.Addins) which serves as the backbone for the distributed grid services architecture. It's the reason why every login protocol, asset backend, authentication API, etc. in the grid services are modules that can be turned on or off with a config file. I would be happy to write a storage module that used the OpenSim assemblies, but I need a clear picture of how much infrastructure that requires pulling over into the forge project.
>
>>> inventory functionality has been merged into the asset server,
>>
>> That sounds a bit worrisome to me; not knowing the specifics, I'd like
>> to hear a bit more about why?
>>
>
> The first reason started out as a hunch that the inventory server and the new distributed asset server are doing very similar jobs: serving up metadata, pointers to data, and data itself, while providing authentication and authorization layers. The theory proved true; while the optimization methods for storing/accessing inventory are slightly different from storing binary blobs of asset data, the majority of the optimizations are shared between assets and inventory. The second reason is that you need to implement nearly the entire job of the inventory service in the asset server to create a permissions system that resembles the current OpenSim permission system. Applying access restrictions for the "next owner" (ownership is an inventory concept) of an asset in a completely decentralized model generally means intertwining those two services.
>
>>>  and a user server project has been started to provide distributed
>> authentication for the other services.
>>
>> Excellent.
>>
>>> * Inventory support only has a file-system backend currently, a MySQL
>> database backend that is compatible with the existing
>> OpenSim.Grid.InventoryServer.exe backend is on the short term roadmap.
>>
>> Again, work towards integrating with what's already there - meet in
>> the code, not in the db?
>
> Reiterating for clarity, the distributed asset services project is researching scalability first, and providing what could be a new reference implementation for OpenSim in the future. If the primary goal was to replace the existing services in SVN trunk as fast as possible I would have made many different design decisions.
>
>>
>>> * Login with the new user server is functional. There are still a lot
>> of XML-RPC calls to implement, which will happen right after the
>> distributed authentication is tested.
>>
>> Cool++ - I guess same things goes for the user server as for the asset
>> servers - build on what's there? A lot of peple have come to use the
>> console to administrate their services, the db interfaces should
>> ideally be the same, and the legacy protocols.
>>
>
> Although the initial release of the services focuses on compatibility with the existing services, I'm going to very much lean in favor of building better interfaces instead of maintaining a familiar database layout for people to work on their grid through mysqladmin. I don't use command line tools to edit MediaWiki pages because the existing interface does almost everything I need to do, the REST API works, and the plugin architecture was well thought out.
>
>>> * LDAP login is working. Since the LL viewer cannot easily support
>> direct LDAP logins (it MD5 hashes the password before sending it), a
>> simple web interface was added at /ldap that, upon login, will create
>> a new avatar with your LDAP first name, last name, and hash of your
>> current password. This could be merged into a larger account creation
>> interface.
>>
>> I love it. Have you started thinkking about an authentication plug-in
>> architecture for these schemes?
>>
>> /Stefan
>
> LDAP is an authentication plugin. I chose LDAP specifically because it illustrates why a plugin architecture is needed in the user server (and all grid services). 98% of people have no desire or need for an LDAP login service, but the other 2% of people are happy they can uncomment it in the .ini file.
>
> John
>
> --~--~---------~--~----~------------~-------~--~----~
> this list: http://groups.google.com/group/realxtend
> realXtend home page: http://www.realxtend.org/
> -~----------~----~----~----~------~----~------~--~---
>
>