[Opensim-dev] A hypergrid architectural and technical summary.
Cristina Videira Lopes
lopes at ics.uci.edu
Wed Nov 19 19:25:33 UTC 2008
Wow, Justin, thank you for the time you took with this, and the
comments. If that's of any worth to you, the quality of your review is
much higher than the reviews I see coming back sometimes on research
paper submissions! :-)
I will incorporate all this feedback into the code and documentation, as
well as any other feedback anyone else sends.
Really impressed,
Crista
Justin Clark-Casey wrote:
> Cool, the patch in http://opensimulator.org/mantis/view.php?id=2640 now builds and passes tests (for what that's worth,
> currently) against r7391 right now.
>
> Having read Cristina's wiki page and the code, I think the hypergrid is a neat architecture. Essentially, a hypergrid
> is a confederation of grids. Each user has their own home grid or standalone, which is where their user profile,
> inventory and assets are stored. When they travel to a different grid via a hyperlink (set up via some funky map and
> region handle manipulation), the new grid is told the address of that user's home asset and inventory services. Thus
>
> 1. If a user rezzes an object from their inventory, the assets for that object are fetched from the home asset service
> and permanently inserted into the foreign asset service. So when that user goes away or logs off, the assets are still
> available to be seen by everybody else.
>
> 2. If a user copies/takes an object from a foreign grid, then the relevant inventory and asset data gets sent to their
> own home services.
>
> Please correct me if I'm wrong, Diva.
>
> As I see it, on a conceptual level, some of the pros are:
>
> * Effectively distributes asset and inventory load over multiple services on multiple grids. This is a really good
> alternative to scaling up a central service to internet scale.
>
> * Allows grids to seamlessly linked to others yet retain control over their own services.
>
> And the cons:
>
> * Assets are liberally spread around grids. I don't think that this is much of an issue in view of the client hole, but
> this is merely my own opinion.
>
> * Regions must be manually linked and appear in a grid's map. One can't just enter an address in a url bar to go to
> another grid, the grid owner must set up the link. As far as I can see this is a limitation of what we have to work
> with in the client. There may be some arguments for restriction (you don't want someone coming to your pg grid from an
> adult grid and depositing god knows what).
>
> * Services need to be exposed to other grids. So malicious grids could possibly fetch and put things they shouldn't. I
> think Diva already has some proposals for dealing with this. This may also be another good argument for controlling who
> can link to you, for now.
>
>
> I've read the code but I don't intend to actually run it right now - I was more concerned about trying to identify any
> nasty architectural problems. As far as I can see there is nothing too significant. However, I do see some
> implementation weaknesses.
>
> 1. Assets associated with worn attachments and appearance are not uploaded to a foreign grid from the home grid on
> teleport in. For other users without cache copies, such avatars will always appear gray and I don't think that any
> attachments would appear. This is fixable.
>
> 2. Prim inventory inspection does not go deep enough. As far as I can see, in the 'asset mapper' you look for
> contained textures when rezzing an item, but not any other contained assets (including contained objects, clothing,
> notecards, scripts, etc.). This will result in an incomplete rez. However, this is fixable. Indeed, I've already
> written all the code required to do this for the OpenSim archiver. If this patch goes in then this should be reused.
>
> 3. Better class and method documentation. From what I've seen, what appears to be there at the moment is probably a
> result of initial copy and paste from existing code. I know this is the pot calling the kettle black to some extent,
> but I feel that proper class and method documentation is something that we ought to be working towards, to help prevent
> parts of the code from turning into the private fiefdom of whoever originally wrote it. Also, logging messages are not
> quite correctly formatted (colons are missing after the [<log name>] section and copyrights are missing or inconsistent.
>
> 4. The code exists in separate Hypergrid packages from all the existing OpenSim packages. This is very understandable
> due to its origin as a forge module. And I think we can take it as is, but these separate packages should probably go
> away over time.
>
> Items 1, 2 and 4 could be fixed up after an initial patch goes in. However, I really would like to see more
> documentation, license fixes and log message uniformity (3). Please could you do this, Cristina?
>
> If this is done then in my opinion the hypergrid facility would be a +1 addition to what is currently in the core.
> However, I'm not an expert on various areas (map, teleport) so would greatly appreciate another pair of eyes if anyone
> can spare them. Therefore, if we are to commit this patch I think we should hold off until Monday of next week to give
> anybody else a chance to look at it.
>
> I'd also quite like to know who intends to support this in the future. It would be a shame if we took it in and then
> nobody was prepared to keep it going (which I think would mean eventual removal).
>
> Sorry for the long post.
>
> Regards,
>
> justincc
>
> Cristina Videira Lopes wrote:
>
>> I added a new patch for r7383. I tested it on a clean slate, and it
>> compiles and behaves correctly. Please let's stick to r7383 for
>> assessment. Turns out 7379 was borked.
>>
>> Thanks!
>>
>> Justin Clark-Casey wrote:
>>
>>> Sure, let's fix this to r7379.
>>>
>>> Realistically, this is probably going to take somewhat longer than a few days. Other core members haven't really had
>>> much of a chance to comment yet. Ideally I think we would take a weekend to give people who are busy with non-OpenSim
>>> jobs during the week to take a look.
>>>
>>> Also, I can only devote a certain fraction of my time to this, so it may take until the end of the week anyway if I'm
>>> working on my own. Of course, any other assistance is very welcome. What I feel we really need is a short summary from
>>> a third party on both the architectural features (pros and cons - there are almost always some cons with everything) and
>>> an assessment of the implementation itself. Among other things, this would help us determine that we're happy to
>>> support it in the future. For instance, is there an expectation that if I make a grid change, I now need to test it in
>>> standalone, grid *and* hypergrid mode? We already see these issues with the bundled MSSQL database support where most
>>> core developers only change MySQL and maybe SQLite (though luckily we now have some excellent people like
>>> StrawberryFride and RuudL to help fix it up).
>>>
>>> I feel that we should also be cautious because we have already had instances where flawed architectural changes have
>>> made it in to OpenSim without any prior analysis or discussion. These cause consternation for a while until they can be
>>> absorbed or pushed aside. I think spending a bit of time upfront on such big changes could save us a lot of time and
>>> pain later on.
>>>
>>>
>>> Cristina Videira Lopes wrote:
>>>
>>>
>>>> OK, I will do that, and thanks for looking into this.
>>>> Can we please fix the assessment of the hypergrid to 7379 for the next
>>>> day or so. Otherwise it's like trying to change the oil on a moving car :-)
>>>> Obviously, if this makes it to core, I will make a patch for whatever
>>>> head that will be, and taking into account whatever feedback is sent;
>>>> it's just the assessment process that I would like to fix, if possible.
>>>>
>>>> Justin Clark-Casey wrote:
>>>>
>>>>
>>>>> I tried to apply the two patches and prebuild.xml in http://opensimulator.org/mantis/view.php?id=2640 to r7379 today,
>>>>> intending to fix up the LLSD name change issues.
>>>>>
>>>>> However, there are various issues.
>>>>>
>>>>> 1. The original hypergrid-1.patch (which contains the bulk of the code) appears to list all diffs twice. So every file
>>>>> ends up containing two copies of everything.
>>>>>
>>>>> 2. The second smaller follow up hypergrid-app.patch does not apply cleanly over hypergrid-1.patch.
>>>>>
>>>>> 3. The prebuild.xml does not list dependences in the correct order. For instance, HGOpenSimNode in
>>>>> OpenSim.Region.Application has a dependency on HyperGrid which is only built after this package. You may have to do
>>>>> some work to resolve circular dependencies. Please do a clean build to make sure everything resolves correctly.
>>>>>
>>>>> Please could you fix these issues (and the LLSD -> OSD changes) and produce a new single patch containing all the code.
>>>>> I recommend simply using svn diff >my.patch from the command line on cygwin/linux/mac if Tortoise SVN is playing up.
>>>>> Then we can evaluate this properly.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Cristina Videira Lopes wrote:
>>>>>
>>>>>
>>>>>
>>>>>> I'm glad this is getting a generally positive reaction. I believe in the
>>>>>> hypergrid, or something like it, strongly enough that I'm going to stick
>>>>>> around and do whatever necessary to see it, or something like it, go
>>>>>> into the box.
>>>>>>
>>>>>> Obviously, I agree with everything said here about revising things
>>>>>> properly and making sure the edges aren't too rough. I will appreciate
>>>>>> one or more of your 'older' ones (he!) looking carefully into it.
>>>>>> Generally, the hypergrid code is cleanly separated from the core code,
>>>>>> but let me tell you what I see as the main ugliness of this contribution:
>>>>>>
>>>>>> The hypergrid touches heavily on Communications and, lightly, on
>>>>>> Environment.Scenes. Communications isn't as well componentized as other
>>>>>> things are; for example, the IClientAPI is a wonderful piece of the
>>>>>> architecture! Unlike that, the construction code for Communications is
>>>>>> hard-coded in OpenSimBase. Because of that, I had to subclass OpenSim,
>>>>>> which is very ugly. If there's a way of making Communications a
>>>>>> component that can be specified in the config file, that would be great!
>>>>>> Not just for the hypergrid, but it would open the door for
>>>>>> experimentation with other alternative interoperability ideas.
>>>>>>
>>>>>> For the extension on Environment.Scenes, again, the Scene-related
>>>>>> classes are being hard-coded in OpenSimBase. If there's a way of
>>>>>> spec-ing that outside, it would be great.
>>>>>>
>>>>>> I know how to quick-fix both of these, but I think this probably needs a
>>>>>> solid fix from those of you who have been making the wonderful job of
>>>>>> componentizing opensim, rather than a quick fix from me.
>>>>>>
>>>>>> Crista
>>>>>>
>>>>>> Stefan Andersson wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As much as I share that sentiment, and despite not having looked at
>>>>>>> the patch, it's usually a good idea to consider splitting large
>>>>>>> patches up into more of babysteps - 'process' over 'product' so to speak.
>>>>>>>
>>>>>>> Ie, is it possible for the hypergrid posse to work with core over time
>>>>>>> to gradually change core into something suitable for them?
>>>>>>>
>>>>>>> Most oftenly to let the code transform in steps leads to the code
>>>>>>> itself 'accumulating wisdom' - which leads to greated flexibility and
>>>>>>> encapsulation. (If it's done with proper love and care in each step) -
>>>>>>> and also lets core + hypergrid communicate over small chunks of code,
>>>>>>> instead of big whoppers. (Historically, big whoppers either rot or
>>>>>>> create havoc, but undesired outcomes)
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Stefan Andersson
>>>>>>> Tribal Media AB
>>>>>>>
>>>>>>> Join the 3d web revolution : http://tribalnet.se/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Date: Tue, 18 Nov 2008 08:08:32 -0500
>>>>>>>> From: sdague at gmail.com
>>>>>>>> To: opensim-dev at lists.berlios.de
>>>>>>>> Subject: Re: [Opensim-dev] Hypergrid patch
>>>>>>>>
>>>>>>>> Justin Clark-Casey wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Dahlia Trimble wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Thanks, that one built properly against 7364, but 7376 (head at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>> the time
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>> I tried) complained about some missing references to LLSD. Seems
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>> a patch
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>> set of this size can go stale quite quickly so hopefully a few of
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>> the
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>> other core devs can chime in real soon and give it a nod... and
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>> then we
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>> can work together to commit it. :)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I think this situation was somewhat unusual with the libOMV update
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> - the names of fairly fundamental classes do not
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> change every day.
>>>>>>>>>
>>>>>>>>> I think with a large patch the submitter has to accept a certain
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> amount of pain in resyncing it to the current trunk -
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> this in itself demonstrates how serious they (and we) are about
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> supporting it. There is a need, I feel, to consider
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> this carefully and not rush in to a decision. This patch requires
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> evaluation on both a raw technical and an
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> architectural level - an evaluation that I hope to start helping
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> with later on today.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I'm +1 for the idea, I'll defer to Justin's judgement on implementation
>>>>>>>> here because I won't have enough time to dig through this of late.
>>>>>>>>
>>>>>>>> I definitely think getting hypergrid, or something like it, into core is
>>>>>>>> a good thing. Letting opensim grids interconnect out of the box is
>>>>>>>> something that has always been on our vision list.
>>>>>>>>
>>>>>>>> -Sean
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sean Dague / Neas Bade
>>>>>>>> sdague at gmail.com
>>>>>>>> http://dague.net
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Opensim-dev mailing list
>>>>>>> Opensim-dev at lists.berlios.de
>>>>>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>> Opensim-dev mailing list
>>>>>> Opensim-dev at lists.berlios.de
>>>>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Opensim-dev mailing list
>>>> Opensim-dev at lists.berlios.de
>>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>>>
>>>>
>>>
>>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20081119/08cd2c5f/attachment-0001.html>
More information about the Opensim-dev
mailing list