[Opensim-dev] Network stack [Re: Unreal viewer: summoning the coders]

Sun Dec 16 18:05:40 UTC 2018

Doing a new network stack suffers from a structural problem: it takes 
two to tango -- the server and the client. The way our community has 
been divided between OpenSim development and LL viewer development 
doesn't encourage these kinds of things, unfortunately. That's why many 
of the server-side devs tend to experiment with completely new clients 
when wanting to experiment with different network stacks. Given how 
complex the LL-based viewers are, it's just easier.

I still think it's worth evolving the LL viewer to use a new network 
stack, because the LL protocol is an obstacle for corporations. Even if 
we go and develop a new client altogether, the LL viewer isn't going 
away, it's still amazing for many things, and OpenSim must (and will) 
continue to support it. But redoing the network stack of the LL client 
will require engagement from LL viewer developers. And knowing what I 
know about it, it will likely require a fork that will be incompatible 
with Second Life.

(This is unlike OpenSim, which is modular enough to be able to support 
pretty much any client. And we will make it even more modular as new 
clients require it, as Misterblue states.)

On 12/16/2018 9:02 AM, Mister Blue wrote:
> I have also experimented with protocols. I am currently focusing on
> gRPC/Protobuf/HTTP2[1] solutions.I built some code around FlatBuffers[2]
> but I came to the conclusion that the 'control plain' part of VR protocols
> does not use the no-copy feature of FlatBuffers (big things like meshes and
> images go by HTTP or other protocols) and that the FlatBuffer data
> structures leak into a lot of the app because of now the message data is
> stored. The other advantage of Protobuf over Flatbuffer is the wider
> language and implementation support.
> 
> Implementing a complete protocol in a region module requires identifying
> all the correct events from the simulator core. A long term goal (mentioned
> in Diva's presentation) is removing the LL protocol implementation from the
> simulator core. I'm willing to do what I can to move protocols out of the
> core and into modules.
> 
> == mb
> 
> [1]: https://grpc.io/
> [2]: https://google.github.io/flatbuffers/
> 
> On Sat, Dec 15, 2018 at 5:23 PM Dahlia Trimble <dahliatrimble at gmail.com>
> wrote:
> 
>> Hi Diva/all,
>>
>> I'm happy to see some interest in this new viewer project. I have my own
>> viewer projects which I wish to continue focusing on but I'd like to offer
>> to share some notes on some of the approaches I'm using regarding network
>> stacks.
>>
>> First: protocols. I'm using Websockets for scene updates and HTTP for asset
>> transfers. Websockets are message-oriented and provide a convenient means
>> of sending variable length messages between endpoints with little
>> additional support code needed. My primary reasoning for choosing
>> Websockets at this time is to support browser-based viewers. I've also
>> developed a custom UDP protocol but with the success I've had with
>> Websockets I've let that protocol succomb to some bit rot and it's probably
>> broken at this point. I've looked several times at WebRTC but it seems to
>> be primarily a peer-to-peer protocol and it's developers don't seem
>> interested in providing means for non-prowsers to act as peers. I've seen a
>> few attempts in the past to create server peer support libraries but I've
>> not (yet) found any I would consider worthy of building a production
>> protocol on top of. It's been nearly a year since I've researched this and
>> it's possible something has changed but for now Websockets seem to be
>> working quite well.
>>
>> I'm using both a region module approach and a proxy approach. The region
>> module seems to have the better user experience in terms of movement and
>> action lag but the proxy approach allows me to connect to any grid,
>> including SL, and also teleport across the Hypergrid. Both approaches seem
>> to have merit and I tend to keep them in sync development-wise. The region
>> module makes heavy use of EventManager events and for any events I may need
>> which aren't currently available, the module will poll some data structures
>> in scene on a Heartbeat End event. This isn't the approach I'd like and it
>> would be better to have all events available but I thought I'd complete the
>> protocol before submitting any possible pull requests or patches to core.
>> Another issue with the region module approach is it doesn't work well with
>> grid services and managing grid presences. I remember Teravus made a few
>> commits long ago to enable some new login and presence creation methods but
>> I'm not sure they still exist. He also added some Websocket support to the
>> HTTP server library but I believe it's been removed or broken in recent
>> commits. I'm currently using the Fleck library for Websockets. I also have
>> an asset conversion and serving capability in my region module which serves
>> images, animations, meshes, terrain, and the like and this functionality is
>> mirrored in my proxy version. I haven't addressed inventory yet in either
>> version. I have some crude region crossing code in my region module but
>> it's a rather kludgy workaround and I'll likely remove it. This is another
>> area that needs to be worked out with grid services.
>>
>> I know I've discussed these approaches in IRC in the past and if anyone is
>> interested I'm willing to share experiences, ideas, and notes. If I see
>> sufficient commonality between our projects I may be willing to collaborate
>> on some aspects of a network stack.  I should also mention that Mic Bowman
>> did a lot of similar work with accessing Scene state over a network via a
>> custom region module and he might be able to provide some useful insights.
>>
>> I wish all the best of luck with this project! I've had a lot of fun and
>> I've learned a lot trying to interface various game engines to
>> OpenSimulator in the past and while I lack experience with Unreal and
>> cannot reasonably predict anyone's success, I am confident that at minimum
>> it will be a valuable learning experience for all involved.
>>
>> -Dahlia
>>
>> On Sat, Dec 15, 2018 at 8:19 AM Diva Canto <diva at metaverseink.com> wrote:
>>
>>> Parts of my wish list can, of course, be done without a new viewer,
>>> simply by evolving the current Linden-based viewers. The network stack
>>> is an obvious one.
>>>
>>> If the current viewer developers and more server-side developers want to
>>> cooperate on a new network stack that is secure and goes through
>>> firewalls, that's a big +1 from me, independent of any other more
>>> ambitious projects that I'm personally more interested in.
>>>
>>> The way OpenSim is architected, this can (and should) be done outside of
>>> the core code, simply by developing region modules and/or plugins. Once
>>> that stack is a bit more solid, if it works well, we'll certainly
>>> consider bringing those modules to core. Stuff like that, if it works
>>> well, would be very valuable! And really, it *must* be developed as
>>> modules, without touching the core code of the OpenSim framework. That's
>>> how I would do it myself, and that's how it can be accepted for
>> inclusion.
>>>
>>> Here is a presentation I wrote a few years ago, as a means to explain
>>> myself the architecture of OpenSim:
>>> https://www.ics.uci.edu/~lopes/opensim/OpenSim-Sep-15.pdf
>>> Please look starting at page 9 and, in particular, pages 17-19, where I
>>> have pictures of how to write new client stacks.
>>>
>>> I encourage everyone who wants to embark on this to bring the
>>> conversations to opensim-dev. There's a lot of knowledge there about how
>>> to do stuff like that in a way that preserves the architecture of
>> OpenSim.
>>>
>>> The private conversations we have been having are about the more
>>> ambitious vision, not about any particular part of OpenSim, which, at
>>> this point, is sufficiently flexible to support all sorts of
>>> evolutionary extensions. A new network stack is one of those.
>>> +1 on that!
>>>
>>>
>>> On 12/15/2018 6:09 AM, Mike Dickson wrote:
>>>> I'd be willing to look at a UDP replacement + continuing to move things
>>> that should be out of the simulator to CAPS.  I've been working on
>> cutting
>>> over my OpenSim tree to LibreMetaverse to take advantage of the buffer
>>> pools that were added w/Halcyon so it could be done there and ride along
>>> side UDP until things were ready.  The eventing model inside OpenSIm also
>>> needs work to support this IMO, the current mechanism is far too fragile
>>> and error prone.
>>>>
>>>> This feels far more promising if there is working being contemplated
>>> than focusing on the VR elements.
>>>>
>>>> Cinder I'll drop you a note with a few more thoughts but I'd be willing
>>> to work on this. Not in core of course since I'm not in it and therefore
>>> not aware of other "conversations" taking place.   Happy to help
>>> nonetheless and I can fit something like this in with the rest of the
>>> "fixing" I'm doing with OpenSim.
>>>>
>>>> Mike
>>>>
>>>> -----Original Message-----
>>>> From: opensim-dev-bounces at opensimulator.org <
>>> opensim-dev-bounces at opensimulator.org> On Behalf Of Cinder Roxley
>>>> Sent: Saturday, December 15, 2018 8:08 AM
>>>> To: opensim-dev at opensimulator.org
>>>> Subject: Re: [Opensim-dev] Unreal viewer: summoning the coders
>>>>
>>>> On December 15, 2018 at 6:33:13 AM, Ai Austin (ai.ai.austin at gmail.com)
>>>> wrote:
>>>>
>>>> f) Running an app launched via a browser is much more common now and we
>>> already have that with secondlife:// and hop:// protocols - which NEARLY
>>> work as expected and would not need much to fix them working everywhere.
>>>>
>>>> For all intents and purposes, hop:// has been abandoned. While I admit,
>>> it’s much friendlier looking than x-grid-info://, x-grid-info:// complies
>>> with BCP 35 and the URL Living Standard.
>>>>
>>>> Exceptions that still need a little fix are on the Firestorm JIRA (e.g.
>>> grid change on teleport rather grid string for hops staying as home
>> grid).
>>> That, to me, would be more worthwhile than starting a whole new viewer
>>> track.
>>>>
>>>> This is due to the way hypergrid works. When you teleport, you aren’t
>>> leaving your grid, rather, the destination region is linked to your grid.
>>>> However, my solution to this in x-grid-info:// reference source was to
>>> use the OpenSimExtras cap to grok a grid uri from the region the agent is
>>> connected to. This works universally in OpenSim 0.8.x and onward besides
>>> OSGrid where OpenSimExtras doesn’t contain Bluewall’s most recent
>> additions
>>> to it.
>>>>
>>>> As far as Unreal goes, I wouldn’t be caught in the royalty payments
>> mess
>>> that all entails.
>>>>
>>>> However, I would love to collaborate on a replacement for LLUDP if
>>> anyone is interested in that. Something like FlatBuffer (and FlatBuffer
>>> over WebRTC on mobile) would make for a good replacement. It would also
>> be
>>> a step towards some of the silly limitations LLUDP has forced on
>>> OpenSimulator, like the 254 entity limit on terse object updates.
>>>> _______________________________________________
>>>> Opensim-dev mailing list
>>>> Opensim-dev at opensimulator.org
>>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>>>
>>>> _______________________________________________
>>>> Opensim-dev mailing list
>>>> Opensim-dev at opensimulator.org
>>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>>>
>>>
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at opensimulator.org
>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at opensimulator.org
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
> 