[Opensim-users] plugin/extension/module system for viewers?

Tue Jun 14 09:52:54 UTC 2011

On Mon, 2011-05-16 at 17:49 -0400, Sean McNamara wrote:

Hi again, sorry for the delayed reply - saw one screenshot now that made
me recall that should comment this:

> >>> just go ahead throw it out there -- Why hasn't there been some sort of
> >>> plugin/module system developed for Viewers? Something along the lines
> > We did this by developing a modular viewer, Naali. It has a module system somewhat similar to Opensim, but in C++.
> then I would *imagine* that (barring a herculean effort) your
> rendering engine is still quite basic compared to SL + Windlight +
> shaders + SSAO + ... ; but from a software engineer's perspective, I
> think I'll much prefer the modular code of RealXtend over the

This is not actually true, because reX has never written a rendering
engine.

Like basically all other features too, it is implemented by a
pre-existing open source library. For graphics, it is Ogre3d. Besides
the core Ogre, there are all sorts of plugins and additions to it, which
we integrate as well .. like this screenshot shows SkyX for skies and
Hydrax for water:

http://dl.dropbox.com/u/16413694/rex/Tundra2SkXHydax.png

There's pretty cool stuff in http://www.ogre3d.org/gallery/

BTW IIRC the guy who originally wrote Ogre was doing some dev for Linden
as well at one point, I think exactly the Windlight integration or so.

The problem with typical SL & Opensim content, and renderers like Ogre,
is that the user generated content with lots of separate prims and
non-optimal texturing etc. needs clever processing (batching and such)
to perform well. This is not a problem for many of the current reX
users, for example our little studio, because we have never made
anything with prims. But certainly is a bummer somewhere like osgrid's
Wright Plaza :p . There are known solutions, dunno if those will be ever
implemented in Naali. I think one unity3d using viewer had done the
similar basics that would be needed with ogre too.

> Too bad that the virtual world community seems to be divided between
> the BSD licensed stuff and the LGPL/GPL licensed stuff.... due to
> fears of inadvertently "polluting" one codebase with code from
> another, there might sometimes restrictions on what a single developer
> can do once they've read / worked on significant parts of the LL
> viewer code. And probably the same applies for folks who work on
> RealXtend code and then want to contribute to the LL viewer or a TPV.

Actually that problem is one way only -- studying reX and Opensimulator
codes doesn't prevent you from working on LL viewer or some TPV based on
it.

But yes this divide has been very real and unfortunate. Hopefully
changes .. the new LGPL adoption on the Linden side is AFAIK BSD
compatible, e.g. Naali could use a lib made from slviewer code.

> Anyway, your post really impressed me and I'll look into your project.
> Do you guys use OpenSim as your server-side? I'm assuming yes... if
> so, at least you might benefit from my contributions to OpenSim, if
> not directly to Naali. :)

Yes, we both use and don't use Opensim actually. Earlier it was the only
option, with the Naali + Taiga (== Opensim + ModRex) combination. But
now there is Tundra SDK too, which is actually a server module added to
the Naali codebase .. and corresponding client side logic and connection
with a new simple protocol.

Because we use and (potentially) dev Opensim as well, and welcome
Opensim devs to work on the viewer side, we've been applying the
contribution policy of Opensim. Which is indeed also very unfortunate
'cause would be good to be able to collaborate with slviewer &/ tpv devs
better.

Thanks for the interest!

> Sean

~Toni

> >>> 2. Nobody has taken the time to implement it
> >> This seems the most likely alternative to me.
> >
> > It did take us a couple of years :)
> >
> >> Nah, it's not impossible. Of course you'd have to use a dynamically
> >> interpreted language for plugins, because the viewer is written in
> >> C++, which compiles to native code. Distributing plugins as native
> >> code is dangerous for two reasons: one, it's impossible to sandbox or
> >
> > Naali supports currently C++ modules, Python add-ons, but for custom functionality, most significantly Javascript which is loaded from the web.
> >
> > The support for interpreted languages was made exactly was this reason -- dealing with native code addons is much more hairy.
> >
> > The C++ module system is still highly useful. For one, by using it for almost all functionality, many of the things are optional. For example the Bullet physics library, which is integrated in one module (yes, works on the viewer side too, which is often *very* useful). Also people sometimes need to do custom stuff in native code for special applications, which can be useful even if that is made in some special module that doesn't need to be distributed.
> >
> >> interpreted scripting language and then designing a stable plugin API
> >> is no walk in the park; just ask the developers who work on Firefox or
> >> Chrome's plugin architecture. The biggest challenge would come in
> >> supporting whatever plugin API you design over the lifetime of the
> >> viewer, or properly versioning the plugin API in a way that doesn't
> >> break too many plugins too often (Firefox does this).
> >
> > We think we are close to freezing the first step of the API now, to declare it 1.0. It is documented in http://www.realxtend.org/doxygen/
> >
> > It still takes us at least some weeks anyway to make the last currently planned breaking changes. so now is a good time to give feedback if something seems wrong.
> >
> > This API is indeed similar to the W3C standardized DOM in web browser. In web browsers, the Javascript scope has things like 'document' and 'window'. In this virtual world browser, instead, there is 'scene' which has all the entities (prims on the opensim side) etc. And renderer which you can use for raycast etc.
> >
> >> The challenging part of this is to engineer a plugins API that is
> >> useful, but not so expansive or general that it would (a) pose a
> >> security risk to the user (stealing their password etc), or (b) allow
> >> the plugin to do crazy things like provide mesh support (yes, I'm
> >> replying to your message DutchGlory), which could never gain any
> >> semblance of performance implemented in an interpreted scripting
> >> language.
> >
> > We believe a good way to advance standardization etc. is to implement stuff, so now there is a working implementation out, even in production use, so we can really evaluate and test all these problems.
> >
> > Security is certainly a huge deal. I did the basics to sandbox our JS API in December, to protect from basic attacks known from the browser land, but it certainly needs more review still. It is now made so that remotely loaded JS code shouldn't be able to get system access in any way.
> >
> > Prim support in Naali is in c++, actually predates really working Python or Javascript support. Possibly is best to remain in c++ too.
> >
> > But some quite central things, like avatars and cameras, are now ported from C++ to Javascript. No performance problems there, it's not like you have thousands of them around in your scene. Object editing has been in Python since the beginning (was the original benchmark for the scripting API: i figured that if doing basic object editing is possible in a script, then also making games is easily possible).
> >
> > Also in the current Tundra 1.0.6 release, the whole UI (which looks like a web browser) is implemented in Javascript now. Naali 0.4 still has a largely c++ written UI (with some Python tools).
> >
> >> If I were to lay out a possible approach for this, I'd say we could
> >> start with QtScript or Lua integration into the viewer. QtScript
> >> provides JavaScript support, and is specifically designed to help you
> >> create an extension language for your app; and, it's written in C++ (a
> >> huge plus considering the rest of the viewer is). The license is
> >
> > This is exactly how we've done it in Naali too.
> >
> > The other guys originally used GTK for first UI tests, just to get debug info of the from-the-scratch written LLUDP implementation (Naali is BSD style apache licensed, no Linden code, respects the Opensim contributions policy). I was experimenting with adding Python support just manually first, to better know what kind of a tool would need for the job.
> >
> > Then we decided to use QT for UI, just to get something for good 2d widgets. Soon I learned about qtscript (and the similar pythonqt) which indeed use the qobject metadata mechanism for automatic exposing of c++ things to scripting, so we went with that and have during the past 1,5years ported all the modules, entity-components etc. to QObjects. So the same API works for c++, py and js. Lua could be easily added in a new c++ module, using QtLua. Then for the security reasons for remotely loaded code, only selected parts are available in the context for those (and new contexts are easy to make for different purposes, e.g. to support LSL style where a script can only see the own object).
> >
> > BTW: the JS system is not (only) a plugin / addon system, but it is also the way to make applications. It is identical to how web works as an app platform: besides getting static data from the servers, the viewer downloads also executable code as a part of the service. This way any sim can implement whatever kind of client side functionality, e.g. UI dialogs, mouse&keyboard handling etc. it wants.
> >
> > Some examples:
> > the current free camera: https://github.com/realXtend/naali/blob/tundra/bin/jsmodules/camera/freelookcamera.js
> > This default camera is instanciated by startup/cameraapplication.js so is always there by default.
> >
> > client side chat UI: https://github.com/realXtend/naali/blob/tundra/bin/scenes/ChatApplication/ChatApplication.js#L237 .
> > This is typically loaded from the server, if the service you connect to has decided to have a chat feature.
> >
> >> say that you should bring this up with the Imprudence developers for
> >> starters. Don't go to Linden Lab because they will say "show us the
> >> code or be quiet", essentially. They have their own feature roadmap
> >
> > You could show Linden our code, I don't know if they've looked at it :)
> >
> > But yah, I've bugged the Imprudence devs about this periodically .. well more late last year, when they said the time was immature as they had a lot to do with basics, but they hoped to get to this later.
> >
> > I guess there's two options for those who want this: a) try to do this on the SLviewer base b) join the Naali effort, where it's already made, to help getting missing SL features (or whatever features you need, that you typically use with Opensim) there. One benefit of Naali is that it's a normal open source project, like Opensim, not controlled by any single company, and has a liberal license compatible with Opensimulator itself. But sure SLviewer has a lot of nice functionality, is quite interesting if we'd get this in e.g. Imprudence too.
> >
> >> Sean
> >
> > ~Toni
> >
> >>> Anyway, just pipe-dreaming here! :)
> >
> > No pipe dreaming, it is already the reality, and quite normal (web browsers have had this for long, so why not vw viewers).
> >
> >>> Cheers,
> >>> ~!CiderJack
> >>> _______________________________________________
> >>> Opensim-users mailing list
> >>> Opensim-users at lists.berlios.de
> >>> https://lists.berlios.de/mailman/listinfo/opensim-users
> >>>
> >> _______________________________________________
> >> Opensim-users mailing list
> >> Opensim-users at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> > _______________________________________________
> > Opensim-users mailing list
> > Opensim-users at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-users
> >
> _______________________________________________
> Opensim-users mailing list
> Opensim-users at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-users