[Opensim-dev] Robotics in OpenSim?

[Ben Goertzel]

Thanks much for the email, which is very interesting and very
on-target, although a bit depressing...

Obviously, what I want is to be able to use an AI to control a
simulated robot, and have this simulated robot interact with
human-controlled avatars and various objects in an extensive virtual
world

The question is what is the shortest path to this goal?

I was thinking integrating Player or Gazebo w/ OpenSim might be the
shortest path, but perhaps this was wrong-headed...

I don't know much about OpenCroquet, and I'm not sure if it's more
appropriate or not... why do you think it is?

-- Ben


On Mon, May 5, 2008 at 2:35 PM, dan miller <danbmil99 at yahoo.com> wrote:
> hi Ben --
>
>  We've met at a conference or two.  I'm danx0r on IRC (not there much lately), and I contributed some of the original ODE physics code here.  I also do robotics by day, and I'm interested (some might say obsessed) with the intersection between robotics, simulation, and immersive 3D environments.
>
>  I'd love to talk more about your proposal.  Here are a couple notes on your original post:
>
>
>
>  --- On Thu, 5/1/08, Ben Goertzel <ben at goertzel.org> wrote:
>
>  > I am interested in the possibility of using OpenSim to have
>  > human-controlled avatars
>  > interact with realistically-controlled simulations of
>  > robots.
>
>  Me too.
>
>
>  > Question is whether it's feasible or not.
>
>  I've ruminated on this here and elsewhere.  The short answer, I think, is maybe, but it may not be the best choice depending on your goals.
>
>
>  > My thought is to integrate the Player framework
>  > into OpenSim somehow
>  ...
>
> > best option is just to hook Player up to OpenSim to that in
>  > effect OpenSim replaces Gazebo.
>
>  The problem here is that the physics as it is presently implemented in opensim will not provide many of the features you need to do realistic robots that you get from a 'real' simulator such as Gazebo.  You'll be chasing features in opensim's ODE layer that are already there in p/s/g.  My first-draft opinion is that it would be more efficient (development-wise) to use Gazebo as the physics plug-in, in place of the ODE plugin or POS.  Then you could start exposing robotics-oriented features of Gazebo.
>
>
>  >  The question is whether OpenSim is open to
>  > having the skeleton of a character controlled by some
>  > external process (which would
>  > simulate a servomotor attached to each joint) rather than
>  > controlled by a fixed set of
>  > animations.
>
>  That's a good question.  My understanding is, the answer here is a resounding No (* see end note).  However, scripted objects (controlled server-side) _can_ behave in ways similar to the way human characters do.  The problem is, they live in a somewhat separate world.  Character animation is done client-side, whereas scripted objects are controlled server-side.  The client side knows absolutely nothing about physics, BTW.  Anything apparently physical about character animation was baked in when the animation was created.
>
>
>  > I suppose the situation where Player controls a skeleton
>  > is not that fundamentally different from the one where a skeleton is
>  > controlled by some inverse kinematics equations or what-not.
>
>  Ahh, that's the crux.  As stated previously, the Player does not control a skeleton in that way.  It launches animations that each client logged onto the system faithfully reproduce.  Player actions are then broadcast, and each player's client code animates each player's character accordingly.
>
>
>  > What I'm looking for is an indication of whether
>  > OpenSim experts think this is
>  > a generally feasible direction; and if so, any clues on the
>  > best technical approach.
>  >
>  > I have more experience with Second Life and Multiverse than
>  > OpenSim.
>
>  Have you looked at OpenCroquet?  Not to say OpenSim isn't feasible for this, but the honest answer is that this concept is well outside its design envelope.  In my opinion, there is a massive cognitive disconnect between how the server handles things (scene db, physics, scripts) and how the client handles its responsibilities (basically a 3D animation package with a parametric humanoid skeleton model).
>
>  To give you an example of the impact of this design, note that avatars are modelled in the physics engine as egg-shaped 'capsules'.  There is no attempt at present to indicate to the physics layer that the character even has limbs, much less where they are located in 3-space.  These design decisions are IMSHO an artifact of SL's (and immersive social worlds in general) technical heritage, having been conceived basically as a MMOG for adult socializing, with different visuals and game goals, but similar technical requirements.  MMOG's in turn inherit the basic philosophy of RPG's and FPS gaming.  If you go back far enough, the human-controlled character (avatar) is basically a cursor in a 2D world (2600 anyone?), controlled with NWSE commands.  Interaction with the environment is achieved through high-level requests such as "pick up that chalice", and an inventory of objects is kept.  This of course long predates any attempt at 3D versimilitude.  The
>   upshot is that the basic philosophy is to treat 3D, including physics, as eye candy that sits on top of the fundamental walkabout/pick up objects/cast spells metaphor.
>
>  It's interesting I think to contrast this development branch with simulation-based games such as flight simulators and the more realistic driving games.  By choice and design, they tended to treat the simulation as the core basis for their reality, and all interaction and gameplay was derived from the concept of physical interaction (weapons and direct interaction, ie crashing).
>
>  I'd love to talk more about this at length if you're game.
>
>  Daniel B. Miller
>  danbmil99 at yahoo dot com
>  artiphys.blogspot.com
>
>
>  * note: there's an unfortunate issue you should be aware of.  OpenSim is BSD-licensed, whereas Linden Lab's open-source viewer (client) is released under GPL.  Without rehashing lots of politics, the upshot is that in general, OpenSim developers will have nothing at all to do with the client code, to the point where any substantial involvement in client code development will get you banished from the OpenSim dev community.  What this means in practice is that anyone who wants to maintain the possibility of contributing to OpenSim (including myself) is, by design, unwilling and unable to understand or converse deeply on how the client actually operates.  In my case, I'm quite curious about certain issues such as client-side animation and how 'prims' are meshed for visualization, but I've kept my options open by not downloading and perusing Linden's code.  This complicates the issue of discussing projects such as yours that may require deep understanding
>   of both server and client operation.
>
>



-- 
Ben Goertzel, PhD
CEO, Novamente LLC and Biomind LLC
Director of Research, SIAI
ben at goertzel.org

"If men cease to believe that they will one day become gods then they
will surely become worms."
-- Henry Miller