Hypergrid

The OpenSim Hypergrid

What is the hypergrid?

The hypergrid is an extension to opensim that allows you to link your opensim to other opensims on the internet, and that supports seamless agent transfers among those opensims. It can be used both in standalone mode and in grid mode. The hypergrid is effectively supporting the emergence of a Web of virtual worlds.

The basic idea for the hypergrid is that region/grid administrations can place hyperlinks on their map to hypergrided regions run by others. Once those hyperlinks are established, users interact with those regions in exactly the same way as they interact with local regions. Specifically, users can choose to teleport there. Once the user reaches the region behind the hyperlink, she is automatically interacting with a different virtual world without having to logout from the world where she came from, and while still having access to her inventory.

The hypergrid started as a GForge project, but it is now included in the standard distribution of OpenSim. To run your OpenSim instance in hypergrid mode, please see Installing and Running.

Virtual World Hyperlinks

A Virtual World Hyperlink

We're all familiar with hypertext links on the Web. But what is a virtual world hyperlink?

In the hypergrid model, we consider the 2D map of the virtual world as the equivalent of a web page. As such, a VW hyperlink is simply a region on that map.

The default model of opensim-based virtual worlds already supports this concept of hyperlink, to some extent. When you teleport from one region to another via the map, chances are you are migrating your agent into a different opensim server. This migration is a glorified "agent transfer" that also exists, in rudimentary form, on the web when hypertext links are followed. The default model, however, imposes two very strong constraints on these hyperlinks:

1. The entire map of regions is controlled by a central service known as the grid service, whose job is to provide a uniform view of the world to all of its regions.
2. The only agents that can be transferred are those pertaining to users known to another central service, the user service; if the incoming user is not on that service's database, the agent transfer doesn't go through.

The hypergrid simply removes these two constraints.

First, it allows individual opensim instances to add "neighbors" to their local map, shifting the control of the map down from the grid server to individual opensim instances (although hyperlinks can also be served by grid servers if grid admins so wish). In doing so, the world becomes a lot more interesting and varied. The map that you see in one opensim instance may be completely different from the map that you see after you teleport via an hyperlink. As an opensim administrator, you are free to define what other opensims you want to see on your map.

Second, it allows the transfer of agents pertaining to foreigner users, i.e. users who are registered elsewhere. Instead of assuming one central user service, the hypergrid assumes an arbitrarily large number of such services distributed all over the world. As such, when agents are transferred among hypergrided opensims, a lot more information is passed about the corresponding user. That information includes the collection of servers that the transferring user needs.

Usage Scenarios

The following are some usage scenarios. There isn't a clear separation between these scenarios, there's a large overlap between them. This is also not an exhaustive list. The purpose of these descriptions is to give you some starting ideas for how to use the hypergrid in practice. Please feel free to add other interesting scenarios to this list.

	Personal Worlds This first scenario pertains to standalone opensims. Normally, standalones are completely disconnected from the internet. However, when run in hypergrid mode, standalones become network-able. As such, you can run your own world in your own computer, and link your world to whoever you want. For example, you can link to your friends' hypergrided opensims and to hypergrid gateways in open grids such as OSGrid. The great thing about this scenario is that all of your assets are stored on your computer, and not on somebody else's server. You can back them up using ordinary backend tools. The not so great thing about this scenario is that all of your assets are stored on your computer! If your disk goes berserk, you loose them. (so make sure you make external backups regularly)
	Communities This second scenario is about communities, broadly construed. The idea here is that a group of people come together to support a small community grid, i.e. a common world where shared activities take place. But at the same time, the members of the community maintain their own standalone worlds. The standalones link to the community grid, and the community grid may link back to the individual members' worlds and other places of interest. The members' identities are probably the identities they have on their standalones, and their assets are also probably stored there. The assets present in the community regions, however, are stored on the grid asset server.
	Grid Public Regions Walled-gardens are here to stay, and they serve many useful purposes. There is a hybrid mode for the hypergrid that some walled-garden grid operators may be interested in supporting. In this hybrid mode, most opensim instances on the grid run in normal, wall-garden mode, so no foreign visitors are allowed there - technically it is impossible to reach them. However, a few opensim instances on that grid can run in hypergrid mode, so that foreign visitors are allowed. This way, there is a gateway for grid-local users and arbitrary visitors to meet. This is also a good strategy for attracting new users to the grid, since random users are able to visit those gateway regions without having to sign up for an account upfront. This hybrid mode is very similar to what happens on the web. For example, anyone can visit Facebook's public pages without having to sign up for a Facebook account. However, only Facebook users can go further inside.
	Level Games The normal version of OpenSim enforces a common map for all the regions on a grid. The hypergrid removes that constraint. As such, it becomes easy to design VW games where the world looks different depending of where the player is.

Hypergrid Implementation

Hyperlinks and Agent Transfers

When you establish a link between your opensim and another, a message is sent out to that other opensim requesting information about it; the required information includes the network information of that opensim host, and the coordinates of its first region on its local grid in the form of a region handle. For example, suppose you are linking node X.com:9000, placing it in your local map at 900, 900. That opensim runs one or more regions that likely are not in 900, 900 on their own map; suppose the first region of that opensim is at 1100, 1100. From your point of view, it doesn't matter what those other coordinates are, and you don't need to know -- that's the key to being able to decentralize the "world" as given by a 2D map; you want to place it in your map at 900, 900. The "true" position of that simulator only matters for the LL viewer, when there are teleports between your world and that other opensim. This mapping between coordinate systems is the essence of hyperlinks for opensim; it's one simple but critical thing that the hypergrid implementation does. The mapping happens on the TeleportFinish event; instead of sending the local coordinates to the viewer, the hypergrid teleport wrapper sends the remote coordinates.

When an agent teleports through that hyperlink the following happens. First, before InformRegionOfChildAgent, the local opensim notifies the remote opensim of this foreign user via the "expect_hg_user" method. That message sends along the addresses of all the servers that this user uses, i.e. user, inventory and asset servers. The remote opensim places an entry for that user in its local user profile cache but not in its user database; the foreign user information is non-persistent. After that, the teleport process is exactly the same as the normal teleport process; the only difference is that the region handles are switched between the remote region's hyperlink position on the local grid and its actual position on its grid.

In summary, the two new concepts introduced by the hypergrid are the concept of an hyperlink and the concept of a "local user" vs. "foreign user".

Inventory Access

Inventory access from abroad is done by wrapping the existing scene-inventory interactions with additional code that gets or posts inventory assets from/to the user's asset server. When inventory is accessed, the hypergrid wrapper checks if the user is foreign and, if she is, the wrapper simply brings the necessary assets from the user's asset server to the local asset cache and server; from then on, the wrapper passes the control to the existing inventory access functions. When something is added to inventory, the hypergrid wrapper is notified via an event, and posts the assets to the user's asset server. A cache of the exchanged item identifiers is maintained so that they aren't brought back over and over again.

The result is that hypergrided opensim instances end up interacting with several asset servers, instead of just one. That interaction is implemented in a straightforward manner by instantiating several GridAssetClient objects, instead of just one.

The Hypergrid Classes

Currently, the hypergrid is implemented outside of the OpenSim namespace, so that there is complete separation between what already exists and this new behavior. It has its own namespace, HyperGrid. In it, there are 4 sub-namespaces that follow directly the software architecture of OpenSim, namely:

• OpenSim.Framework:
  • ForeignUserProfileData extends UserProfileData by introducing information about the user's "home", namely the home address, port and remoting port. The user's home is not that user's user service; it's the opensim that the user has defined to be her home. This is necessary for supporting the home jump (Ctrl-Shift-H).
  • HGNetworkServersInfo follows the spirit of NetworkServersInfo, although it neither extends it nor uses it. For now, it's a utility class whose two main functions are to convert domain names of servers to IP addresses, and to uniformly provide the answer to the question bool IsLocalUser(...).

• OpenSim.Region.Framework.Scenes.Hypergrid:
  • HGSceneCommunicationService extends SceneCommunicationService, overriding RequestTeleportToLocation. There are two very small but critical changes to the base method: (a) on the TeleportFinish event, we switch the region handles when the destination region is an hyperlink; (b) the connections at the end are always closed for hyperlink TPs.
  • HGScene extends Scene, overriding TeleportClientHome(...). The only change to the base method is to stay away from the user server, for now, because the user service is still not completely wrapped up for foreign users. Once the user service is properly wrapped up, this class will become unnecessary.
  • HGScene.Inventory is a partial class of HGScene, just like what happens in the OpenSim framework. This part of HGScene overrides some inventory-scene interaction methods, so that assets are fetched/posted from/to the user's asset server. Once that extra fetching/posting is done, these methods simply pass the ball to the base methods.
  • HGAssetMapper: this is a new class specific to the hypergrid that manages the fetching and posting of assets between foreign regions where the user is and the user's asset server.

• OpenSim.Region.Communications.HyperGrid is a mashup of OpenSim.Region.Communications.*. This is the place where most of the hypergrid extension lies. One of the reasons for this is that the hypergrid communications part is doing one additional thing: it is making standalones network-able.
  • HGCommunicationsStandalone extends CommuniationsLocal. Just as its base, it is a hub for the several network services available in standalone mode. The main difference is that those services are extensions of what's in OpenSim.
  • HGCommunicationsGridMode extends CommunicationsManager directly. Again, it's a hub for the network services available in grid mode, those services being extensions of OpenSim.
  • The cluster HGGridServices (superclass), HGGridServicesStandalone and HGGridServicesGridMode (subclasses) implements the OpenSim interfaces IGridServices and IInterRegionCommunications. The 2 subclasses are wrappers for LocalBackEndServices and OGS1GridServices, respectively. There is one common pattern throughout these classes: check if the region to talk to is an hyperlink; if it's not, simply delegate the work to LocalBackEndServices/OGS1GridServices; if it is, push the work to the base class HGGridServices. HGGridServices, in turn, does the management of hyperlink regions, and defines two additional pieces of inter-region protocol:
    • region_uuid: for linking regions
    • expect_hg_user: similar to the existing expect_user interface, but with a lot more information about the user being passed around, namely all the user's servers (inventory, asset, user, home, etc.)
  • HGInventoryService extends LocalInventoryService and implements ISecureInventoryService. This class is the most obvious mashup of the pack, mixing local service access for standalone users and remote inventory access for when users are out and about. Right now, there is a fair amount of selective copy-and-paste, to stay away from the ugliness coming from OGS1InventoryService and OGS1SecureInventoryService. HGInventoryService is always a ISecureInventoryService. Its methods all follow the same pattern: check if the user is a local standalone user; if it is, pass the work to the base method (in LocalInventoryService); if it's not perform secure remote access.
  • HGUserServices wraps OSG1UserServices, but it's not functional yet.

• OpenSim.Region.CoreModules.HyperGrid is a collection of 3 region modules:
  • HGWorldMapModule extends WorldMapModule. It reuses almost everything from the base class. The only small change is in RequestMapBlocks, where it tries to send Offline mapblocks to the client.
  • HGStandaloneInventoryService and HGStandaloneAssetService do what their names say. They are region modules that allow access to inventory and assets for standalones, when the standalone user is out and about. In spirit, there is a lot in common between these modules and the REST inventory/asset plugin. Unfortunately, that plugin could not be used because it defines a completely different interface than that used by existing inventory and asset servers, and the access for the hypergrid must use a consistent interface.

Installing and Running Hypergrid

Installing

1. Checkout OpenSim, prebuild and build as normal.
2. • If you're running your opensim in grid mode with the UGAIM servers on other machines, you're done. If you're running in standalone and you want it to be network-able, or if you have your grid on loopback (127.0.0.1) change all the [Network] server addresses to "http://<external_host_name>:<http_port>". See below.
3. Run opensim like this: [mono] OpenSim.exe -hypergrid=true. To make sure the hypergrid is running type this on your console: link-region. If you don't hear anything back, the hypergrid is not properly installed.

[mono] OpenSim.exe -hypergrid=true

Here is an example of the Network settings for a standalone:

[Network]
http_listener_port = 9300
remoting_listener_port = 9895

grid_server_url = http://example.com:9300
grid_send_key = null
grid_recv_key = null

user_server_url = http://example.com:9300
user_send_key = null
user_recv_key = null

asset_server_url = http://example.com:9300
inventory_server_url = http://example.com:9300
messaging_server_url = http://example.com:9300

Here is an example of the Network settings for a grided opensim:

[Network]
http_listener_port = 9300
remoting_listener_port = 9895

grid_server_url = http://example.com:8001
grid_send_key = null
grid_recv_key = null

user_server_url = http://example.com:8002
user_send_key = null
user_recv_key = null

asset_server_url = http://example.com:8003
inventory_server_url = http://example.com:8004
; Port 8005 reserved
messaging_server_url = http://example.com:8006

Important Note

Make sure you have a 'home' set. If your home region doesn't exist, the hyperlink TPs may not work. To set your home, go to one of your local regions and "Set Home" from the viewer.

Linking regions

Method 1

On the console, type for example:

link-region <Xloc> <Yloc> osl2.nac.uci.edu:9006

• Use Xloc and Yloc that make sense to your world, i.e. close to your regions, but not adjacent.
• replace osl2.nac.uci.edu and 9006 with the domain name / ip address and the http_listener_port of the simulator where the region is running you want to link to

You can link to a specific region within an instance, by using the name of the region at the end, for example:

link-region 997 997 osl2.nac.uci.edu:9006:UCI Welcome

Method 2

There is also some initial support for reading the links from a xml file.

Use the console command: link-region <URI> [<excludeList>]

The uri can be either the path of a local xml file or a xml document on a http server.

The format of the xml file is:

<Nini>
 <Section Name="Region1"> 
   <Key Name="xloc" Value="1002"/>
   <Key Name="yloc" Value="1006" />
   <Key Name="externalPort" Value="9006" />
   <Key Name="externalHostName" Value="osl2.nac.uci.edu" />
   <Key Name="localName" Value="OSGrid-Gateway" />
   <Key Name="real-xloc" Value="10222"/> //optional field that gives the region's real location on its home grid
   <Key Name="real-yloc" Value="10265" /> //optional field that gives the region's real location on its home grid
 </Section>
 <Section Name="Region2"> 
   ...
 </Section>
 ...
</Nini>

[Note] The section names can be anything you want, but they all should be different and have no spaces in the name.

ExcludeList:

The exclude list is a single string paramater with the format: excludeList:<SectionName>[;<SectionName>]

This means that while reading from the xml file any sections that are listed in the excludeList will be ignored and no HyperGrid link created for them.

This could allow, link lists to be created on a webserver that everyone could add their own regions to, and then they just make sure they add their own section name(s) to the exclude list on their own region(s).

So for example, someone might create a editable online list for the up coming OpenSimulator's 2nd birthday. Which might look something like:

<Nini>
 <Section Name="OSGrid-Party"> 
   <Key Name="xloc" Value="1002"/>
   <Key Name="yloc" Value="1006" />
   <Key Name="externalPort" Value="9006" />
   <Key Name="externalHostName" Value="osl2.nac.uci.edu" />
   <Key Name="localName" Value="OSGrid-Gateway" />
 </Section>
 <Section Name="UCIGrid-Party"> 
   ...
 </Section>
</Nini>

I could then add my own region to the list with the section name "MW-Party". Then when I startup that region that I want to be part of this hypergrid, I use the command: "link-region <URI of xml file> excludeList:MW-Party"

This is so that my region doesn't try to create a hyper link to itself.

Method 3 (dynamic)

Starting in r8193, if you're in an HG-enabled region, you'll be able to dynamically link sims, and TP there, in any one of these ways (and probably more). All you need to know is the target addresse, e.g. from the list below.

1) Type for example secondlife://ucigrid04.nacs.uci.edu:9007/ in the 
chat box, pull up the chat history and click on that link

2) Pull up the map and search for things like 
ucigrid04.nacs.uci.edu:9007

3) Using the embedded browser visit pages that have links like 
secondlife://ucigrid04.nacs.uci.edu:9007/ (there's one up at
http://www.ics.uci.edu/~lopes/hypergrid/test.html)

Again, you can link to a specific region within an instance by adding the name of that region at the end, like this: secondlife://ucigrid04.nacs.uci.edu:9007:Gateway 7000/

Important Note

Due to a viewer bug, you can only TP between regions that are no more than 4096 cells apart in any dimension. What this means in practice is that if you want to link to OSGrid, you must have your own regions reachable from the (10,000; 10,000) point on the map, which is where OSGrid is centered. Place your regions somewhere in the 8,000s or the 12,000s.

Public Hypergrid Nodes

Please see Public Hypergrid Nodes.

Hypergrid Lists

Please see Hypergrid Lists.

Manuals and Tutorials

• Unofficial Hypernauta's Basic Manual: If you are a real beginner it can be for you. Talking about "personal" worlds created using domestic computers. LINK

Development Meetings

Hypergrid Meetings