[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

[Mark Malewski]

OpenSim/RealXtend,

Another thought is the issue of "scalability" with the current OpenSim
and/or realXtend Architecture.

Running hundreds (or thousands) of IDLE regions require too many system
resources (RAM, CPU cycles, etc.)

I believe that the basic server core (OpenSim) should eventually be modified
so that ALL regions should have a configurable "timeout" option field (in
the regions.ini file).

http://opensimulator.org/wiki/Configuring_Regions

That way an idle "region timeout" can be set (on a region per region basis),
and this would allow the OpenSim server to shut down idle regions (based on
the "idle timeout" configured in the regions.ini file).

So if a region were completely empty for a particular number of minutes
(let's say a region were empty and idle for 5 minutes, then the idle region
could be shut down automatically after the set "idle timeout" period for
that particular region).

This way idle (and empty) regions are not consuming valuable system
resources.

Then as a user attempts to "teleport" to an idle (empty) region, the server
would simply start that idle region (on-the-fly) so that the user could
teleport into the idle region.

It may create a little bit of a slight "pause" (as an idle region is
started) while teleporting, but you normally experience a pause anyways
while teleporting, so a few additional two or three seconds shouldn't make
that much of a difference when teleporting to an empty or idle region.

A second improvement upon this, would be a "smart idle regions" grid option
(configurable at the grid and/or region level).  The "smart idle regions"
grid option would take things one step further, and would automatically
allow the configuration option to bring nearby idle regions out of "sleep"
mode, and start them automatically (if an avatar is within a certain number
of meters from a nearby neighboring region).

For example this number could be configurable at the Grid and/or region
level to be anything from 1 meter, to 100 meters, or even 256 meters (thus
automatically loading the idle region to the North, the idle region to the
South, the idle region to the East and the idle region to the West). So that
nearby regions could be loaded (as an avatar gets close to the border of
another nearby region).

This way if avatars are using high speed vehicles (aircraft, boats, trains,
etc.) and they are getting close to a region crossing (to another region
that may be idle) that region could automatically be started (if "smart
regions" was enabled), thus making the region crossing seamless (and in
real-time).  So that flight simulators and ship simulators could be
developed (with large global high resolution land region maps of the whole
world) that could easily be stored on any single server.  (The idle regions
would simply load "on-demand" as needed).

This way OpenSim core could be used for Science, Educational,
Flight/Train/Ship simulation or even gaming purposes.

So that way an OpenSim/realXtend server could technically store thousands of
regions all on a single server instance (possibly even 50-60GB's of high
resolution GIS/terrain/region maps of the whole world).

So that things like high resolution "Global Scenery" could be created (and
downloaded) under a "Creative Commons" license for high resolution scenery
(stored in Mega regions) for OpenSim.

http://www.global-scenery.org/

I do have a military GIS (Imagery Analysis/Intelligence) background, and
don't mind working on things like basic high resolution OpenSim/realXtend
region content creation (empty regions based on high resolution GIS/DTED
data).  DTED data could be used to create elevation maps, and high
resolution imagery could be used for terrain textures.

I believe an OpenSim GIS project would be a very good "basis" for other
projects (educational, gaming, city planning, development, architectural
development and multi-person Flight/Train/Ship simulation & training
development) since terrain/region modules would be available free for
download.

The reason "Smart Regions" (idle region timeouts) would be necessary, is so
things like modeling rural areas like "Yellowstone" National Park could
easily be possible on a small single server (or even a small single
workstation running a single OpenSim/osgrid/realXtend server instance),
since the majority of those regions would usually remain idle/empty (and
with the idle regions shutting down they would consume NO CPU clock cycles
or RAM, thus making "idle" regions identical to idle web pages that take up
nothing more than hard drive space).

Certain regions (like a lobby, main Plaza, or high-traffic regions) would
need to remain active (no idle timeout) and would simply have a idle timeout
value of "9999".  (Which would equate to requiring the region to be
completely idle and empty for a minimum of 9999 minutes, or about 7 days)
before going into a "sleep" or idle "shutdown" mode.

I supposed a timeout value of "0000" could also be used for an "infinite"
time (no idle timeout or idle shut down) so that regions that want to remain
up (forever) with no idle timeout period or idle shutdown could thus stay
running all the time (consuming valuable RAM, CPU and system resources).

But as we begin to look at global region mapping and developing large
regions (especially from a GIS perspective) then we need to look at the
"flaws" in the current OpenSim server design architecture, and see that it
would simply not be possible to create something like a virtual 3D world (on
a 1:1 virtual/GIS scale) without having billions of physical servers (and
trillions of dollars for physical hardware, let alone the costs of
electricity and hardware just to even run a single instance).  It seems to
be a major flaw in the basic design architecture to require the "always-on"
concept of regions without having a region "idle-timeout" feature.

Wasting RAM and CPU clock cycles on empty/idle regions (without having an
"idle timeout" option would seem like a huge waste of energy, and system
resources, especially when those valuable system resources could be better
spent on active regions with physical avatars that require the additional
processing power and RAM that could be freed up by shutting down any idle
regions (after a specified "time out" period).

Since the majority of the regions would remain empty and idle on a server
(similar to static web pages that are not being accessed or viewed on a web
server), it seems extremely foolish to waste such valuable system resources
(such as server CPU clock cycles and server RAM, let alone the energy/power)
wasted on idle regions that remain empty and idle (but must be loaded as
"always on").

Having an "on-the-fly" system that will shut down idle regions (based on
their configurable regions.ini "time out" value in the regions.ini file, and
automatically load idle regions "on-the-fly" would be much closer to the way
the current web is designed, so that servers can hold thousands (possibly
even millions) of web pages that consume no system resources (other than
physical hard drive space) unless the web pages are being viewed.

By allowing regions to startup/shutdown dynamically (with a "region timeout"
feature) thus making servers able to hold infinite amounts of idle
regions/land (only limited by the amount of hard drive/disk storage space,
which could easily be expanded via RAID or SAN devices).

System Administrators could put "heavy load" regions on separate dedicated
servers, while keeping other "idle load" regions all on the same server (for
regions that are generally idle and not visited very frequently).

As we evolve and slowly get into simulation systems, such as aircraft, train
or ship simulators I can see how having large numbers of "idle" regions
would be a requirement for realistic train/ship/aircraft simulation and
gaming (as well as GIS and global flight) and how it simply would not even
be possible to have millions of regions running all on a single server (due
to the current "architectural" problems with OpenSim).

It just wouldn't even be possible to even create a simple flight, train or
ship simulator using OpenSim (because OpenSim couldn't even store the
image/region scenery maps).

Yet, a small simple "X-Plane" flight simulator can store high resolution
"global scenery" data for the WHOLE world on just 6 or 7 double layer DVD's
(about 60+ GB of terrain data) all on a single workstation.

http://www.global-scenery.org/

I don't mind starting a Global-Scenery package for OpenSim/realXtend and I
do think it would be extremely nice to offer the community high resolution
realistic regions (free of charge under a "creative commons" license).

The problem with even starting a large content creation project, is that the
current OpenSim architecture is unable to even use it.

It would be a complete waste of time and effort to have realistic global
high resolution terrain regions available for OpenSim to use for
Flight/Train/Ship (as well as Education & Science) simulation but the whole
idea of a large scale GIS/terrain project for OpenSim/realXtend seems
somewhat pointless since the current realXtend/OpenSim server architecture
is not designed to currently even use/store thousands (or even millions) of
idle regions (and the current server architecture of keeping ALL idle
regions running is "flawed" by architectural design).

It would seem to make much more sense (from an engineering standpoint) to
have the option to shut any idle regions down, and only leave active regions
running (or at least give users/operators/grid owners the ability to
configure regions & a grid to allow for idle region timeouts).  I believe
this "region idle time out" time should be configured in the regions.ini
configuration file.  With an option of "9999" for a 7 day idle timeout and a
"0000" option to disable the idle timeout for a particular region (for
regions that don't ever want to shut down due to an idle timeout).

It would make sense to see OpenSim/realXtend support a "smart region" type
of system which servers would/could be configured to shut idle regions down
by default (configurable in the regions.ini file for each region) and the
server could load idle regions "On-The-Fly" freeing up valuable system
resources on a physical server.

By implementing "smart regions" into the OpenSim core, a high resolution
global GIS/terrain project would seem like a viable project to start (and I
wouldn't mind starting such a project) especially for future
science/flight/ship/train simulation.  I could probably find other
universities (and sponsors) that would be willing to host mirror copies of
the empty raw .OAR global terrain region files.  So that other developers,
educators or users could freely download high resolution realistic region
terrains (for educational, gaming or simulation purposes).

If OpenSim was able to shut down idle regions on the fly, then it could be
used for flight simulation, terrain simulation, or even realistic game
simulation (could have large high resolution realistic global terrain maps).

I don't mind starting/doing free "creative commons" (attribution by) based
global GIS/terrain content creation work for the OpenSim/realXtend
community, but the current problem that I see is that the current OpenSim
architecture design would NEVER even be able to support such a feature
(since the current server architecture would require thousands, or millions
or possibly even billions of OpenSim servers just to handle ONE single copy
of "the world" terrain in high-res region data.

Thus making it completely impossible to ever "sail a ship" or "fly an
aircraft" around the world on a standalone OpenSim server.

I believe the basics of good simulation programs starts with high resolution
image maps (and scenery data).  A flight simulation program is fairly
useless without realistic global imagery.

Although the basic default high resolution terrain/region data would
probably only consume about 50-60 GB's of storage space (empty terrain
regions stored in "mega region" OAR format), it would still be impossible to
even store or use the terrain region data on a single OpenSim server (since
an OpenSim server would require that all the regions be running at the same
time, with no way to load/unload the regions (on-the-fly) as they are
accessed or needed.  We would need to have an "idle time-out" feature and
also a "load-on-demand" feature to load regions and shut down idle regions
based on a region "idle time-out" system (and to load nearby regions, and
load regions that users attempt to teleport to).

I apologize for sounding repetitive.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100706/df6c41ae/attachment-0001.html>