[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

[Mark Malewski]

>
> OpenSim/RealXtend,
>
> Another thought is the issue of "scalability" with the current OpenSim
> and/or realXtend Architecture.
>
> Running hundreds (or thousands) of IDLE regions require too many system
> resources (RAM, CPU cycles, etc.)
>
> I believe that the basic server core (OpenSim) should eventually be
> modified so that ALL regions should have a configurable "timeout" option
> field (in the regions.ini file).
>
> http://opensimulator.org/wiki/Configuring_Regions
>
> That way an idle "region timeout" can be set (on a region per region
> basis), and this would allow the OpenSim server to shut down idle regions
> (based on the "idle timeout" configured in the regions.ini file).
>
> So if a region were completely empty for a particular number of minutes
> (let's say a region were empty and idle for 5 minutes, then the idle region
> could be shut down automatically after the set "idle timeout" period for
> that particular region).
>
> This way idle (and empty) regions are not consuming valuable system
> resources.
>
> Then as a user attempts to "teleport" to an idle (empty) region, the server
> would simply start that idle region (on-the-fly) so that the user could
> teleport into the idle region.
>
> It may create a little bit of a slight "pause" (as an idle region is
> started) while teleporting, but you normally experience a pause anyways
> while teleporting, so a few additional two or three seconds shouldn't make
> that much of a difference when teleporting to an empty or idle region.
>
> A second improvement upon this, would be a "smart idle regions" grid option
> (configurable at the grid and/or region level).  The "smart idle regions"
> grid option would take things one step further, and would automatically
> allow the configuration option to bring nearby idle regions out of "sleep"
> mode, and start them automatically (if an avatar is within a certain number
> of meters from a nearby neighboring region).
>
> For example this number could be configurable at the Grid and/or region
> level to be anything from 1 meter, to 100 meters, or even 256 meters (thus
> automatically loading the idle region to the North, the idle region to the
> South, the idle region to the East and the idle region to the West). So that
> nearby regions could be loaded (as an avatar gets close to the border of
> another nearby region).
>
> This way if avatars are using high speed vehicles (aircraft, boats, trains,
> etc.) and they are getting close to a region crossing (to another region
> that may be idle) that region could automatically be started (if "smart
> regions" was enabled), thus making the region crossing seamless (and in
> real-time).  So that flight simulators and ship simulators could be
> developed (with large global high resolution land region maps of the whole
> world) that could easily be stored on any single server.  (The idle regions
> would simply load "on-demand" as needed).
>
> This way OpenSim core could be used for Science, Educational,
> Flight/Train/Ship simulation or even gaming purposes.
>
> So that way an OpenSim/realXtend server could technically store thousands
> of regions all on a single server instance (possibly even 50-60GB's of high
> resolution GIS/terrain/region maps of the whole world).
>
> So that things like high resolution "Global Scenery" could be created (and
> downloaded) under a "Creative Commons" license for high resolution scenery
> (stored in Mega regions) for OpenSim.
>
> http://www.global-scenery.org/
>
> I do have a military GIS (Imagery Analysis/Intelligence) background, and
> don't mind working on things like basic high resolution OpenSim/realXtend
> region content creation (empty regions based on high resolution GIS/DTED
> data).  DTED data could be used to create elevation maps, and high
> resolution imagery could be used for terrain textures.
>
> I believe an OpenSim GIS project would be a very good "basis" for other
> projects (educational, gaming, city planning, development, architectural
> development and multi-person Flight/Train/Ship simulation & training
> development) since terrain/region modules would be available free for
> download.
>
> The reason "Smart Regions" (idle region timeouts) would be necessary, is so
> things like modeling rural areas like "Yellowstone" National Park could
> easily be possible on a small single server (or even a small single
> workstation running a single OpenSim/osgrid/realXtend server instance),
> since the majority of those regions would usually remain idle/empty (and
> with the idle regions shutting down they would consume NO CPU clock cycles
> or RAM, thus making "idle" regions identical to idle web pages that take up
> nothing more than hard drive space).
>
> Certain regions (like a lobby, main Plaza, or high-traffic regions) would
> need to remain active (no idle timeout) and would simply have a idle timeout
> value of "9999".  (Which would equate to requiring the region to be
> completely idle and empty for a minimum of 9999 minutes, or about 7 days)
> before going into a "sleep" or idle "shutdown" mode.
>
> I supposed a timeout value of "0000" could also be used for an "infinite"
> time (no idle timeout or idle shut down) so that regions that want to remain
> up (forever) with no idle timeout period or idle shutdown could thus stay
> running all the time (consuming valuable RAM, CPU and system resources).
>
> But as we begin to look at global region mapping and developing large
> regions (especially from a GIS perspective) then we need to look at the
> "flaws" in the current OpenSim server design architecture, and see that it
> would simply not be possible to create something like a virtual 3D world (on
> a 1:1 virtual/GIS scale) without having billions of physical servers (and
> trillions of dollars for physical hardware, let alone the costs of
> electricity and hardware just to even run a single instance).  It seems to
> be a major flaw in the basic design architecture to require the "always-on"
> concept of regions without having a region "idle-timeout" feature.
>
> Wasting RAM and CPU clock cycles on empty/idle regions (without having an
> "idle timeout" option would seem like a huge waste of energy, and system
> resources, especially when those valuable system resources could be better
> spent on active regions with physical avatars that require the additional
> processing power and RAM that could be freed up by shutting down any idle
> regions (after a specified "time out" period).
>
> Since the majority of the regions would remain empty and idle on a server
> (similar to static web pages that are not being accessed or viewed on a web
> server), it seems extremely foolish to waste such valuable system resources
> (such as server CPU clock cycles and server RAM, let alone the energy/power)
> wasted on idle regions that remain empty and idle (but must be loaded as
> "always on").
>
> Having an "on-the-fly" system that will shut down idle regions (based on
> their configurable regions.ini "time out" value in the regions.ini file, and
> automatically load idle regions "on-the-fly" would be much closer to the way
> the current web is designed, so that servers can hold thousands (possibly
> even millions) of web pages that consume no system resources (other than
> physical hard drive space) unless the web pages are being viewed.
>
> By allowing regions to startup/shutdown dynamically (with a "region
> timeout" feature) thus making servers able to hold infinite amounts of idle
> regions/land (only limited by the amount of hard drive/disk storage space,
> which could easily be expanded via RAID or SAN devices).
>
> System Administrators could put "heavy load" regions on separate dedicated
> servers, while keeping other "idle load" regions all on the same server (for
> regions that are generally idle and not visited very frequently).
>
> As we evolve and slowly get into simulation systems, such as aircraft,
> train or ship simulators I can see how having large numbers of "idle"
> regions would be a requirement for realistic train/ship/aircraft simulation
> and gaming (as well as GIS and global flight) and how it simply would not
> even be possible to have millions of regions running all on a single server
> (due to the current "architectural" problems with OpenSim).
>
> It just wouldn't even be possible to even create a simple flight, train or
> ship simulator using OpenSim (because OpenSim couldn't even store the
> image/region scenery maps).
>
> Yet, a small simple "X-Plane" flight simulator can store high resolution
> "global scenery" data for the WHOLE world on just 6 or 7 double layer DVD's
> (about 60+ GB of terrain data) all on a single workstation.
>
> http://www.global-scenery.org/
>
> I don't mind starting a Global-Scenery package for OpenSim/realXtend and I
> do think it would be extremely nice to offer the community high resolution
> realistic regions (free of charge under a "creative commons" license).
>
> The problem with even starting a large content creation project, is that
> the current OpenSim architecture is unable to even use it.
>
> It would be a complete waste of time and effort to have realistic global
> high resolution terrain regions available for OpenSim to use for
> Flight/Train/Ship (as well as Education & Science) simulation but the whole
> idea of a large scale GIS/terrain project for OpenSim/realXtend seems
> somewhat pointless since the current realXtend/OpenSim server architecture
> is not designed to currently even use/store thousands (or even millions) of
> idle regions (and the current server architecture of keeping ALL idle
> regions running is "flawed" by architectural design).
>
> It would seem to make much more sense (from an engineering standpoint) to
> have the option to shut any idle regions down, and only leave active regions
> running (or at least give users/operators/grid owners the ability to
> configure regions & a grid to allow for idle region timeouts).  I believe
> this "region idle time out" time should be configured in the regions.ini
> configuration file.  With an option of "9999" for a 7 day idle timeout and a
> "0000" option to disable the idle timeout for a particular region (for
> regions that don't ever want to shut down due to an idle timeout).
>
> It would make sense to see OpenSim/realXtend support a "smart region" type
> of system which servers would/could be configured to shut idle regions down
> by default (configurable in the regions.ini file for each region) and the
> server could load idle regions "On-The-Fly" freeing up valuable system
> resources on a physical server.
>
> By implementing "smart regions" into the OpenSim core, a high resolution
> global GIS/terrain project would seem like a viable project to start (and I
> wouldn't mind starting such a project) especially for future
> science/flight/ship/train simulation.  I could probably find other
> universities (and sponsors) that would be willing to host mirror copies of
> the empty raw .OAR global terrain region files.  So that other developers,
> educators or users could freely download high resolution realistic region
> terrains (for educational, gaming or simulation purposes).
>
> If OpenSim was able to shut down idle regions on the fly, then it could be
> used for flight simulation, terrain simulation, or even realistic game
> simulation (could have large high resolution realistic global terrain maps).
>
> I don't mind starting/doing free "creative commons" (attribution by) based
> global GIS/terrain content creation work for the OpenSim/realXtend
> community, but the current problem that I see is that the current OpenSim
> architecture design would NEVER even be able to support such a feature
> (since the current server architecture would require thousands, or millions
> or possibly even billions of OpenSim servers just to handle ONE single copy
> of "the world" terrain in high-res region data.
>
> Thus making it completely impossible to ever "sail a ship" or "fly an
> aircraft" around the world on a standalone OpenSim server.
>
> I believe the basics of good simulation programs starts with high
> resolution image maps (and scenery data).  A flight simulation program is
> fairly useless without realistic global imagery.
>
> Although the basic default high resolution terrain/region data would
> probably only consume about 50-60 GB's of storage space (empty terrain
> regions stored in "mega region" OAR format), it would still be impossible to
> even store or use the terrain region data on a single OpenSim server (since
> an OpenSim server would require that all the regions be running at the same
> time, with no way to load/unload the regions (on-the-fly) as they are
> accessed or needed.  We would need to have an "idle time-out" feature and
> also a "load-on-demand" feature to load regions and shut down idle regions
> based on a region "idle time-out" system (and to load nearby regions, and
> load regions that users attempt to teleport to).
>
> I apologize for sounding repetitive.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100706/f64e3839/attachment-0001.html>