[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

[Michael Cerquoni]

you do realize to do the earth to scale = 510,072,000 km² your literally
talking about millions of regions to represent the entire earth to scale.
This would require literally 100's of 1000's of servers even using your
method you describe.  I personally do not think its possible in its current
form using LL based viewer and opensimulator to do what you propose on any
level. not to mention that if you could fit the entire earth into 60gb, that
would take 130+ hours to download all that data at 1Mbps your talking about
massive amount of bandwidth.  Not that your idea is a bad one, but even the
most empty regions take atleast 60 seconds to start and become usable after
scripts and physics meshing is complete, If your talking about flight
simulation 60 second border crossings every 256 meters is unacceptable.  I
am just not sure this would work as you describe without completely re-doing
opensimulator from the ground up using a different viewer than is currently
available today.  That said if you could show an example of this, that works
successfully I would certainly say its worth doing,  I think though until we
have a viewer that is capable of doing larger than 256m2 regions something
like this is just not possible, id like to hear what others think though, i
have been known to be wrong before ;)

On Tue, Jul 6, 2010 at 6:57 PM, Mark Malewski <mark.malewski at gmail.com>wrote:

> OpenSim/RealXtend,
>>
>> Another thought is the issue of "scalability" with the current OpenSim
>> and/or realXtend Architecture.
>>
>> Running hundreds (or thousands) of IDLE regions require too many system
>> resources (RAM, CPU cycles, etc.)
>>
>> I believe that the basic server core (OpenSim) should eventually be
>> modified so that ALL regions should have a configurable "timeout" option
>> field (in the regions.ini file).
>>
>> http://opensimulator.org/wiki/Configuring_Regions
>>
>> That way an idle "region timeout" can be set (on a region per region
>> basis), and this would allow the OpenSim server to shut down idle regions
>> (based on the "idle timeout" configured in the regions.ini file).
>>
>> So if a region were completely empty for a particular number of minutes
>> (let's say a region were empty and idle for 5 minutes, then the idle region
>> could be shut down automatically after the set "idle timeout" period for
>> that particular region).
>>
>> This way idle (and empty) regions are not consuming valuable system
>> resources.
>>
>> Then as a user attempts to "teleport" to an idle (empty) region, the
>> server would simply start that idle region (on-the-fly) so that the user
>> could teleport into the idle region.
>>
>> It may create a little bit of a slight "pause" (as an idle region is
>> started) while teleporting, but you normally experience a pause anyways
>> while teleporting, so a few additional two or three seconds shouldn't make
>> that much of a difference when teleporting to an empty or idle region.
>>
>> A second improvement upon this, would be a "smart idle regions" grid
>> option (configurable at the grid and/or region level).  The "smart idle
>> regions" grid option would take things one step further, and would
>> automatically allow the configuration option to bring nearby idle regions
>> out of "sleep" mode, and start them automatically (if an avatar is within a
>> certain number of meters from a nearby neighboring region).
>>
>> For example this number could be configurable at the Grid and/or region
>> level to be anything from 1 meter, to 100 meters, or even 256 meters (thus
>> automatically loading the idle region to the North, the idle region to the
>> South, the idle region to the East and the idle region to the West). So that
>> nearby regions could be loaded (as an avatar gets close to the border of
>> another nearby region).
>>
>> This way if avatars are using high speed vehicles (aircraft, boats,
>> trains, etc.) and they are getting close to a region crossing (to another
>> region that may be idle) that region could automatically be started (if
>> "smart regions" was enabled), thus making the region crossing seamless (and
>> in real-time).  So that flight simulators and ship simulators could be
>> developed (with large global high resolution land region maps of the whole
>> world) that could easily be stored on any single server.  (The idle regions
>> would simply load "on-demand" as needed).
>>
>> This way OpenSim core could be used for Science, Educational,
>> Flight/Train/Ship simulation or even gaming purposes.
>>
>> So that way an OpenSim/realXtend server could technically store thousands
>> of regions all on a single server instance (possibly even 50-60GB's of high
>> resolution GIS/terrain/region maps of the whole world).
>>
>> So that things like high resolution "Global Scenery" could be created (and
>> downloaded) under a "Creative Commons" license for high resolution scenery
>> (stored in Mega regions) for OpenSim.
>>
>> http://www.global-scenery.org/
>>
>> I do have a military GIS (Imagery Analysis/Intelligence) background, and
>> don't mind working on things like basic high resolution OpenSim/realXtend
>> region content creation (empty regions based on high resolution GIS/DTED
>> data).  DTED data could be used to create elevation maps, and high
>> resolution imagery could be used for terrain textures.
>>
>> I believe an OpenSim GIS project would be a very good "basis" for other
>> projects (educational, gaming, city planning, development, architectural
>> development and multi-person Flight/Train/Ship simulation & training
>> development) since terrain/region modules would be available free for
>> download.
>>
>> The reason "Smart Regions" (idle region timeouts) would be necessary, is
>> so things like modeling rural areas like "Yellowstone" National Park could
>> easily be possible on a small single server (or even a small single
>> workstation running a single OpenSim/osgrid/realXtend server instance),
>> since the majority of those regions would usually remain idle/empty (and
>> with the idle regions shutting down they would consume NO CPU clock cycles
>> or RAM, thus making "idle" regions identical to idle web pages that take up
>> nothing more than hard drive space).
>>
>> Certain regions (like a lobby, main Plaza, or high-traffic regions) would
>> need to remain active (no idle timeout) and would simply have a idle timeout
>> value of "9999".  (Which would equate to requiring the region to be
>> completely idle and empty for a minimum of 9999 minutes, or about 7 days)
>> before going into a "sleep" or idle "shutdown" mode.
>>
>> I supposed a timeout value of "0000" could also be used for an "infinite"
>> time (no idle timeout or idle shut down) so that regions that want to remain
>> up (forever) with no idle timeout period or idle shutdown could thus stay
>> running all the time (consuming valuable RAM, CPU and system resources).
>>
>> But as we begin to look at global region mapping and developing large
>> regions (especially from a GIS perspective) then we need to look at the
>> "flaws" in the current OpenSim server design architecture, and see that it
>> would simply not be possible to create something like a virtual 3D world (on
>> a 1:1 virtual/GIS scale) without having billions of physical servers (and
>> trillions of dollars for physical hardware, let alone the costs of
>> electricity and hardware just to even run a single instance).  It seems to
>> be a major flaw in the basic design architecture to require the "always-on"
>> concept of regions without having a region "idle-timeout" feature.
>>
>> Wasting RAM and CPU clock cycles on empty/idle regions (without having an
>> "idle timeout" option would seem like a huge waste of energy, and system
>> resources, especially when those valuable system resources could be better
>> spent on active regions with physical avatars that require the additional
>> processing power and RAM that could be freed up by shutting down any idle
>> regions (after a specified "time out" period).
>>
>> Since the majority of the regions would remain empty and idle on a server
>> (similar to static web pages that are not being accessed or viewed on a web
>> server), it seems extremely foolish to waste such valuable system resources
>> (such as server CPU clock cycles and server RAM, let alone the energy/power)
>> wasted on idle regions that remain empty and idle (but must be loaded as
>> "always on").
>>
>> Having an "on-the-fly" system that will shut down idle regions (based on
>> their configurable regions.ini "time out" value in the regions.ini file, and
>> automatically load idle regions "on-the-fly" would be much closer to the way
>> the current web is designed, so that servers can hold thousands (possibly
>> even millions) of web pages that consume no system resources (other than
>> physical hard drive space) unless the web pages are being viewed.
>>
>> By allowing regions to startup/shutdown dynamically (with a "region
>> timeout" feature) thus making servers able to hold infinite amounts of idle
>> regions/land (only limited by the amount of hard drive/disk storage space,
>> which could easily be expanded via RAID or SAN devices).
>>
>> System Administrators could put "heavy load" regions on separate dedicated
>> servers, while keeping other "idle load" regions all on the same server (for
>> regions that are generally idle and not visited very frequently).
>>
>> As we evolve and slowly get into simulation systems, such as aircraft,
>> train or ship simulators I can see how having large numbers of "idle"
>> regions would be a requirement for realistic train/ship/aircraft simulation
>> and gaming (as well as GIS and global flight) and how it simply would not
>> even be possible to have millions of regions running all on a single server
>> (due to the current "architectural" problems with OpenSim).
>>
>> It just wouldn't even be possible to even create a simple flight, train or
>> ship simulator using OpenSim (because OpenSim couldn't even store the
>> image/region scenery maps).
>>
>> Yet, a small simple "X-Plane" flight simulator can store high resolution
>> "global scenery" data for the WHOLE world on just 6 or 7 double layer DVD's
>> (about 60+ GB of terrain data) all on a single workstation.
>>
>> http://www.global-scenery.org/
>>
>> I don't mind starting a Global-Scenery package for OpenSim/realXtend and I
>> do think it would be extremely nice to offer the community high resolution
>> realistic regions (free of charge under a "creative commons" license).
>>
>> The problem with even starting a large content creation project, is that
>> the current OpenSim architecture is unable to even use it.
>>
>> It would be a complete waste of time and effort to have realistic global
>> high resolution terrain regions available for OpenSim to use for
>> Flight/Train/Ship (as well as Education & Science) simulation but the whole
>> idea of a large scale GIS/terrain project for OpenSim/realXtend seems
>> somewhat pointless since the current realXtend/OpenSim server architecture
>> is not designed to currently even use/store thousands (or even millions) of
>> idle regions (and the current server architecture of keeping ALL idle
>> regions running is "flawed" by architectural design).
>>
>> It would seem to make much more sense (from an engineering standpoint) to
>> have the option to shut any idle regions down, and only leave active regions
>> running (or at least give users/operators/grid owners the ability to
>> configure regions & a grid to allow for idle region timeouts).  I believe
>> this "region idle time out" time should be configured in the regions.ini
>> configuration file.  With an option of "9999" for a 7 day idle timeout and a
>> "0000" option to disable the idle timeout for a particular region (for
>> regions that don't ever want to shut down due to an idle timeout).
>>
>> It would make sense to see OpenSim/realXtend support a "smart region" type
>> of system which servers would/could be configured to shut idle regions down
>> by default (configurable in the regions.ini file for each region) and the
>> server could load idle regions "On-The-Fly" freeing up valuable system
>> resources on a physical server.
>>
>> By implementing "smart regions" into the OpenSim core, a high resolution
>> global GIS/terrain project would seem like a viable project to start (and I
>> wouldn't mind starting such a project) especially for future
>> science/flight/ship/train simulation.  I could probably find other
>> universities (and sponsors) that would be willing to host mirror copies of
>> the empty raw .OAR global terrain region files.  So that other developers,
>> educators or users could freely download high resolution realistic region
>> terrains (for educational, gaming or simulation purposes).
>>
>> If OpenSim was able to shut down idle regions on the fly, then it could be
>> used for flight simulation, terrain simulation, or even realistic game
>> simulation (could have large high resolution realistic global terrain maps).
>>
>> I don't mind starting/doing free "creative commons" (attribution by) based
>> global GIS/terrain content creation work for the OpenSim/realXtend
>> community, but the current problem that I see is that the current OpenSim
>> architecture design would NEVER even be able to support such a feature
>> (since the current server architecture would require thousands, or millions
>> or possibly even billions of OpenSim servers just to handle ONE single copy
>> of "the world" terrain in high-res region data.
>>
>> Thus making it completely impossible to ever "sail a ship" or "fly an
>> aircraft" around the world on a standalone OpenSim server.
>>
>> I believe the basics of good simulation programs starts with high
>> resolution image maps (and scenery data).  A flight simulation program is
>> fairly useless without realistic global imagery.
>>
>> Although the basic default high resolution terrain/region data would
>> probably only consume about 50-60 GB's of storage space (empty terrain
>> regions stored in "mega region" OAR format), it would still be impossible to
>> even store or use the terrain region data on a single OpenSim server (since
>> an OpenSim server would require that all the regions be running at the same
>> time, with no way to load/unload the regions (on-the-fly) as they are
>> accessed or needed.  We would need to have an "idle time-out" feature and
>> also a "load-on-demand" feature to load regions and shut down idle regions
>> based on a region "idle time-out" system (and to load nearby regions, and
>> load regions that users attempt to teleport to).
>>
>> I apologize for sounding repetitive.
>>
>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>


-- 
Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100707/7536135d/attachment-0001.html>