[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

Wed Jul 7 16:20:53 UTC 2010

*> perhaps, in a future, will be possible to build a *
*> complet planet, a real scale*

Planets, galaxies and solar systems could probably all easily be rendered
using procedural generation.  Whole galaxies could be rendered (on-the-fly)
using procedural generation.

Planet terrain, star systems and a galaxy should/could all be easily created
using procedural generation.

http://en.wikipedia.org/wiki/Procedural_generation

Elite (by David Braben and Ian Bell back in 1984) was developed using
procedural generation.

It's quite common for modern games (such as Soldier of Fortune, or "Just
Cause") to use procedural generation to create large and varied groups of
tropical islands (or other terrain).

Space flight sims often use procedural generation to develop whole planets,
terrain, and even galaxies.

I believe "Elite" (designed back in 1984) planned to contain 248 (approximately
282 trillion) galaxies with 256 solar systems each, and easily could have,
but they were afraid that such a gigantic universe would cause "disbelief"
by players.

So instead of 282 Trillion Galaxies, they settled on just 8 Galaxies.
 That's still plenty of space for you to travel around in.  ;-)

There are a lot of programs that use procedural generation.

SpeedTree is just one example of a "middleware" that is used to generate a
large variety of trees (procedurally).  Yet it's leaf textures are fetched
from regular files, thus creating "high resolution" real foliage.

http://www.speedtree.com/

http://en.wikipedia.org/wiki/SpeedTree

CityScape uses procedural generation and generates cities and terrain just
from GIS data:
http://pixelactive3d.com/Products/CityScape/

CityScape supports GIS data import:
http://pixelactive3d.com/Products/CityScape/?FeatureId=GisData

It can also do procedural Traffic Data generation:
http://pixelactive3d.com/Products/CityScape/?FeatureId=TrafficData

It's all possible... but we'd probably need thousands of developers just to
get OpenSim to do 'everything' that we'd like it to do.  ;-)

<http://en.wikipedia.org/wiki/SpeedTree>
On Wed, Jul 7, 2010 at 10:44 AM, Gustavo Alberto Navarro Bilbao <
alberto.navarro.bilbao at gmail.com> wrote:

> Excuse me if this is not the adecuate place to speak about it,but
> ...perhaps, in a future, will be possible to build a complet planet, a real
> scale, using the Pandromeda <http://www.pandromeda.com/products/> fractal
> concept o a free licence similar one?
>
> Alberto
>
> 2010/7/7 Diva Canto <diva at metaverseink.com>
>
>  There is a ton of interesting engineering challenges on the server side of
>> these systems. We haven't even begun to scratch the surface. As such, it's
>> completely unrealistic to expect things like what you describe to happen any
>> time soon -- at least for interactive, shared, immersive spaces;
>> non-interactive, non-shared, non-immersive (like Google Earth) it's another
>> matter. It's also fairly unrealistic to expect those kinds of optimizations
>> to happen spontaneously and for free. There needs to be a lot of money
>> thrown in for those things to be prototyped and experimented with --
>> research money, for starters; that's the kind of thing that will keep a lot
>> of graduate students busy for the next few years.
>>
>>
>> On 7/6/2010 6:54 PM, Mark Malewski wrote:
>>
>> OpenSim/RealXtend,
>>
>>  Another thought is the issue of "scalability" with the current OpenSim
>> and/or realXtend Architecture.
>>
>>  Running hundreds (or thousands) of IDLE regions require too many system
>> resources (RAM, CPU cycles, etc.)
>>
>>  I believe that the basic server core (OpenSim) should eventually be
>> modified so that ALL regions should have a configurable "timeout" option
>> field (in the regions.ini file).
>>
>>  http://opensimulator.org/wiki/Configuring_Regions
>>
>>  That way an idle "region timeout" can be set (on a region per region
>> basis), and this would allow the OpenSim server to shut down idle regions
>> (based on the "idle timeout" configured in the regions.ini file).
>>
>>  So if a region were completely empty for a particular number of minutes
>> (let's say a region were empty and idle for 5 minutes, then the idle region
>> could be shut down automatically after the set "idle timeout" period for
>> that particular region).
>>
>>  This way idle (and empty) regions are not consuming valuable system
>> resources.
>>
>>  Then as a user attempts to "teleport" to an idle (empty) region, the
>> server would simply start that idle region (on-the-fly) so that the user
>> could teleport into the idle region.
>>
>>  It may create a little bit of a slight "pause" (as an idle region is
>> started) while teleporting, but you normally experience a pause anyways
>> while teleporting, so a few additional two or three seconds shouldn't make
>> that much of a difference when teleporting to an empty or idle region.
>>
>>  A second improvement upon this, would be a "smart idle regions" grid
>> option (configurable at the grid and/or region level).  The "smart idle
>> regions" grid option would take things one step further, and would
>> automatically allow the configuration option to bring nearby idle regions
>> out of "sleep" mode, and start them automatically (if an avatar is within a
>> certain number of meters from a nearby neighboring region).
>>
>>  For example this number could be configurable at the Grid and/or region
>> level to be anything from 1 meter, to 100 meters, or even 256 meters (thus
>> automatically loading the idle region to the North, the idle region to the
>> South, the idle region to the East and the idle region to the West). So that
>> nearby regions could be loaded (as an avatar gets close to the border of
>> another nearby region).
>>
>>  This way if avatars are using high speed vehicles (aircraft, boats,
>> trains, etc.) and they are getting close to a region crossing (to another
>> region that may be idle) that region could automatically be started (if
>> "smart regions" was enabled), thus making the region crossing seamless (and
>> in real-time).  So that flight simulators and ship simulators could be
>> developed (with large global high resolution land region maps of the whole
>> world) that could easily be stored on any single server.  (The idle regions
>> would simply load "on-demand" as needed).
>>
>>  This way OpenSim core could be used for Science, Educational,
>> Flight/Train/Ship simulation or even gaming purposes.
>>
>>  So that way an OpenSim/realXtend server could technically store
>> thousands of regions all on a single server instance (possibly even
>> 50-60GB's of high resolution GIS/terrain/region maps of the whole world).
>>
>>  So that things like high resolution "Global Scenery" could be created
>> (and downloaded) under a "Creative Commons" license for high resolution
>> scenery (stored in Mega regions) for OpenSim.
>>
>>  http://www.global-scenery.org/
>>
>>  I do have a military GIS (Imagery Analysis/Intelligence) background, and
>> don't mind working on things like basic high resolution OpenSim/realXtend
>> region content creation (empty regions based on high resolution GIS/DTED
>> data).  DTED data could be used to create elevation maps, and high
>> resolution imagery could be used for terrain textures.
>>
>>  I believe an OpenSim GIS project would be a very good "basis" for other
>> projects (educational, gaming, city planning, development, architectural
>> development and multi-person Flight/Train/Ship simulation & training
>> development) since terrain/region modules would be available free for
>> download.
>>
>>  The reason "Smart Regions" (idle region timeouts) would be necessary, is
>> so things like modeling rural areas like "Yellowstone" National Park could
>> easily be possible on a small single server (or even a small single
>> workstation running a single OpenSim/osgrid/realXtend server instance),
>> since the majority of those regions would usually remain idle/empty (and
>> with the idle regions shutting down they would consume NO CPU clock cycles
>> or RAM, thus making "idle" regions identical to idle web pages that take up
>> nothing more than hard drive space).
>>
>>  Certain regions (like a lobby, main Plaza, or high-traffic regions)
>> would need to remain active (no idle timeout) and would simply have a idle
>> timeout value of "9999".  (Which would equate to requiring the region to be
>> completely idle and empty for a minimum of 9999 minutes, or about 7 days)
>> before going into a "sleep" or idle "shutdown" mode.
>>
>>  I supposed a timeout value of "0000" could also be used for an
>> "infinite" time (no idle timeout or idle shut down) so that regions that
>> want to remain up (forever) with no idle timeout period or idle shutdown
>> could thus stay running all the time (consuming valuable RAM, CPU and system
>> resources).
>>
>>  But as we begin to look at global region mapping and developing large
>> regions (especially from a GIS perspective) then we need to look at the
>> "flaws" in the current OpenSim server design architecture, and see that it
>> would simply not be possible to create something like a virtual 3D world (on
>> a 1:1 virtual/GIS scale) without having billions of physical servers (and
>> trillions of dollars for physical hardware, let alone the costs of
>> electricity and hardware just to even run a single instance).  It seems to
>> be a major flaw in the basic design architecture to require the "always-on"
>> concept of regions without having a region "idle-timeout" feature.
>>
>>  Wasting RAM and CPU clock cycles on empty/idle regions (without having
>> an "idle timeout" option would seem like a huge waste of energy, and system
>> resources, especially when those valuable system resources could be better
>> spent on active regions with physical avatars that require the additional
>> processing power and RAM that could be freed up by shutting down any idle
>> regions (after a specified "time out" period).
>>
>>  Since the majority of the regions would remain empty and idle on a
>> server (similar to static web pages that are not being accessed or viewed on
>> a web server), it seems extremely foolish to waste such valuable system
>> resources (such as server CPU clock cycles and server RAM, let alone the
>> energy/power) wasted on idle regions that remain empty and idle (but must be
>> loaded as "always on").
>>
>>  Having an "on-the-fly" system that will shut down idle regions (based on
>> their configurable regions.ini "time out" value in the regions.ini file, and
>> automatically load idle regions "on-the-fly" would be much closer to the way
>> the current web is designed, so that servers can hold thousands (possibly
>> even millions) of web pages that consume no system resources (other than
>> physical hard drive space) unless the web pages are being viewed.
>>
>>  By allowing regions to startup/shutdown dynamically (with a "region
>> timeout" feature) thus making servers able to hold infinite amounts of idle
>> regions/land (only limited by the amount of hard drive/disk storage space,
>> which could easily be expanded via RAID or SAN devices).
>>
>>  System Administrators could put "heavy load" regions on separate
>> dedicated servers, while keeping other "idle load" regions all on the same
>> server (for regions that are generally idle and not visited very
>> frequently).
>>
>>  As we evolve and slowly get into simulation systems, such as aircraft,
>> train or ship simulators I can see how having large numbers of "idle"
>> regions would be a requirement for realistic train/ship/aircraft simulation
>> and gaming (as well as GIS and global flight) and how it simply would not
>> even be possible to have millions of regions running all on a single server
>> (due to the current "architectural" problems with OpenSim).
>>
>>  It just wouldn't even be possible to even create a simple flight, train
>> or ship simulator using OpenSim (because OpenSim couldn't even store the
>> image/region scenery maps).
>>
>>  Yet, a small simple "X-Plane" flight simulator can store high resolution
>> "global scenery" data for the WHOLE world on just 6 or 7 double layer DVD's
>> (about 60+ GB of terrain data) all on a single workstation.
>>
>>  http://www.global-scenery.org/
>>
>>  I don't mind starting a Global-Scenery package for OpenSim/realXtend and
>> I do think it would be extremely nice to offer the community high resolution
>> realistic regions (free of charge under a "creative commons" license).
>>
>>  The problem with even starting a large content creation project, is that
>> the current OpenSim architecture is unable to even use it.
>>
>>  It would be a complete waste of time and effort to have realistic global
>> high resolution terrain regions available for OpenSim to use for
>> Flight/Train/Ship (as well as Education & Science) simulation but the whole
>> idea of a large scale GIS/terrain project for OpenSim/realXtend seems
>> somewhat pointless since the current realXtend/OpenSim server architecture
>> is not designed to currently even use/store thousands (or even millions) of
>> idle regions (and the current server architecture of keeping ALL idle
>> regions running is "flawed" by architectural design).
>>
>>  It would seem to make much more sense (from an engineering standpoint)
>> to have the option to shut any idle regions down, and only leave active
>> regions running (or at least give users/operators/grid owners the ability to
>> configure regions & a grid to allow for idle region timeouts).  I believe
>> this "region idle time out" time should be configured in the regions.ini
>> configuration file.  With an option of "9999" for a 7 day idle timeout and a
>> "0000" option to disable the idle timeout for a particular region (for
>> regions that don't ever want to shut down due to an idle timeout).
>>
>>  It would make sense to see OpenSim/realXtend support a "smart region"
>> type of system which servers would/could be configured to shut idle regions
>> down by default (configurable in the regions.ini file for each region) and
>> the server could load idle regions "On-The-Fly" freeing up valuable system
>> resources on a physical server.
>>
>>  By implementing "smart regions" into the OpenSim core, a high resolution
>> global GIS/terrain project would seem like a viable project to start (and I
>> wouldn't mind starting such a project) especially for future
>> science/flight/ship/train simulation.  I could probably find other
>> universities (and sponsors) that would be willing to host mirror copies of
>> the empty raw .OAR global terrain region files.  So that other developers,
>> educators or users could freely download high resolution realistic region
>> terrains (for educational, gaming or simulation purposes).
>>
>>  If OpenSim was able to shut down idle regions on the fly, then it could
>> be used for flight simulation, terrain simulation, or even realistic game
>> simulation (could have large high resolution realistic global terrain maps).
>>
>>  I don't mind starting/doing free "creative commons" (attribution by)
>> based global GIS/terrain content creation work for the OpenSim/realXtend
>> community, but the current problem that I see is that the current OpenSim
>> architecture design would NEVER even be able to support such a feature
>> (since the current server architecture would require thousands, or millions
>> or possibly even billions of OpenSim servers just to handle ONE single copy
>> of "the world" terrain in high-res region data.
>>
>>  Thus making it completely impossible to ever "sail a ship" or "fly an
>> aircraft" around the world on a standalone OpenSim server.
>>
>>  I believe the basics of good simulation programs starts with high
>> resolution image maps (and scenery data).  A flight simulation program is
>> fairly useless without realistic global imagery.
>>
>>  Although the basic default high resolution terrain/region data would
>> probably only consume about 50-60 GB's of storage space (empty terrain
>> regions stored in "mega region" OAR format), it would still be impossible to
>> even store or use the terrain region data on a single OpenSim server (since
>> an OpenSim server would require that all the regions be running at the same
>> time, with no way to load/unload the regions (on-the-fly) as they are
>> accessed or needed.  We would need to have an "idle time-out" feature and
>> also a "load-on-demand" feature to load regions and shut down idle regions
>> based on a region "idle time-out" system (and to load nearby regions, and
>> load regions that users attempt to teleport to).
>>
>>  I apologize for sounding repetitive.
>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.dehttps://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100707/49ee152c/attachment-0001.html>