[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

[Jeroen van Veen]

Using some form of GIS with opensim isnt too difficult. Creating regions 
anywhere on a worldmap can be done with osmaps. Basically it takes a gridsize 
of openstreetmap on zoom 17 or 18, and places regions on top of osm-tiles. 
Gridlocations can then be transformed back into lon/lat. Only problem is that 
the viewers dont support teleports to regions that are positioned on positions 
like 130000,130000. Another problem is that the inworld mapviewer works with a 
different mappingsystem. Snowglobe and sl viewer 2 dont have this problem, but 
it's still clumpsy because the sl map is mirrored on it's y axis. I guess the 
first step is to remove that weird teleport restriction, and implement the new 
snowglobe mapper in a viewer like imprudence. Robust already has mapservice-
url support.

kind regards,

Jeroen

On Wednesday 07 July 2010 18:20:53 Mark Malewski wrote:
> *> perhaps, in a future, will be possible to build a *
> *> complet planet, a real scale*
> 
> Planets, galaxies and solar systems could probably all easily be rendered
> using procedural generation.  Whole galaxies could be rendered (on-the-fly)
> using procedural generation.
> 
> Planet terrain, star systems and a galaxy should/could all be easily
> created using procedural generation.
> 
> http://en.wikipedia.org/wiki/Procedural_generation
> 
> Elite (by David Braben and Ian Bell back in 1984) was developed using
> procedural generation.
> 
> It's quite common for modern games (such as Soldier of Fortune, or "Just
> Cause") to use procedural generation to create large and varied groups of
> tropical islands (or other terrain).
> 
> Space flight sims often use procedural generation to develop whole planets,
> terrain, and even galaxies.
> 
> I believe "Elite" (designed back in 1984) planned to contain 248
> (approximately 282 trillion) galaxies with 256 solar systems each, and
> easily could have, but they were afraid that such a gigantic universe
> would cause "disbelief" by players.
> 
> So instead of 282 Trillion Galaxies, they settled on just 8 Galaxies.
>  That's still plenty of space for you to travel around in.  ;-)
> 
> There are a lot of programs that use procedural generation.
> 
> SpeedTree is just one example of a "middleware" that is used to generate a
> large variety of trees (procedurally).  Yet it's leaf textures are fetched
> from regular files, thus creating "high resolution" real foliage.
> 
> http://www.speedtree.com/
> 
> http://en.wikipedia.org/wiki/SpeedTree
> 
> CityScape uses procedural generation and generates cities and terrain just
> from GIS data:
> http://pixelactive3d.com/Products/CityScape/
> 
> CityScape supports GIS data import:
> http://pixelactive3d.com/Products/CityScape/?FeatureId=GisData
> 
> It can also do procedural Traffic Data generation:
> http://pixelactive3d.com/Products/CityScape/?FeatureId=TrafficData
> 
> It's all possible... but we'd probably need thousands of developers just to
> get OpenSim to do 'everything' that we'd like it to do.  ;-)
> 
> 
> <http://en.wikipedia.org/wiki/SpeedTree>
> On Wed, Jul 7, 2010 at 10:44 AM, Gustavo Alberto Navarro Bilbao <
> 
> alberto.navarro.bilbao at gmail.com> wrote:
> > Excuse me if this is not the adecuate place to speak about it,but
> > ...perhaps, in a future, will be possible to build a complet planet, a
> > real scale, using the Pandromeda <http://www.pandromeda.com/products/>
> > fractal concept o a free licence similar one?
> > 
> > Alberto
> > 
> > 2010/7/7 Diva Canto <diva at metaverseink.com>
> > 
> >  There is a ton of interesting engineering challenges on the server side
> >  of
> >  
> >> these systems. We haven't even begun to scratch the surface. As such,
> >> it's completely unrealistic to expect things like what you describe to
> >> happen any time soon -- at least for interactive, shared, immersive
> >> spaces; non-interactive, non-shared, non-immersive (like Google Earth)
> >> it's another matter. It's also fairly unrealistic to expect those kinds
> >> of optimizations to happen spontaneously and for free. There needs to
> >> be a lot of money thrown in for those things to be prototyped and
> >> experimented with -- research money, for starters; that's the kind of
> >> thing that will keep a lot of graduate students busy for the next few
> >> years.
> >> 
> >> 
> >> On 7/6/2010 6:54 PM, Mark Malewski wrote:
> >> 
> >> OpenSim/RealXtend,
> >> 
> >>  Another thought is the issue of "scalability" with the current OpenSim
> >> 
> >> and/or realXtend Architecture.
> >> 
> >>  Running hundreds (or thousands) of IDLE regions require too many system
> >> 
> >> resources (RAM, CPU cycles, etc.)
> >> 
> >>  I believe that the basic server core (OpenSim) should eventually be
> >> 
> >> modified so that ALL regions should have a configurable "timeout" option
> >> field (in the regions.ini file).
> >> 
> >>  http://opensimulator.org/wiki/Configuring_Regions
> >>  
> >>  That way an idle "region timeout" can be set (on a region per region
> >> 
> >> basis), and this would allow the OpenSim server to shut down idle
> >> regions (based on the "idle timeout" configured in the regions.ini
> >> file).
> >> 
> >>  So if a region were completely empty for a particular number of minutes
> >> 
> >> (let's say a region were empty and idle for 5 minutes, then the idle
> >> region could be shut down automatically after the set "idle timeout"
> >> period for that particular region).
> >> 
> >>  This way idle (and empty) regions are not consuming valuable system
> >> 
> >> resources.
> >> 
> >>  Then as a user attempts to "teleport" to an idle (empty) region, the
> >> 
> >> server would simply start that idle region (on-the-fly) so that the user
> >> could teleport into the idle region.
> >> 
> >>  It may create a little bit of a slight "pause" (as an idle region is
> >> 
> >> started) while teleporting, but you normally experience a pause anyways
> >> while teleporting, so a few additional two or three seconds shouldn't
> >> make that much of a difference when teleporting to an empty or idle
> >> region.
> >> 
> >>  A second improvement upon this, would be a "smart idle regions" grid
> >> 
> >> option (configurable at the grid and/or region level).  The "smart idle
> >> regions" grid option would take things one step further, and would
> >> automatically allow the configuration option to bring nearby idle
> >> regions out of "sleep" mode, and start them automatically (if an avatar
> >> is within a certain number of meters from a nearby neighboring region).
> >> 
> >>  For example this number could be configurable at the Grid and/or region
> >> 
> >> level to be anything from 1 meter, to 100 meters, or even 256 meters
> >> (thus automatically loading the idle region to the North, the idle
> >> region to the South, the idle region to the East and the idle region to
> >> the West). So that nearby regions could be loaded (as an avatar gets
> >> close to the border of another nearby region).
> >> 
> >>  This way if avatars are using high speed vehicles (aircraft, boats,
> >> 
> >> trains, etc.) and they are getting close to a region crossing (to
> >> another region that may be idle) that region could automatically be
> >> started (if "smart regions" was enabled), thus making the region
> >> crossing seamless (and in real-time).  So that flight simulators and
> >> ship simulators could be developed (with large global high resolution
> >> land region maps of the whole world) that could easily be stored on any
> >> single server.  (The idle regions would simply load "on-demand" as
> >> needed).
> >> 
> >>  This way OpenSim core could be used for Science, Educational,
> >> 
> >> Flight/Train/Ship simulation or even gaming purposes.
> >> 
> >>  So that way an OpenSim/realXtend server could technically store
> >> 
> >> thousands of regions all on a single server instance (possibly even
> >> 50-60GB's of high resolution GIS/terrain/region maps of the whole
> >> world).
> >> 
> >>  So that things like high resolution "Global Scenery" could be created
> >> 
> >> (and downloaded) under a "Creative Commons" license for high resolution
> >> scenery (stored in Mega regions) for OpenSim.
> >> 
> >>  http://www.global-scenery.org/
> >>  
> >>  I do have a military GIS (Imagery Analysis/Intelligence) background,
> >>  and
> >> 
> >> don't mind working on things like basic high resolution
> >> OpenSim/realXtend region content creation (empty regions based on high
> >> resolution GIS/DTED data).  DTED data could be used to create elevation
> >> maps, and high resolution imagery could be used for terrain textures.
> >> 
> >>  I believe an OpenSim GIS project would be a very good "basis" for other
> >> 
> >> projects (educational, gaming, city planning, development, architectural
> >> development and multi-person Flight/Train/Ship simulation & training
> >> development) since terrain/region modules would be available free for
> >> download.
> >> 
> >>  The reason "Smart Regions" (idle region timeouts) would be necessary,
> >>  is
> >> 
> >> so things like modeling rural areas like "Yellowstone" National Park
> >> could easily be possible on a small single server (or even a small
> >> single workstation running a single OpenSim/osgrid/realXtend server
> >> instance), since the majority of those regions would usually remain
> >> idle/empty (and with the idle regions shutting down they would consume
> >> NO CPU clock cycles or RAM, thus making "idle" regions identical to
> >> idle web pages that take up nothing more than hard drive space).
> >> 
> >>  Certain regions (like a lobby, main Plaza, or high-traffic regions)
> >> 
> >> would need to remain active (no idle timeout) and would simply have a
> >> idle timeout value of "9999".  (Which would equate to requiring the
> >> region to be completely idle and empty for a minimum of 9999 minutes,
> >> or about 7 days) before going into a "sleep" or idle "shutdown" mode.
> >> 
> >>  I supposed a timeout value of "0000" could also be used for an
> >> 
> >> "infinite" time (no idle timeout or idle shut down) so that regions that
> >> want to remain up (forever) with no idle timeout period or idle shutdown
> >> could thus stay running all the time (consuming valuable RAM, CPU and
> >> system resources).
> >> 
> >>  But as we begin to look at global region mapping and developing large
> >> 
> >> regions (especially from a GIS perspective) then we need to look at the
> >> "flaws" in the current OpenSim server design architecture, and see that
> >> it would simply not be possible to create something like a virtual 3D
> >> world (on a 1:1 virtual/GIS scale) without having billions of physical
> >> servers (and trillions of dollars for physical hardware, let alone the
> >> costs of electricity and hardware just to even run a single instance). 
> >> It seems to be a major flaw in the basic design architecture to require
> >> the "always-on" concept of regions without having a region
> >> "idle-timeout" feature.
> >> 
> >>  Wasting RAM and CPU clock cycles on empty/idle regions (without having
> >> 
> >> an "idle timeout" option would seem like a huge waste of energy, and
> >> system resources, especially when those valuable system resources could
> >> be better spent on active regions with physical avatars that require
> >> the additional processing power and RAM that could be freed up by
> >> shutting down any idle regions (after a specified "time out" period).
> >> 
> >>  Since the majority of the regions would remain empty and idle on a
> >> 
> >> server (similar to static web pages that are not being accessed or
> >> viewed on a web server), it seems extremely foolish to waste such
> >> valuable system resources (such as server CPU clock cycles and server
> >> RAM, let alone the energy/power) wasted on idle regions that remain
> >> empty and idle (but must be loaded as "always on").
> >> 
> >>  Having an "on-the-fly" system that will shut down idle regions (based
> >>  on
> >> 
> >> their configurable regions.ini "time out" value in the regions.ini file,
> >> and automatically load idle regions "on-the-fly" would be much closer
> >> to the way the current web is designed, so that servers can hold
> >> thousands (possibly even millions) of web pages that consume no system
> >> resources (other than physical hard drive space) unless the web pages
> >> are being viewed.
> >> 
> >>  By allowing regions to startup/shutdown dynamically (with a "region
> >> 
> >> timeout" feature) thus making servers able to hold infinite amounts of
> >> idle regions/land (only limited by the amount of hard drive/disk
> >> storage space, which could easily be expanded via RAID or SAN devices).
> >> 
> >>  System Administrators could put "heavy load" regions on separate
> >> 
> >> dedicated servers, while keeping other "idle load" regions all on the
> >> same server (for regions that are generally idle and not visited very
> >> frequently).
> >> 
> >>  As we evolve and slowly get into simulation systems, such as aircraft,
> >> 
> >> train or ship simulators I can see how having large numbers of "idle"
> >> regions would be a requirement for realistic train/ship/aircraft
> >> simulation and gaming (as well as GIS and global flight) and how it
> >> simply would not even be possible to have millions of regions running
> >> all on a single server (due to the current "architectural" problems
> >> with OpenSim).
> >> 
> >>  It just wouldn't even be possible to even create a simple flight, train
> >> 
> >> or ship simulator using OpenSim (because OpenSim couldn't even store the
> >> image/region scenery maps).
> >> 
> >>  Yet, a small simple "X-Plane" flight simulator can store high
> >>  resolution
> >> 
> >> "global scenery" data for the WHOLE world on just 6 or 7 double layer
> >> DVD's (about 60+ GB of terrain data) all on a single workstation.
> >> 
> >>  http://www.global-scenery.org/
> >>  
> >>  I don't mind starting a Global-Scenery package for OpenSim/realXtend
> >>  and
> >> 
> >> I do think it would be extremely nice to offer the community high
> >> resolution realistic regions (free of charge under a "creative commons"
> >> license).
> >> 
> >>  The problem with even starting a large content creation project, is
> >>  that
> >> 
> >> the current OpenSim architecture is unable to even use it.
> >> 
> >>  It would be a complete waste of time and effort to have realistic
> >>  global
> >> 
> >> high resolution terrain regions available for OpenSim to use for
> >> Flight/Train/Ship (as well as Education & Science) simulation but the
> >> whole idea of a large scale GIS/terrain project for OpenSim/realXtend
> >> seems somewhat pointless since the current realXtend/OpenSim server
> >> architecture is not designed to currently even use/store thousands (or
> >> even millions) of idle regions (and the current server architecture of
> >> keeping ALL idle regions running is "flawed" by architectural design).
> >> 
> >>  It would seem to make much more sense (from an engineering standpoint)
> >> 
> >> to have the option to shut any idle regions down, and only leave active
> >> regions running (or at least give users/operators/grid owners the
> >> ability to configure regions & a grid to allow for idle region
> >> timeouts).  I believe this "region idle time out" time should be
> >> configured in the regions.ini configuration file.  With an option of
> >> "9999" for a 7 day idle timeout and a "0000" option to disable the idle
> >> timeout for a particular region (for regions that don't ever want to
> >> shut down due to an idle timeout).
> >> 
> >>  It would make sense to see OpenSim/realXtend support a "smart region"
> >> 
> >> type of system which servers would/could be configured to shut idle
> >> regions down by default (configurable in the regions.ini file for each
> >> region) and the server could load idle regions "On-The-Fly" freeing up
> >> valuable system resources on a physical server.
> >> 
> >>  By implementing "smart regions" into the OpenSim core, a high
> >>  resolution
> >> 
> >> global GIS/terrain project would seem like a viable project to start
> >> (and I wouldn't mind starting such a project) especially for future
> >> science/flight/ship/train simulation.  I could probably find other
> >> universities (and sponsors) that would be willing to host mirror copies
> >> of the empty raw .OAR global terrain region files.  So that other
> >> developers, educators or users could freely download high resolution
> >> realistic region terrains (for educational, gaming or simulation
> >> purposes).
> >> 
> >>  If OpenSim was able to shut down idle regions on the fly, then it could
> >> 
> >> be used for flight simulation, terrain simulation, or even realistic
> >> game simulation (could have large high resolution realistic global
> >> terrain maps).
> >> 
> >>  I don't mind starting/doing free "creative commons" (attribution by)
> >> 
> >> based global GIS/terrain content creation work for the OpenSim/realXtend
> >> community, but the current problem that I see is that the current
> >> OpenSim architecture design would NEVER even be able to support such a
> >> feature (since the current server architecture would require thousands,
> >> or millions or possibly even billions of OpenSim servers just to handle
> >> ONE single copy of "the world" terrain in high-res region data.
> >> 
> >>  Thus making it completely impossible to ever "sail a ship" or "fly an
> >> 
> >> aircraft" around the world on a standalone OpenSim server.
> >> 
> >>  I believe the basics of good simulation programs starts with high
> >> 
> >> resolution image maps (and scenery data).  A flight simulation program
> >> is fairly useless without realistic global imagery.
> >> 
> >>  Although the basic default high resolution terrain/region data would
> >> 
> >> probably only consume about 50-60 GB's of storage space (empty terrain
> >> regions stored in "mega region" OAR format), it would still be
> >> impossible to even store or use the terrain region data on a single
> >> OpenSim server (since an OpenSim server would require that all the
> >> regions be running at the same time, with no way to load/unload the
> >> regions (on-the-fly) as they are accessed or needed.  We would need to
> >> have an "idle time-out" feature and also a "load-on-demand" feature to
> >> load regions and shut down idle regions based on a region "idle
> >> time-out" system (and to load nearby regions, and load regions that
> >> users attempt to teleport to).
> >> 
> >>  I apologize for sounding repetitive.
> >> 
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.dehttps://lists.berlios.de/mailman/listinfo/op
> >> ensim-dev
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> Opensim-dev mailing list
> >> Opensim-dev at lists.berlios.de
> >> https://lists.berlios.de/mailman/listinfo/opensim-dev
> > 
> > _______________________________________________
> > Opensim-dev mailing list
> > Opensim-dev at lists.berlios.de
> > https://lists.berlios.de/mailman/listinfo/opensim-dev