[Opensim-dev] Feature Request: Smart Regions - Idle Region Timeouts and "On The Fly" loading/unloading of inactive idle regions

Wed Jul 7 18:20:40 UTC 2010

I agree it certainly is possible, but every piece of software you listed
costs an a lot of money and would require a new viewer, its not something
that can be done simulator side alone:

CityScape is $19,000 per seat license. CityScape licenses are perpetual and
include 12 months of maintenance with purchase.

Speedtree $9,995 per title
Speedtree Cinema : $39,995 per year

and it still doesn't change the fact you would need a few million dollars in
hardware to achieve this, even with "smart regions"

so unless someone is willing to cough up a few million for development,
probably what Cityscape and Speedtree spent to develop their systems, i just
don't see this happening honestly, at least not anytime soon.

On Wed, Jul 7, 2010 at 9:20 AM, Mark Malewski <mark.malewski at gmail.com>wrote:

>
> *> perhaps, in a future, will be possible to build a *
> *> complet planet, a real scale*
>
> Planets, galaxies and solar systems could probably all easily be rendered
> using procedural generation.  Whole galaxies could be rendered (on-the-fly)
> using procedural generation.
>
> Planet terrain, star systems and a galaxy should/could all be easily
> created using procedural generation.
>
> http://en.wikipedia.org/wiki/Procedural_generation
>
> Elite (by David Braben and Ian Bell back in 1984) was developed using
> procedural generation.
>
> It's quite common for modern games (such as Soldier of Fortune, or "Just
> Cause") to use procedural generation to create large and varied groups of
> tropical islands (or other terrain).
>
> Space flight sims often use procedural generation to develop whole planets,
> terrain, and even galaxies.
>
> I believe "Elite" (designed back in 1984) planned to contain 248 (approximately
> 282 trillion) galaxies with 256 solar systems each, and easily could have,
> but they were afraid that such a gigantic universe would cause "disbelief"
> by players.
>
> So instead of 282 Trillion Galaxies, they settled on just 8 Galaxies.
>  That's still plenty of space for you to travel around in.  ;-)
>
> There are a lot of programs that use procedural generation.
>
> SpeedTree is just one example of a "middleware" that is used to generate a
> large variety of trees (procedurally).  Yet it's leaf textures are fetched
> from regular files, thus creating "high resolution" real foliage.
>
> http://www.speedtree.com/
>
> http://en.wikipedia.org/wiki/SpeedTree
>
> CityScape uses procedural generation and generates cities and terrain just
> from GIS data:
> http://pixelactive3d.com/Products/CityScape/
>
> CityScape supports GIS data import:
> http://pixelactive3d.com/Products/CityScape/?FeatureId=GisData
>
> It can also do procedural Traffic Data generation:
> http://pixelactive3d.com/Products/CityScape/?FeatureId=TrafficData
>
> It's all possible... but we'd probably need thousands of developers just to
> get OpenSim to do 'everything' that we'd like it to do.  ;-)
>
>
> <http://en.wikipedia.org/wiki/SpeedTree>
> On Wed, Jul 7, 2010 at 10:44 AM, Gustavo Alberto Navarro Bilbao <
> alberto.navarro.bilbao at gmail.com> wrote:
>
>> Excuse me if this is not the adecuate place to speak about it,but
>> ...perhaps, in a future, will be possible to build a complet planet, a real
>> scale, using the Pandromeda <http://www.pandromeda.com/products/> fractal
>> concept o a free licence similar one?
>>
>> Alberto
>>
>> 2010/7/7 Diva Canto <diva at metaverseink.com>
>>
>>  There is a ton of interesting engineering challenges on the server side
>>> of these systems. We haven't even begun to scratch the surface. As such,
>>> it's completely unrealistic to expect things like what you describe to
>>> happen any time soon -- at least for interactive, shared, immersive spaces;
>>> non-interactive, non-shared, non-immersive (like Google Earth) it's another
>>> matter. It's also fairly unrealistic to expect those kinds of optimizations
>>> to happen spontaneously and for free. There needs to be a lot of money
>>> thrown in for those things to be prototyped and experimented with --
>>> research money, for starters; that's the kind of thing that will keep a lot
>>> of graduate students busy for the next few years.
>>>
>>>
>>> On 7/6/2010 6:54 PM, Mark Malewski wrote:
>>>
>>> OpenSim/RealXtend,
>>>
>>>  Another thought is the issue of "scalability" with the current OpenSim
>>> and/or realXtend Architecture.
>>>
>>>  Running hundreds (or thousands) of IDLE regions require too many system
>>> resources (RAM, CPU cycles, etc.)
>>>
>>>  I believe that the basic server core (OpenSim) should eventually be
>>> modified so that ALL regions should have a configurable "timeout" option
>>> field (in the regions.ini file).
>>>
>>>  http://opensimulator.org/wiki/Configuring_Regions
>>>
>>>  That way an idle "region timeout" can be set (on a region per region
>>> basis), and this would allow the OpenSim server to shut down idle regions
>>> (based on the "idle timeout" configured in the regions.ini file).
>>>
>>>  So if a region were completely empty for a particular number of minutes
>>> (let's say a region were empty and idle for 5 minutes, then the idle region
>>> could be shut down automatically after the set "idle timeout" period for
>>> that particular region).
>>>
>>>  This way idle (and empty) regions are not consuming valuable system
>>> resources.
>>>
>>>  Then as a user attempts to "teleport" to an idle (empty) region, the
>>> server would simply start that idle region (on-the-fly) so that the user
>>> could teleport into the idle region.
>>>
>>>  It may create a little bit of a slight "pause" (as an idle region is
>>> started) while teleporting, but you normally experience a pause anyways
>>> while teleporting, so a few additional two or three seconds shouldn't make
>>> that much of a difference when teleporting to an empty or idle region.
>>>
>>>  A second improvement upon this, would be a "smart idle regions" grid
>>> option (configurable at the grid and/or region level).  The "smart idle
>>> regions" grid option would take things one step further, and would
>>> automatically allow the configuration option to bring nearby idle regions
>>> out of "sleep" mode, and start them automatically (if an avatar is within a
>>> certain number of meters from a nearby neighboring region).
>>>
>>>  For example this number could be configurable at the Grid and/or region
>>> level to be anything from 1 meter, to 100 meters, or even 256 meters (thus
>>> automatically loading the idle region to the North, the idle region to the
>>> South, the idle region to the East and the idle region to the West). So that
>>> nearby regions could be loaded (as an avatar gets close to the border of
>>> another nearby region).
>>>
>>>  This way if avatars are using high speed vehicles (aircraft, boats,
>>> trains, etc.) and they are getting close to a region crossing (to another
>>> region that may be idle) that region could automatically be started (if
>>> "smart regions" was enabled), thus making the region crossing seamless (and
>>> in real-time).  So that flight simulators and ship simulators could be
>>> developed (with large global high resolution land region maps of the whole
>>> world) that could easily be stored on any single server.  (The idle regions
>>> would simply load "on-demand" as needed).
>>>
>>>  This way OpenSim core could be used for Science, Educational,
>>> Flight/Train/Ship simulation or even gaming purposes.
>>>
>>>  So that way an OpenSim/realXtend server could technically store
>>> thousands of regions all on a single server instance (possibly even
>>> 50-60GB's of high resolution GIS/terrain/region maps of the whole world).
>>>
>>>  So that things like high resolution "Global Scenery" could be created
>>> (and downloaded) under a "Creative Commons" license for high resolution
>>> scenery (stored in Mega regions) for OpenSim.
>>>
>>>  http://www.global-scenery.org/
>>>
>>>  I do have a military GIS (Imagery Analysis/Intelligence) background,
>>> and don't mind working on things like basic high resolution
>>> OpenSim/realXtend region content creation (empty regions based on high
>>> resolution GIS/DTED data).  DTED data could be used to create elevation
>>> maps, and high resolution imagery could be used for terrain textures.
>>>
>>>  I believe an OpenSim GIS project would be a very good "basis" for other
>>> projects (educational, gaming, city planning, development, architectural
>>> development and multi-person Flight/Train/Ship simulation & training
>>> development) since terrain/region modules would be available free for
>>> download.
>>>
>>>  The reason "Smart Regions" (idle region timeouts) would be necessary,
>>> is so things like modeling rural areas like "Yellowstone" National Park
>>> could easily be possible on a small single server (or even a small single
>>> workstation running a single OpenSim/osgrid/realXtend server instance),
>>> since the majority of those regions would usually remain idle/empty (and
>>> with the idle regions shutting down they would consume NO CPU clock cycles
>>> or RAM, thus making "idle" regions identical to idle web pages that take up
>>> nothing more than hard drive space).
>>>
>>>  Certain regions (like a lobby, main Plaza, or high-traffic regions)
>>> would need to remain active (no idle timeout) and would simply have a idle
>>> timeout value of "9999".  (Which would equate to requiring the region to be
>>> completely idle and empty for a minimum of 9999 minutes, or about 7 days)
>>> before going into a "sleep" or idle "shutdown" mode.
>>>
>>>  I supposed a timeout value of "0000" could also be used for an
>>> "infinite" time (no idle timeout or idle shut down) so that regions that
>>> want to remain up (forever) with no idle timeout period or idle shutdown
>>> could thus stay running all the time (consuming valuable RAM, CPU and system
>>> resources).
>>>
>>>  But as we begin to look at global region mapping and developing large
>>> regions (especially from a GIS perspective) then we need to look at the
>>> "flaws" in the current OpenSim server design architecture, and see that it
>>> would simply not be possible to create something like a virtual 3D world (on
>>> a 1:1 virtual/GIS scale) without having billions of physical servers (and
>>> trillions of dollars for physical hardware, let alone the costs of
>>> electricity and hardware just to even run a single instance).  It seems to
>>> be a major flaw in the basic design architecture to require the "always-on"
>>> concept of regions without having a region "idle-timeout" feature.
>>>
>>>  Wasting RAM and CPU clock cycles on empty/idle regions (without having
>>> an "idle timeout" option would seem like a huge waste of energy, and system
>>> resources, especially when those valuable system resources could be better
>>> spent on active regions with physical avatars that require the additional
>>> processing power and RAM that could be freed up by shutting down any idle
>>> regions (after a specified "time out" period).
>>>
>>>  Since the majority of the regions would remain empty and idle on a
>>> server (similar to static web pages that are not being accessed or viewed on
>>> a web server), it seems extremely foolish to waste such valuable system
>>> resources (such as server CPU clock cycles and server RAM, let alone the
>>> energy/power) wasted on idle regions that remain empty and idle (but must be
>>> loaded as "always on").
>>>
>>>  Having an "on-the-fly" system that will shut down idle regions (based
>>> on their configurable regions.ini "time out" value in the regions.ini file,
>>> and automatically load idle regions "on-the-fly" would be much closer to the
>>> way the current web is designed, so that servers can hold thousands
>>> (possibly even millions) of web pages that consume no system resources
>>> (other than physical hard drive space) unless the web pages are being
>>> viewed.
>>>
>>>  By allowing regions to startup/shutdown dynamically (with a "region
>>> timeout" feature) thus making servers able to hold infinite amounts of idle
>>> regions/land (only limited by the amount of hard drive/disk storage space,
>>> which could easily be expanded via RAID or SAN devices).
>>>
>>>  System Administrators could put "heavy load" regions on separate
>>> dedicated servers, while keeping other "idle load" regions all on the same
>>> server (for regions that are generally idle and not visited very
>>> frequently).
>>>
>>>  As we evolve and slowly get into simulation systems, such as aircraft,
>>> train or ship simulators I can see how having large numbers of "idle"
>>> regions would be a requirement for realistic train/ship/aircraft simulation
>>> and gaming (as well as GIS and global flight) and how it simply would not
>>> even be possible to have millions of regions running all on a single server
>>> (due to the current "architectural" problems with OpenSim).
>>>
>>>  It just wouldn't even be possible to even create a simple flight, train
>>> or ship simulator using OpenSim (because OpenSim couldn't even store the
>>> image/region scenery maps).
>>>
>>>  Yet, a small simple "X-Plane" flight simulator can store high
>>> resolution "global scenery" data for the WHOLE world on just 6 or 7 double
>>> layer DVD's (about 60+ GB of terrain data) all on a single workstation.
>>>
>>>  http://www.global-scenery.org/
>>>
>>>  I don't mind starting a Global-Scenery package for OpenSim/realXtend
>>> and I do think it would be extremely nice to offer the community high
>>> resolution realistic regions (free of charge under a "creative commons"
>>> license).
>>>
>>>  The problem with even starting a large content creation project, is
>>> that the current OpenSim architecture is unable to even use it.
>>>
>>>  It would be a complete waste of time and effort to have realistic
>>> global high resolution terrain regions available for OpenSim to use for
>>> Flight/Train/Ship (as well as Education & Science) simulation but the whole
>>> idea of a large scale GIS/terrain project for OpenSim/realXtend seems
>>> somewhat pointless since the current realXtend/OpenSim server architecture
>>> is not designed to currently even use/store thousands (or even millions) of
>>> idle regions (and the current server architecture of keeping ALL idle
>>> regions running is "flawed" by architectural design).
>>>
>>>  It would seem to make much more sense (from an engineering standpoint)
>>> to have the option to shut any idle regions down, and only leave active
>>> regions running (or at least give users/operators/grid owners the ability to
>>> configure regions & a grid to allow for idle region timeouts).  I believe
>>> this "region idle time out" time should be configured in the regions.ini
>>> configuration file.  With an option of "9999" for a 7 day idle timeout and a
>>> "0000" option to disable the idle timeout for a particular region (for
>>> regions that don't ever want to shut down due to an idle timeout).
>>>
>>>  It would make sense to see OpenSim/realXtend support a "smart region"
>>> type of system which servers would/could be configured to shut idle regions
>>> down by default (configurable in the regions.ini file for each region) and
>>> the server could load idle regions "On-The-Fly" freeing up valuable system
>>> resources on a physical server.
>>>
>>>  By implementing "smart regions" into the OpenSim core, a high
>>> resolution global GIS/terrain project would seem like a viable project to
>>> start (and I wouldn't mind starting such a project) especially for future
>>> science/flight/ship/train simulation.  I could probably find other
>>> universities (and sponsors) that would be willing to host mirror copies of
>>> the empty raw .OAR global terrain region files.  So that other developers,
>>> educators or users could freely download high resolution realistic region
>>> terrains (for educational, gaming or simulation purposes).
>>>
>>>  If OpenSim was able to shut down idle regions on the fly, then it could
>>> be used for flight simulation, terrain simulation, or even realistic game
>>> simulation (could have large high resolution realistic global terrain maps).
>>>
>>>  I don't mind starting/doing free "creative commons" (attribution by)
>>> based global GIS/terrain content creation work for the OpenSim/realXtend
>>> community, but the current problem that I see is that the current OpenSim
>>> architecture design would NEVER even be able to support such a feature
>>> (since the current server architecture would require thousands, or millions
>>> or possibly even billions of OpenSim servers just to handle ONE single copy
>>> of "the world" terrain in high-res region data.
>>>
>>>  Thus making it completely impossible to ever "sail a ship" or "fly an
>>> aircraft" around the world on a standalone OpenSim server.
>>>
>>>  I believe the basics of good simulation programs starts with high
>>> resolution image maps (and scenery data).  A flight simulation program is
>>> fairly useless without realistic global imagery.
>>>
>>>  Although the basic default high resolution terrain/region data would
>>> probably only consume about 50-60 GB's of storage space (empty terrain
>>> regions stored in "mega region" OAR format), it would still be impossible to
>>> even store or use the terrain region data on a single OpenSim server (since
>>> an OpenSim server would require that all the regions be running at the same
>>> time, with no way to load/unload the regions (on-the-fly) as they are
>>> accessed or needed.  We would need to have an "idle time-out" feature and
>>> also a "load-on-demand" feature to load regions and shut down idle regions
>>> based on a region "idle time-out" system (and to load nearby regions, and
>>> load regions that users attempt to teleport to).
>>>
>>>  I apologize for sounding repetitive.
>>>
>>>
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at lists.berlios.dehttps://lists.berlios.de/mailman/listinfo/opensim-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at lists.berlios.de
>>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>>
>>>
>>
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at lists.berlios.de
>> https://lists.berlios.de/mailman/listinfo/opensim-dev
>>
>>
>
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/opensim-dev
>
>

-- 
Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20100707/4a9b1a90/attachment-0001.html>