[Opensim-dev] DB refactor for Terrain storage

[Frank Nichols]

Probably a bad idea, but i could see mesh being sliced up into small pieces, and stitched back together when used.. Like i said, probably a bad idea... 

Sent from my iPad Air 2

> On May 9, 2015, at 5:16 PM, Mister Blue <misterblue at misterblue.com> wrote:
> 
> When I added varregions, I added the TerrainData class that wraps the terrain data representation (whether heightmap, mesh, patches, ...). There are two methods for getting and loading a 'blob' representation of the terrain that is used by the database interfaces. 
> 
> The change would be to add to those methods to get pieces of the terrain rather than the whole blob. Should consider, though, that someday terrain could be a mesh or procedural so what should the database do then? TerrainData should have a general enough interface that many terrain representations are possible. 
> 
> == mb
> 
> 
>> On Fri, May 8, 2015 at 10:47 PM, Frank Nichols <j.frank.nichols at gmail.com> wrote:
>> I am not familiar enough with the code to say if it can be done, but I like the idea of handling smaller patches of terrain instead of the entire region.
>> 
>> Frank
>> 
>>> On May 8, 2015, at 10:28 PM, Dev Random <digimancy+opensim at gmail.com> wrote:
>>> 
>>> OpenSimulator now supports variable-sized regions, some of which can be quite large. The recommended maximum is 8192m x 8192m, which is (4 x 8) x (4 * 8) = 1024 times larger than the “standard” 256m x 256m region. Regions of greatly increased size have exposed some weaknesses in the current Terrain model.  Some of the issues are discussed on the Wiki page  http://opensimulator.org/wiki/Varregion .
>>> 
>>> Database communications (in mysql – not sure about other DBs) is one place where the change is very noticeable. Since a region's terrain is stored as a single blob, the “max_allowed_packet” setting must be increased to accommodate the greatly increased blob size for a large VarRegion. This large size also appears to lead to noticeable pauses while the database is updated during terraforming operations.
>>> 
>>> 
>>> Here's what I'm thinking:
>>> 
>>> Since logic already exists for working with 16x16 patches, perhaps the terrain can be persisted in the same layout. This might remove the need for extremely large DB packet sizes, and may drastically reduce delays while terraforming.
>>> 
>>> Start by adding a new table "Terrain_Patches", as shown below (naming conventions, etc. TBD):
>>> 
>>> Terrain_Patches:
>>> * region_uuid (varchar)
>>> * x_pos (short)
>>> * y_pos (short)
>>> * blob? (16x16 float?)
>>> 
>>> Moving the actual elevation data out of the "terrain" table leaves it as basically a header table.
>>> 
>>> terrain:
>>> * RegionUUID (varchar)
>>> * Revision (int) <-- 28?
>>> * Heightfield (x_size (short), y_size (short)) <-- no height data here
>>> 
>>> When a region is terraformed, the already-existing taint logic can be used to determine which patches need to be persisted. By adding a second taint array ("m_db_taint"...?), and setting an element every time an "m_taint" element is set, there will always be an up-to-date list of patches requiring persistence. No need to re-write an entire region when only a few patches changed. Removing the all-or-nothing requirement for loading terrain can help prevent the immense packet sizes required by large regions.
>>> 
>>> I lack the skills to prototype this, so I'm throwing it out to the mailing list for comments. Obviously, more re-work would be required in the code to stitch together the patches, etc...  This email is more about concept than details.
>>> 
>>> _______________________________________________
>>> Opensim-dev mailing list
>>> Opensim-dev at opensimulator.org
>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>> 
>> 
>> _______________________________________________
>> Opensim-dev mailing list
>> Opensim-dev at opensimulator.org
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
>> 
> 
> _______________________________________________
> Opensim-dev mailing list
> Opensim-dev at opensimulator.org
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://opensimulator.org/pipermail/opensim-dev/attachments/20150509/bf049f9a/attachment.html>