Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007765opensim[REGION] OpenSim Corepublic2015-11-26 06:232015-11-28 14:04
Reporteraiaustin 
Assigned Tomelanie 
PriorityhighSeveritymajorReproducibilityalways
StatusassignedResolutionreopened 
PlatformPCOSWindowsOS Version10
Product Versionmaster (dev code) 
Target Versionmaster (dev code)Fixed in Version 
Summary0007765: Region with deep water (some items at negative Z) trashed and load oar now fails for items with negative Z
DescriptionI have found that a region with normal 20m sea level but with build items that have centres that are down to -20m has been completely trashed. Many items have been moved up so their centres are not -ve Z. All items that were shifted that I have checked have had their Z position set to 0.0.

This is so on several grids that have such a deep water region on them including OSGrid. Some changes were made some time ago to Bulletsim to ensure negative Z was allowed. ODE already was okay.

Has any change occurred in the code that properly allows -ve Z. This was all fixed up some time ago, and a possible interaction may have occurred with the avination code... as this was seen immediately after installing a version of this code on OSGrid.

Image attached of region with water layer removed.

Note using Bulletsim on Windows 10/.NET and latst dev master opensim-0.9.0-101-g172bb05
TagsNo tags attached.
Git Revision or version number
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Environment.NET / Windows64
Mono VersionNone
ViewerFirestorm 4.7.5
Attached Filesjpg file icon 2015-11-26-Undersea-Minus-Trashed_001.jpg [^] (133,628 bytes) 2015-11-26 06:23


jpg file icon 2015-11-26-Undersea-Minus-Trashed_002.jpg [^] (78,070 bytes) 2015-11-26 06:24


jpg file icon 2015-11-26-OSGrid-Marineville-Negative-Z.jpg [^] (105,512 bytes) 2015-11-26 07:19


patch file icon 0006-Fix-changes-to-boundingSize-check-defaults.patch [^] (2,016 bytes) 2015-11-26 16:05 [Show Content]
jpg file icon 2015-11-27-Deep-Build-Test-Firestorm-4.7.5.jpg [^] (380,133 bytes) 2015-11-28 07:05

- Relationships

-  Notes
(0029681)
aiaustin (developer)
2015-11-26 06:46
edited on: 2015-11-26 06:55

Could the issue be introduced by Jak's extended load oar changes and some things related to that as when I tried to restore the region using load oar it reported

14:45:33 - [ARCHIVER]: Ignored 1160 scene objects that already existed in the scene or were out of bounds

(0029682)
aiaustin (developer)
2015-11-26 07:19
edited on: 2015-11-26 07:36

I have been able to bring back a version of OSGrid add on region release from 21st October 2015 (osgrid-opensim-10212015.v0.8.3.41b2855.zip) prior to the avination merge and load oar changes (in case one or other introduced the problem) and have restored my data base and the OSGrid Marineviille region with 20m water level and build down to -20m works fine on that.

2015-11-26-OSGrid-Marineville-Negative-Z.jpg with water surface removed.

(0029683)
aiaustin (developer)
2015-11-26 07:47

To help others who have deep sea builds... FIX....

Important... do NOT try a load oar or a save of the oar in the trashed state.

The region is not actually trashed, its just that all items with negative Z are moved up to Z=0 when loaded with latest versions of OpenSim 0.9.0 dev master.

Use a version of Opensim such as opensim-2015-11-13-02-40-554d4ba.zip and log back in and your region is still all in good shape with items with centres below Z-0m located correctly.

Pending investigation take care and perhaps do not upgrade until this is resolved.
(0029685)
Dev Random (reporter)
2015-11-26 08:09

aiaustin: Can your avatar navigate down to the negative-z areas? If not, it's probably not the "load oar" changes.
(0029686)
melanie (administrator)
2015-11-26 08:31

9afe2b0
(0029687)
aiaustin (developer)
2015-11-26 09:34
edited on: 2015-11-26 09:43

Thanks for that correction Melanie. Note that the objects that were shifted were already in world on regions and grids that have been running for some years. They were not coming from an "external source".

That may be part of the issue fix but load oar definitely still does not work. I am working my way through the changes that Jak Daniels made to add parameters to load OAR and change the land/object handling accordingly. They may be part of the cause too. I hope he can take a look to see if he is blocking the loading of OAR saved items which have a negative Z. Those are not out of bounds Jak.

In dev master opensim-0.9.0-102-g9afe2b0.zip a load oar for an OAR recently saved (in October 2015) with items placed at -ve Z are now not working... and report

[ARCHIVER]: Ignored 1160 scene objects that already existed in the scene or were out of bounds

An attempt to load the OAR with an object displacement of Z=+20m (as all my region objects are only down to -20m in fact) also fails to load exactly those same objects, so the block seems to be coming from the archiver as well rather than just at the point that the object is stored.

(0029688)
aiaustin (developer)
2015-11-26 09:50
edited on: 2015-11-29 11:59

Dev Random, there is no problem navigating down deeper than 0m, and indeed doing most editing in world, with just a few glitches reported to some viewer developer mantis/JIRAs.

We have deep water regions for Oil Rig training, and one for a deep sea classroom, as well as hobby regions for underwater SciFi.

For a bit more on this, and some images, see
https://blog.inf.ed.ac.uk/atate/2014/05/10/opensim-supports-deep-seas-and-minus-0m-underwater-builds/ [^]

(0029689)
melanie (administrator)
2015-11-26 09:57

The "external source" is the database. From my POV that is external, you may see this differently.
(0029690)
melanie (administrator)
2015-11-26 10:10

I checked the code path for OAR loading and it should also be fixed with this patch. It all boils down to that message you're getting - it can't actually be triggered by out of bounds objects but only by bad/duplicate UUIDs. The code path has no bounds check I could see in my quick scan.
(0029691)
Jak Daniels (reporter)
2015-11-26 11:54

During the OAR import objects will be skipped that fall outside the bounding box (if specified), with the debug message:

[ARCHIVER]: Skipping object from OAR in scene because it's position <x,y,z> is outside of bounding cube.

Do you get any of those messages using --debug in the 'load oar' command?

Do you have an OAR I can try?
(0029692)
Jak Daniels (reporter)
2015-11-26 12:21

Do any of those objects have scripts in them to move them to <x,y,-20> ?

if so what is the .ini setting 'DisableUndergroundMovement' set to?

Can you tell me the load oar command you are using and the source and destination region sizes?
(0029693)
Jak Daniels (reporter)
2015-11-26 12:34

Ah yes, I see the new code in SceneGraph.cs now. That'll be it then :)
(0029694)
aiaustin (developer)
2015-11-26 12:35
edited on: 2015-11-26 12:40

All my regions and any adjacent regions are 256x256 and always have been. I have never used mega or var regions, and none are nearby.

The OAR is one saved as a back up of the AiLand Marineville region. All the objects that are not loaded have a Z set to negative values down to -20m in this particular OAR. They are not moving at all. Its the static build that is not working now. Whereas it was up to the last OSGrid built and dev master I was using and where this was checked,

I am using plain

change region <regionnname>
load oar <filename>

I also tried a load with the objects lifted up 20m in case that was working and did that with and without "" as I am not sure from help text if that is needed:

load oar --displacement <0,0,20> <filename>
load oar --displacement "<0,0,20>" <filename>

In all cases every object that has a Z that is < 0m is not loaded.. even with the latest commit that Melanie put in place.

I have now made a one region standalone new test grid and the same problem occurs. I will keep this grid for underwater and deep water testing while this issue remains, so can try things out without risk to my main grids or our OSGrid regions.

(0029695)
aiaustin (developer)
2015-11-26 12:43
edited on: 2015-11-26 12:59

@Jak I cannot see any of your load oar enhancement commits that change anything in SceneGraph.cs. But I do see some lines like this in your commits...

Vector3 boundingOrigin = new Vector3(0f, 0f, 0f);

Did you assume 0 for lower bound on Z? There is no lower bound. But I think the physics engines have a lower limit for SOME computations for efficiency. But its still not an object position limit. I cannot recall if its -512m or something. MisterBlue who adjusted this last in Bulletsim about 18 months ago may be able to advise.

Could it be that you also limit Z max to 4096m? I have seem builds way up to 10,000m. And code in there has some 359f figures built in. Surely we can have 359.95 as a permitted rotation?

(0029696)
Jak Daniels (reporter)
2015-11-26 12:51

Melanie committed a patch at 17.30... 9afe2b0 the focus of which was to change

if (npos.Z < 0.0) npos.Z = 0.0f;

to

if (npos.Z < 0.0 && clampZ) npos.Z = 0.0f;

in SceneGraph.cs

The .ini setting ClampNegativeZ = false in OpenSimDefaults.ini (also just patched)
should restore the original behaviour of allowing Z<0
(0029698)
aiaustin (developer)
2015-11-26 13:03

I was doing my latest tests using commit 9afe2b0 (from 16:29 GMT) and download file name opensim-0.9.0-102-g9afe2b0.zip

load oar <filename>

does not load any item in the OAR that has a negative Z.

I will do clean testing tomorrow on a fresh grid and saved OARs to make sure I am not in an intermediate state and that I am sure I am using the very latest patch by Melanie.
(0029700)
Jak Daniels (reporter)
2015-11-26 16:12

the patch fixes a problem with the --bounding-size parameter when it is not supplied.

One piece of the code was changed in commit 71f5c2b856aeab2b535094804f15317d5dc544e1 to assume default max Z is float.MaxValue, but the command server provided Region.Constants.RegionHeight as a default.

This fixes the placing of objects, but I noticed another bug with terrain.

commands on the console like 'terrain fill -20' produce terrain at Z=0m
Also the load from oar puts negative terrain at 0m
The terrain problem is possibly a avnmerge issue... still investigating
(0029701)
UbitUmarov (administrator)
2015-11-26 16:19

patch 6 commited
(0029702)
melanie (administrator)
2015-11-26 16:22

I would not call it an "issue". That is why I added a config option. There are good, sane reasons not to allow negative Z and although I believe in people's freedom to do so, and even do so by default, the option to restrict Z to 0 < Z < 4096 needs to remain even if not default.

Z < 0 may not be compatible with future clients or physics engines and Z > 4096 runs the risk of build distortion due to type width limitations. The option to restrict has to remain and therefore this isn't an "issue", it's a valid choice.

Of course, as I did in my importing patch, it needs to be made conditional and default to Opensim behavior, but it's not a problem to be removed but one of a number of valid choices.
(0029703)
Jak Daniels (reporter)
2015-11-26 16:32

I fully support the patch you did for objects < 0m, that's the way to go... and making it an option that defaults to the previous behaviour, but it seems we have a problem with terrain heights < 0m too that I'm still investigating.

I can do a
# terrain fill -20
on pre avnmerge
but on current master it places terrain at 0m
Same with 'load oar' containing a terrain below 0m. So I happened to call it an 'issue'. From what you said earlier about not building under 0m, in AVN it is a sensible rule. Here, we should probably keep the original behaviour, but with an option ;)
(0029704)
melanie (administrator)
2015-11-26 16:37

What I said. Allow it, allow it by default. But don't remove the option to disallow.
(0029712)
aiaustin (developer)
2015-11-27 01:02
edited on: 2015-11-27 03:24

Absolutely no problem at all with making it an option as Melanie has done. That's as good way to go. Do we need a ClampPostitiveZ too? But what is the "current" setting to use? There is one built in limit I found set at 10000 for example (for the "delete object outside" console command), but is that a REAL limit?

So in investigating this, I spotted one more potential place where the check ought to now be respectful of the ClampNegativeZ setting... this now really ought to be modified to change the lower bound check so that Z < 0 only applies if ClampNegativeZ = false

delete object outside - Delete all scene objects outside region boundaries.
This is currently if z < 0 or z > 10000.

(0029715)
aiaustin (developer)
2015-11-27 01:31

In response to an earlier question from Jak... all my tests are done with the default DisableUndergroundMovement as I see this is commented out in my OpenSim.ini files.

    ; DisableUndergroundMovement = true

I am not actually moving any prims that are below Z = 0, luckily. Though I now see that I should probably allow that in case of whales, fish, etc that may be programmed to dive below that level. At the moment whales move in the region between sea level at 20m and 0m, so I had not spotted that issue. Thanks for ointing it out.
(0029716)
aiaustin (developer)
2015-11-27 01:48
edited on: 2015-11-27 03:10

Using a freshly created single region standalone... I can confirm the following.

a) Melanie's patch to set ClampNegativeZ = false by default allows any current deep water build and deep terrain to remain.

b) Jak's fix to the load oar bounding box now allows the OBJECTS in previously saved OAR files with deep water build objects below Z=0m to load successfully.

c) As Jak has noted if any part of the TERRAIN in an OAR is below 0m, it is currently incorrectly flattened and set to 0m.

d) While this area is being tidied up, and given Melanie's comment that Z heights above 4096 are problematic in some instances, should a ClampMaximumZ be added (set to 10000 by default) with comments to explain the value of using 4096 instead?

e) The "delete object outside" has hardwired limits (Z < 0 and Z > 10000) that ought to be adjusted to account for ((Z < 0 if ClampNegativeZ = true) and Z > ClampMaximumZ)?

(0029718)
aiaustin (developer)
2015-11-27 03:24
edited on: 2015-11-27 14:25

I wonder if someone with knowledge of how this works can answer these questions... if we have the default...

DisableUndergroundMovement = true

I assumed that mean that scripted objects and the avatar could not move below the terrain level whatever level that was set to. But that makes me wonder about two thiongs now...

1. If the terrain levels at some points in the region have negative Z does this setting stop movement of such objects below that anyway even though they are above the surface terrain model?

2. Using the build tool and editing objects I assume can be set to be at any allowed Z level even under the surface terrain?

I may try a test with these various mods, but someone may just know the answer to these two questions to save a lot of time.

(0029745)
Jak Daniels (reporter)
2015-11-28 06:11

terrain below 0m is now also fixed as of commit 9928076d1a16b3b62fb6cd5e24f43b9f9c648457. However there are still some other issues:

Using bullet it is possible for the avatar to walk below 0m. On UbODE it is not, the av is clamped at 0m

Teleports to Z < 0m will throw you to <128,128,128>

In old master using bullet it is possible to rez a prim on the ground below 0m

In current master the prim is now thrown to <0,0,0>, but it is possible to move it below 0m with the pull bars, but it's clamped at -3.5 if a value is input into the Z box in the viewer (viewer issue?)

It's a can of worms :)
(0029746)
aiaustin (developer)
2015-11-28 06:17
edited on: 2015-11-28 06:18

Terrain with elements below Z = 0m now loads as at opensim-0.9.0-118-g9928076.zip

Some things such as the Build Tool/Edit type in box for the Z A few other issues are probably remaining hard coded figures in OpenSim.

(0029747)
aiaustin (developer)
2015-11-28 06:53
edited on: 2015-11-28 07:05

Teleports to negative Z in the map work fine in Firestorm 4.7.5. E.g I just teleported to Marineville on the OSGrid locally and via a HG avatar with no problem.

hop://login.osgrid.org:8002/Marineville/182/75/-19 [^]

Placing prims with the build tool below Z=0m also works fine (see attached image 2015-11-27-Deep-Build-Test-Firestorm-4.7.5.jpg) and editing as expected using the move bars is good... except you cannot type a -ve value into the Z edit box. You have to move it using the bars when below 0m. This is a Firestorm bug. http://jira.phoenixviewer.com/browse/FIRE-13582 [^]

Anyone who want to try a deep see sim is welcome to visit. Pick up the diving suit or be prepared to hold your breath. Stay safe down there.

(0029749)
aiaustin (developer)
2015-11-28 11:12
edited on: 2015-11-29 11:52

A note from Misterblue who set the lower terrain limit in the BulletSim physics engine as a configurable parameter.. with default -500m

> On 29/04/14 03:29, Mister Blue wrote:
>
>> I checked in changes to BulletSim which default the ground plane to -500
>> instead of zero. There is a new BulletSim
>> parameter of "[BulletSim]TerrainGroundPlane = -500" which allows that to
>> change.

This is a plane to act as a base to stop physical objects that are set lower than or fall though the terrain dropping to infinity.

(0029751)
aiaustin (developer)
2015-11-28 11:35
edited on: 2015-11-29 11:50

Firestorm viewer developers are asking if we should agree a way to communicate current min and max Z settings in some way.

See discussion at http://jira.phoenixviewer.com/browse/FIRE-13582 [^]

Lower value comes from higher of [BulletSim]TerrainGroundPlane or 0m if boolean ClampNegativeZ is true. Should one ClampNegative Z set to a height in metres replace these and apply to all physics engines in future?

Higher limit is not currently configurable, but I have suggested it should be maybe using a ClampPositveZ parameter. Hardwired limit of 10000m is in an inbult command at present.

Current OpenSim default using .ini.example and default is -500m (set by default BulletSim engine default lower limit plane for physical objects that fall though terrain) and 10000m (set by a hardwired console command).

(0029752)
Robert Adams (administrator)
2015-11-28 12:28

OpenSimulator coordinates are floats which have a bit over 6 significant digits. At 9999 meters, there are about two digits below the decimal point. Positioning will get weirder the higher you go.

BulletSim just uses whatever Z values are passed and could handle the whole range of Z values. It implements a 'ground plane', though, which is below the terrain and through which nothing can pass. This keeps things that get through the terrain from falling to infinity.
(0029753)
aiaustin (developer)
2015-11-28 14:04
edited on: 2015-11-28 14:14

Thanks for that clarification Robert.

Cinder has also reminded us that Second Life has a Z range of 0m to 4096m...
http://wiki.secondlife.com/wiki/Limits [^]


- Issue History
Date Modified Username Field Change
2015-11-26 06:23 aiaustin New Issue
2015-11-26 06:23 aiaustin File Added: 2015-11-26-Undersea-Minus-Trashed_001.jpg
2015-11-26 06:24 aiaustin File Added: 2015-11-26-Undersea-Minus-Trashed_002.jpg
2015-11-26 06:28 aiaustin Summary Region with deep water (minus Z) trashed => Region with deep water (some items at minus Z) trashed
2015-11-26 06:28 aiaustin Description Updated View Revisions
2015-11-26 06:46 aiaustin Note Added: 0029681
2015-11-26 06:48 aiaustin Summary Region with deep water (some items at minus Z) trashed => Region with deep water (some items at negative Z) trashed and load oar now fails for items with negative Z
2015-11-26 06:55 aiaustin Note Edited: 0029681 View Revisions
2015-11-26 07:19 aiaustin Note Added: 0029682
2015-11-26 07:19 aiaustin File Added: 2015-11-26-OSGrid-Marineville-Negative-Z.jpg
2015-11-26 07:19 aiaustin Note Edited: 0029682 View Revisions
2015-11-26 07:36 aiaustin Note Edited: 0029682 View Revisions
2015-11-26 07:47 aiaustin Note Added: 0029683
2015-11-26 08:09 Dev Random Note Added: 0029685
2015-11-26 08:31 melanie Note Added: 0029686
2015-11-26 08:31 melanie Status new => resolved
2015-11-26 08:31 melanie Resolution open => fixed
2015-11-26 08:31 melanie Assigned To => melanie
2015-11-26 09:34 aiaustin Note Added: 0029687
2015-11-26 09:34 aiaustin Status resolved => feedback
2015-11-26 09:34 aiaustin Resolution fixed => reopened
2015-11-26 09:39 aiaustin Note Edited: 0029687 View Revisions
2015-11-26 09:43 aiaustin Note Edited: 0029687 View Revisions
2015-11-26 09:50 aiaustin Note Added: 0029688
2015-11-26 09:50 aiaustin Status feedback => assigned
2015-11-26 09:53 aiaustin Note Edited: 0029688 View Revisions
2015-11-26 09:57 melanie Note Added: 0029689
2015-11-26 10:10 melanie Note Added: 0029690
2015-11-26 11:54 Jak Daniels Note Added: 0029691
2015-11-26 12:21 Jak Daniels Note Added: 0029692
2015-11-26 12:34 Jak Daniels Note Added: 0029693
2015-11-26 12:35 aiaustin Note Added: 0029694
2015-11-26 12:40 aiaustin Note Edited: 0029694 View Revisions
2015-11-26 12:43 aiaustin Note Added: 0029695
2015-11-26 12:50 aiaustin Note Edited: 0029695 View Revisions
2015-11-26 12:51 Jak Daniels Note Added: 0029696
2015-11-26 12:51 aiaustin Note Edited: 0029695 View Revisions
2015-11-26 12:54 aiaustin Note Edited: 0029695 View Revisions
2015-11-26 12:59 aiaustin Note Edited: 0029695 View Revisions
2015-11-26 13:03 aiaustin Note Added: 0029698
2015-11-26 16:05 Jak Daniels File Added: 0006-Fix-changes-to-boundingSize-check-defaults.patch
2015-11-26 16:12 Jak Daniels Note Added: 0029700
2015-11-26 16:19 UbitUmarov Note Added: 0029701
2015-11-26 16:22 melanie Note Added: 0029702
2015-11-26 16:32 Jak Daniels Note Added: 0029703
2015-11-26 16:37 melanie Note Added: 0029704
2015-11-27 01:02 aiaustin Note Added: 0029712
2015-11-27 01:31 aiaustin Note Added: 0029715
2015-11-27 01:48 aiaustin Note Added: 0029716
2015-11-27 01:49 aiaustin Note Edited: 0029716 View Revisions
2015-11-27 01:49 aiaustin Note Edited: 0029716 View Revisions
2015-11-27 01:50 aiaustin Note Edited: 0029716 View Revisions
2015-11-27 03:10 aiaustin Note Edited: 0029716 View Revisions
2015-11-27 03:10 aiaustin Note Edited: 0029716 View Revisions
2015-11-27 03:24 aiaustin Note Added: 0029718
2015-11-27 03:24 aiaustin Note Edited: 0029712 View Revisions
2015-11-27 12:18 aiaustin Description Updated View Revisions
2015-11-27 14:25 aiaustin Note Edited: 0029718 View Revisions
2015-11-27 14:25 aiaustin Note Edited: 0029718 View Revisions
2015-11-28 06:11 Jak Daniels Note Added: 0029745
2015-11-28 06:17 aiaustin Note Added: 0029746
2015-11-28 06:18 aiaustin Note Edited: 0029746 View Revisions
2015-11-28 06:53 aiaustin Note Added: 0029747
2015-11-28 06:54 aiaustin Note Edited: 0029747 View Revisions
2015-11-28 06:55 aiaustin Note Edited: 0029747 View Revisions
2015-11-28 06:55 aiaustin Note Edited: 0029747 View Revisions
2015-11-28 07:05 aiaustin Note Edited: 0029747 View Revisions
2015-11-28 07:05 aiaustin File Added: 2015-11-27-Deep-Build-Test-Firestorm-4.7.5.jpg
2015-11-28 11:12 aiaustin Note Added: 0029749
2015-11-28 11:35 aiaustin Note Added: 0029751
2015-11-28 11:37 aiaustin Note Edited: 0029751 View Revisions
2015-11-28 11:37 aiaustin Note Edited: 0029749 View Revisions
2015-11-28 11:37 aiaustin Note Edited: 0029749 View Revisions
2015-11-28 11:57 aiaustin Note Edited: 0029751 View Revisions
2015-11-28 12:28 Robert Adams Note Added: 0029752
2015-11-28 14:04 aiaustin Note Added: 0029753
2015-11-28 14:14 aiaustin Note Edited: 0029753 View Revisions
2015-11-29 11:50 aiaustin Note Edited: 0029751 View Revisions
2015-11-29 11:52 aiaustin Note Edited: 0029749 View Revisions
2015-11-29 11:59 aiaustin Note Edited: 0029688 View Revisions


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker