|Description||In many places in the OpenSimulator and libopenmetaverse code, floats (32-bit floating-point values) are used. A float uses 23 bits for the mantissa, which gives a precision of 1/2^23 = 1/8,388,608 ? 0.00000012 to 1/(2^22-1) = 1/4,194,303 ? 0.00000024 of a value. When doing e.g. local trigonometry, with values ranging from -6.28 to 6.28, this gives an absolute precision of about 0.000001, which is usually quite enough, and often a margin of 0.000001 is used when comparing if values are equal.|
However, when used to handle coordinates in a 32x32 varregion, with a 8,192 m side, the absolute precision in X-Y coordinates in the NE end is only about 2 mm, enough to cause visible glitches or cracks in constructions. The same goes for the Z coordinate in high altitude skyboxes, at 4,000 m, with an absolute precision of about 1 mm. Besides visible glitches, several functions may also give unexpected results when comparisons expecting better coordinate precision than 1 mm are used.
In several cases these problems are hidden, because comparisons using, or really needing, <= or >= will see two values with poor precision usually "rounded" to the same value. There is however a risk that they will fall on different sides of the rounding border, e.g. 8191.9969995 m => 8191.996 m, 8191.9970005 m => 8191.998 m, giving an apparent difference of 0.002 m = 2 mm, despite the actual difference is 0.0000001 m = 0.0001 mm, well within the 0.000001 (m) margin often used (values rounded to decimals for clarity).
In the above example, with a margin of 0.000001 m, the risk of a false high difference is 0.05 % (0.000001/0.002*100%), which I think many can accept. However, if the margin instead is 0.001 m, the risk of a false high difference is 50%, which I think most would consider too high. Also, this small risk may be behind some rare and hard to reproduce glitches in OpenSimulator behavior, since they will only happen maybe 1/2,000 times under random conditions, but never for 99.95% of repeated conditions and always for 0.05% of repeated conditions.