0006104: vector math precision

View Issue Details [ Jump to Notes ]

[ Issue History ] [ Print ]

Project

Category

View Status

Date Submitted

Last Update

0006104

opensim

[REGION] Script Functions

public

2012-07-24 04:27

2012-08-02 19:04

Reporter

SignpostMarv

Assigned To

Priority

normal

Severity

minor

Reproducibility

always

Status

new

Resolution

open

Platform

OS Version

Product Version

master (dev code)

Target Version

Fixed in Version

Summary

0006104: vector math precision

Description

If one performs certain math operations on a vector and then perform an equality check on that vector with another vector, one will get FALSE.

Steps To Reproduce

Use a script with similar arguments:

float scaleMultiplier = 1.5;
list supportedSizes = [
    <.440, .140, .215>
];

Set a prim's size to be a vector multiplied by the scaleMultiplier.

Get the prim's new size then:

vector foo = primNewSize / scaleMultiplier;

integer j;
integer k=llGetListLength(supportedSizes);
integer valid = FALSE;
vector current;
for(j=0;j<k;++j){
    current = llList2Vector(supportedSizes,j);
    if(curren == foo){
        valid = TRUE;
    }
}

after exiting the loop, valid will be FALSE.

In order for valid to equate to TRUE, one needs to do this before entering the loop:

foo *= 1000000;
foo.x = llRound(foo.x);
foo.y = llRound(foo.y);
foo.z = llRound(foo.z);
foo /= 1000000;

Tags

No tags attached.

Git Revision or version number

892dd59b131

Run Mode

Standalone (1 Region)

Physics Engine

BasicPhysics

Script Engine

Environment

.NET / Windows32

Mono Version

None

Viewer

Attached Files

Relationships

Notes
(0021971) kcozens (administrator) 2012-08-02 19:04	Comparing floats for equality is never a good idea. You assign a variable a given float value but you don't know for certain how accurately that value is internally. If you want to do some tests on this I would suggest you print out the original vector, the one multiplied by the constant, and the one that was multiplied and then divided. That will let you see where you are running into limits in the accuracy of floating point numbers in the computer.

Issue History
Date Modified	Username	Field	Change
2012-07-24 04:27	SignpostMarv	New Issue
2012-08-02 19:04	kcozens	Note Added: 0021971