Fixed axisAngle implementation

`acos` domain is in range [-1.0, 1.0]. Due to inaccuracies the value `angleCos` may be slightly outside that range for a correct matrix and `acos(angleCos)` produces `NaN` in that case.

The fix is we check `angleCos` value and return `acos(1)` for `angleCos > 1` and `acos(-1)` for `angleCos < -1`.

The original code checked only for `angleCos` close to `1.0` and returned an incorrect value for `acos(1)`, which is `0`, not  `pi/4`.
This commit is contained in:
Sergey Krivohatskiy 2021-02-20 20:16:31 +03:00 committed by GitHub
parent 6347f62261
commit acab24129d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -78,10 +78,18 @@ namespace glm
if (glm::abs(s) < T(0.001))
s = static_cast<T>(1);
T const angleCos = (m[0][0] + m[1][1] + m[2][2] - static_cast<T>(1)) * static_cast<T>(0.5);
if(abs(angleCos - static_cast<T>(1)) < epsilon)
angle = pi<T>() * static_cast<T>(0.25);
if(angleCos >= static_cast<T>(1.0))
{
angle = static_cast<T>(0.0);
}
else if (angleCos <= static_cast<T>(-1.0))
{
angle = pi<T>();
}
else
{
angle = acos(angleCos);
}
axis.x = (m[1][2] - m[2][1]) / s;
axis.y = (m[2][0] - m[0][2]) / s;
axis.z = (m[0][1] - m[1][0]) / s;