Babylon animations: Matrix interpolation and scaling from Blender exporter

[kpko]

Hi guys,

 

I want to bump this topic since nobody answered it yet. I tried to implement matrix interpolation, which led to the problem described here: http://www.html5gamedevs.com/topic/12023-blender-animations-inverting-mesh-normals/

 

The interpolation itself worked, but we got some weird inverted normals in some models. Since blender exporter only exports key matrices, we can't interpolate these animations. This means we have to create as many keyframes as the frames per second in babylon.js. An animation with 3 keyframes has to be blown up to 60 keyframes if the framerate equals 60 fps. 

 

I don't know which operation in my code led to inverted normals, I encountered a strange exported scaling from the Blender exporter as I described in the other thread. Deltakosh was right when he wrote, the animations without interpolations worked. But my question regarding the negative scaling at z axis is still unanswered.

 

Do we know anyone who knows the Blender exporter and could tell us where the -1 z scaling comes from? Maybe I'm wrong in my interpretation of this value, though. Could someone with deeper knowledge of matrices clarify this? :-)

 

Looking forward to hear from you guys.

 

For reference

 

My original thread regarding this issue: http://www.html5gamedevs.com/topic/11753-issues-with-babylonjs-blender-exporter/

 

 

I'm having two problems with the exporter for Blender at the moment.

I have 3 key frames in my animation in Blender, the exporter exports 100 key frames (one for each frame in the animation). This doesn't really make sense to me, since we could use those 3 key frames and interpolate. Which leads to my second problem...

Babylon.js doesn't support interpolation between matrices. I animated the position and rotation of a bone in Blender, so it should export the key frames with the position as a vector3 and the rotation as a quaternion - both data types where Babylon.js could interpolate by itself. Shouldn't the exporter export the simple vectors and quaternions? Or is it possible to somehow interpolate between matrices? 

 

My assumption regarding matrix interpolation and the scale values:

 

Hi again,

 

I think I found a clue leading to the problem. I looked into the file exported by Blender:

...
"skeletons": [
{
"name": "Skeleton",
"id": 0,
"bones": [
{
"name": "pelvis",
"index": 0,
"matrix": [
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
-1,
0,
0,
4,
-0.1425,
1
],
...

If you have no rotation, the scale values of the matrix (the values at position 0 (scale x), 5 (scale y) and 10 (scale z)) would be X: 1, Y: 1 and Z: -1. This could lead to the flipped normals problem. These values are created from the Blender exporter which I'm not very familiar with. 

 

Maybe my code did reveal that issue because I multiply with those matrices when I do the interpolation. I'm still looking into this issue, but could anyone who knows the Blender exporter tell me where the value "-1" is coming from? Just to be sure :-)

 

[GameMonetize]

can you provide simple examples of mesh exported wrong?

[kpko]

Hi,

 

I'm not sure the mesh is exported wrong though, I want to investigate that with you guys. Could be as well a problem with my interpolation code, but I would really like to have a better understanding of what's going on here :-)

 

One example would be the model from this post: http://www.html5gamedevs.com/topic/12023-blender-animations-inverting-mesh-normals/

You can find the link to his exported model here: https://dl.dropboxusercontent.com/u/70260871/webgl/amorgan/figure.zip

 

When opening the PlayerBody3.babylon file, I wonder how this matrix was constructed (the first one I've found on the first bone (pelvis) of the first skeleton):

[1,0,0,0,0,1,0,0,0,0,-1,0,0,4,-0.1425,1]

Given we have no rotation on this model and the scaling is defined as in https://www.opengl.org/discussion_boards/showthread.php/159215-Is-it-possible-to-extract-rotation-translation-scale-given-a-matrix

[sx 0 0 0]| 0 sy 0 0|| 0 0 sz 0|[ 0 0 0 1]

The scaling would be -1 on z axis. 

[GameMonetize]

Oh that this is because blender is a right handed system and babylon.js a left handed so to switch hand I invert the z

[kpko]

Aaah, I see :-) But doesn't this lead to an inverted mesh? I mean, isn't the mesh wrapped inside out after scaling -1 on z?

[GameMonetize]

Actually no, this works well (you can test it easily)

[kpko]

Ah, because the normals are not influenced by the scaling, I guess, they are laid out separately in the babylon.js file. I guess the problems lies in the composition / decomposition code itself. 

 

Thank you very much so far for clarifying those points! :-) Just wanted to make sure it's not one of those more obvious things before I disassemble my current interpolation code.

 

I used the following code to decompose the incoming matrix:

public decompose(scale: Vector3, rotation: Quaternion, translation: Vector3) {    translation.x = this.m[12];    translation.y = this.m[13];    translation.z = this.m[14];    var xs = Tools.Sign(this.m[0] * this.m[1] * this.m[2] * this.m[3]) < 0 ? -1 : 1;    var ys = Tools.Sign(this.m[4] * this.m[5] * this.m[6] * this.m[7]) < 0 ? -1 : 1;    var zs = Tools.Sign(this.m[8] * this.m[9] * this.m[10] * this.m[11]) < 0 ? -1 : 1;    scale.x = xs * Math.sqrt(this.m[0] * this.m[0] + this.m[1] * this.m[1] + this.m[2] * this.m[2]);    scale.y = ys * Math.sqrt(this.m[4] * this.m[4] + this.m[5] * this.m[5] + this.m[6] * this.m[6]);    scale.z = zs * Math.sqrt(this.m[8] * this.m[8] + this.m[9] * this.m[9] + this.m[10] * this.m[10]);    if (scale.x == 0 || scale.y == 0 || scale.z == 0) {        rotation.x = 0;        rotation.y = 0;        rotation.z = 0;        rotation.w = 1;        return false;    }    var rotationMatrix = BABYLON.Matrix.FromValues(        this.m[0] / scale.x, this.m[1] / scale.x, this.m[2] / scale.x, 0,        this.m[4] / scale.y, this.m[5] / scale.y, this.m[6] / scale.y, 0,        this.m[8] / scale.z, this.m[9] / scale.z, this.m[10] / scale.z, 0,        0, 0, 0, 1);    rotation.fromRotationMatrix(rotationMatrix);    return true;}

which I based on the matrix decomposition code from MonoGame: https://github.com/mono/MonoGame/blob/develop/MonoGame.Framework/Matrix.cs#L1055-L1082

 

I use the builtin vector and quaternion interpolation functions of Babylon.js to interpolate the individual values. After doing that, I compose the matrix again which the Compose method:

public static Compose(scale: Vector3, rotation: Quaternion, translation: Vector3): Matrix {    var result = Matrix.FromValues(scale.x, 0, 0, 0,        0, scale.y, 0, 0,        0, 0, scale.z, 0,        0, 0, 0, 1);    var rotationMatrix = Matrix.Identity();    rotation.toRotationMatrix(rotationMatrix);    result = result.multiply(rotationMatrix);    result.setTranslation(translation);    return result;}

I will run a few tests later this week, being a little busy because of my current main projects. I really want this to work though, we have an upcoming project which was supposed to be a windows application. BabylonJS could help us bringing the software to the web which our solution would really benefit from. We rely heavily on skeletal animation though. If you guys have any ideas, feel free to share them with us :-)

[GameMonetize]

Sounds good at first glance