Recalculate mesh shading

[focomoso]

I've searched the forums and found these threads:

But neither is helping me with my issue.

If you take a look at the playground here:

https://www.babylonjs-playground.com/indexstable#V3A6F8#1

I'm importing an stl. It looks great on import, but when scaled, the shading doesn't seem to update.

Is there a way to tell a mesh to update it's shading?

Thanks

 

[GameMonetize]

This is normal as the normals are pointing downwards

Just add a second light from the bottom: https://www.babylonjs-playground.com/indexstable#V3A6F8#2

 

[focomoso]

How do we get the normals to update, then? That would solve this.

I've implemented the updateFacetData() mentioned in a comment to one of the threads above, but it doesn't seem to take scaling into account. 

[GameMonetize]

Normal are correct just that your light is not pointing to the right direction 

[focomoso]

I'm not sure that's what's going on. Take a look at this playground. A regular sphere vs an imported flat sphere that's then scaled up to look regular:

https://www.babylonjs-playground.com/indexstable#V3A6F8#3

The shading looks very different between the two despite them both having flat shading. 

 

ps - this one is a little more apples to apples:

https://www.babylonjs-playground.com/indexstable#V3A6F8#4

[focomoso]

And here's one that's even more "apples to apples".

Both spheres here were created in Tinkercad. They are identical except that sphere-flat.stl was scaled down to 25% in the y.

As you can see, when the flattened sphere is scaled back up again, its shading is very different from the non-scaled sphere.

https://www.babylonjs-playground.com/indexstable#V3A6F8#6

I wonder if the stl import is doing something to the normals? I'm not sure what exactly is going on here.

[GameMonetize]

Ok I get your point

Here is the fix: You need to actually recompute the normals: https://www.babylonjs-playground.com/indexstable#V3A6F8#7

 

[focomoso]

Thankyou, thankyou, thankyou...

I knew it had to be something relatively simple. 

[focomoso]

Is there a way to recalculate the normals without baking the transform in? The baking process can be very slow on complex meshes (2 to 3 seconds) and it remakes the bounding box which is not ideal. 

[focomoso]

I think I have a solution. You just need to transform the vertices by the mesh's world matrix like so:

    var worldMatrix = mesh.computeWorldMatrix();

    var positions = mesh.getVerticesData(BABYLON.VertexBuffer.PositionKind, false, true);
    for (var i = 0; i < positions.length / 3; i++) {
      var idx = i * 3;
      var vertex = BABYLON.Vector3.TransformCoordinates(BABYLON.Vector3.FromArray(positions, idx), worldMatrix);
      positions[idx] = vertex.x;
      positions[idx+1] = vertex.y;
      positions[idx+2] = vertex.z;
    }

    var indices = mesh.getIndices();

    var normals = [];
    BABYLON.VertexData.ComputeNormals(positions, indices, normals);

    mesh.setVerticesData(BABYLON.VertexBuffer.NormalKind, normals, true);

 

It might be good to add an "updateNormals" to the Mesh class that does this so you don't have to bake the transform every time.

[GameMonetize]

Like the idea  fancy doing a PR?

[jerome]

I would sugget you to NOT create intermediate vector3 during the process by using TransformCoordinatesToRef() instead and by using a single temp Vector3

http://doc.babylonjs.com/classes/3.0/vector3#static-transformcoordinatesfromfloatstoref-x-y-z-transformation-result-rarr-void

something like this :

    var positions = mesh.getVerticesData(BABYLON.VertexBuffer.PositionKind, false, true);
      var vertex = BABYLON.Vector3.Zero();
      for (var i = 0; i < positions.length / 3; i++) {
      var idx = i * 3;
      BABYLON.Vector3.TransformCoordinatesFromFloatsToRef(positions[idx],positions[idx+1],positions[idx+2] , worldMatrix, vertex);
      positions[idx] = vertex.x;
      positions[idx+1] = vertex.y;
      positions[idx+2] = vertex.z;

    }

 

[focomoso]

So, it took me way too long to figure this out, but there's something strange going on with the way mesh.setVerticesData(...NormalKind...) works. It is exaggerating the normals along scaled axes. This is the root cause of the problem. The workaround is to calculate the normals by hand and then divide each component of the vector by the square of the scale in that direction (which is why it took me so long to figure out). This makes our normal updating function look like:

export function updateNormals(mesh, scene) {

  const worldMatrix = mesh.computeWorldMatrix(true);

  const scale = BABYLON.Vector3.Zero();
  worldMatrix.decompose(scale, new BABYLON.Quaternion(), new BABYLON.Vector3());

  var positions = mesh.getVerticesData(BABYLON.VertexBuffer.PositionKind, false, true);
  var normals = [];

  var v1 = BABYLON.Vector3.Zero();
  var v2 = BABYLON.Vector3.Zero();
  var v3 = BABYLON.Vector3.Zero();
  var normal = BABYLON.Vector3.Zero();

  for (var i = 0; i < positions.length / 9; i++) {

    v1 = BABYLON.Vector3.FromArray(positions, i * 9);
    v2 = BABYLON.Vector3.FromArray(positions, i * 9 + 6); // flipped
    v3 = BABYLON.Vector3.FromArray(positions, i * 9 + 3);

    normal = BABYLON.Vector3.Cross(v1.subtract(v2), v1.subtract(v3));

    normal.x /= scale.x**2;
    normal.y /= scale.y**2;
    normal.z /= scale.z**2;

    normal = normal.normalize();

    //  each normal pushed 3 times, once for each vert
    normals.push(normal.x);normals.push(normal.y);normals.push(normal.z);
    normals.push(normal.x);normals.push(normal.y);normals.push(normal.z);
    normals.push(normal.x);normals.push(normal.y);normals.push(normal.z);
  }
  mesh.setVerticesData(BABYLON.VertexBuffer.NormalKind, normals, true);
}

I'm happy to do a pr, but this probably isn't the best way to handle this. To me, this is a bug in the babylon code. Somewhere, when it scales a mesh, it's multiplying the normals by the scale twice (or three times) which is what's making scaled meshes look so strange.

[focomoso]

By the way, here's a playground of it working:

https://www.babylonjs-playground.com/indexstable#V3A6F8#15

[GameMonetize]

So getVerticesData does not change the data. It just returns the value of the required vertex buffer. The value is not affected by world matrix.

What you are doing is updating the normals before they will be moved to world with this line: https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/default.vertex.fx#L115

 

While it works in your case I'm not really convinced it is a general solution.

[focomoso]

I'll make the pr, then. It just seems strange that we have to "unscale" the normals before we move them to the mesh. 

[GameMonetize]

My point is: I'm not sure this is the good reasoning here. Will it work if you apply positioning, rotation AND non uniform scaling?

[Pryme8]

Why are you scaling the normals?  I've always thought they should just be a 1 unit vector, why all the extra calculations?

[focomoso]

Yes - that's the whole point. Without this, the normals are not correct, even if you explicitly call to recalculate the normals, they are exaggerated in the in the direction of the scale. This solution is the only way I have come up with to get the normals to be correct. This is why I think this is actually an underlying bug. I'll make a playground to show the problem.

[focomoso]

2 minutes ago, Pryme8 said:

Why are you scaling the normals?  I've always thought they should just be a 1 unit vector, why all the extra calculations?

Because if you don't scale the normals, they are scaled "elsewhere" and are incorrect. This is the bug I've been dealing with for days now.

[SvenFrankson]

Isn't it due to the fact the mesh worldMatrix is applied to the normal vector in the shader ? 

Something like vNormalW = world * vNormal , but I don't know where it happens when there's not Material assigned. Here maybe ? https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/default.vertex.fx#L115 maybe ? 

#ifdef NORMAL vNormalW = normalize(vec3(finalWorld * vec4(normalUpdated, 0.0))); #endif

[Pryme8]

https://www.babylonjs-playground.com/#FDNU12

the normals do not get scaled...  maybe its in the shader?

[focomoso]

Here's the issue layed out as best I can. In this playground:

https://www.babylonjs-playground.com/indexstable#V3A6F8#24

There are 4 spheres. The underlying geometry for the one on the left is a regular sphere, so there is no scaling applied and it looks as expected.

The other three are flat and have been scaled up 4x in the y to become spherical.

Sphere number 2 uses VertexData.ComputeNormals to explicitly calculate the normals. I would expect this method to produce a sphere with exactly the same normals as sphere 1, but it doesn't. The normals are scaled 2x more in the y than they should be (which is why the top and bottom of the sphere look darker). 

With sphere number 3 I calculate the normals explicitly, but again, they seem to be over scaled in the y, this time by the square of the scale factor.

With sphere number 4, I calculate the normals by hand, then apply the "descaling" and it looks as expected. 

The results are the same if you translate or rotate the spheres. It is only scaling that gives strange results.

It's possible that there's something wrong with my normal calculations, but I've been banging my head against this for a while and it seems that the way the normals are applied to the mesh is the culprit. 

[Pryme8]

https://www.babylonjs-playground.com/indexstable#V3A6F8#25

Take a look at the second sphere.

 

Its because your position calculations are off after the scaling.

https://www.babylonjs-playground.com/indexstable#V3A6F8#26

https://doc.babylonjs.com/overviews/how_rotations_and_translations_work
read the section on "Baking Transform"

[focomoso]

Thanks Pryme8, but the idea here is to find a solution that doesn't require baking the transform because it takes too long and destroys the local bounding box.