Transparency & alpha issues

[acarvallo]

Hello Babylon.js people,

I am trying to make a viewer for working with BIM stuff (building and all the stuff inside), so there can be quite a lot of objects and having each one in its own mesh isn't really an option.

So I tried to use two SPS, a single mesh and two meshes, one mesh/sps for opaque objects and another for transparent ones (saw the idea in another post on the forum).

Thing is I need to "highlight", hide and have transparent not pickable objects too.

Here is what I have for now :

There is a space (big transparent blue-ish), with objects in it (yellow-ish opaque ones in the front) and also transparent grey "boxes".

To be able to switch objects visibility/color quickly I tried to put all objects in both meshes :

• opaque ones hidden in the transparent mesh
• transparent ones hidden in the opaque mesh

Then just toggle visibility, but then this kind of thing can happen :

The front box is now transparent with a red color, the other "yellow-ish" box is partly hidden with a red tint (that's correct), but the grey opaque box isn't hidden/tinted (that's not ok) at all and keeps its grey color, looking like it is closer to the camera while it is actually not.

I feel like I tried quite a lot of things and combinations so I may have missed the working one.

 

I also posted an issue trying to have a simple example to demonstrate what felt like a bug : https://github.com/BabylonJS/Babylon.js/issues/2832

As I wrote there, using needDepthPrePass kind of fixes it, but not really since it messes up transparent objects inside another transparent object.

Alpha mode is promising but the ghost/shiny effect is too powerful :

I would need the behaviour, without the side effect...

Any help/suggestions is welcome, and if you have questions or need more details, feel free to ask

 

References :

https://doc.babylonjs.com/resources/transparency_and_how_meshes_are_rendered

https://doc.babylonjs.com/resources/how_to_use_blend_modes

 

Edited April 6, 2018 by acarvallo
Update dead links (tutorials > resources)

[GameMonetize]

Hello in your case the only way out seems to be using premultiplied alpha mode

 

something along these lines: https://www.babylonjs-playground.com/#JB6H3P#3

Check line #19,  63, 64, 65

[acarvallo]

Well, I see more or less the idea (the ALPHA_PREMULTIPLIED isn't on the page about blend modes though), and I guess the scale() on the color is to counter the ghost/glow effect, produced by the ALPHA_PREMULTIPLIED :p

Side note, I didn't include in my first post, my objects can have different vertex colors, when I add an object, I create a blank mesh, assign a BABYLON.VertexData with positions, indices, normals, colors and then add that mesh to the SPS, setting computeParticleColor to false.

I multiplied each vertex color by its matching alpha though, and it is kind of working, here is the behaviour (a video is better than trying to explain with words) :

https://sendvid.com/0wsdq652

In short, it works if I start by highlighting one of the grey box behind.

Here is how I "highlight" an object :

    var colors = new Float32Array(vertexCount * 4);
    for (var j = 0; j < vertexCount; j++) {
        colors[j * 4 + 0] = color.r;
        colors[j * 4 + 1] = color.g;
        colors[j * 4 + 2] = color.b;
        colors[j * 4 + 3] = color.a;
    }

    SPST.mesh.geometry.updateVerticesDataDirectly(BABYLON.VertexBuffer.ColorKind, colors, startIndex * 4 * 4);

    // If object is opaque
    if (obj.geometry.sps.name == "SPSO") {
        var pIndex = obj.geometry.particleIndex;
        SPST.particles[pIndex].isVisible = true;
        SPST.setParticles(pIndex, pIndex);
        SPSO.particles[pIndex].isVisible = false;
        SPSO.setParticles(pIndex, pIndex);
    }

And the "unhighlight" :

    // Omitted, colors is filled with original vertex colors for the matching object

    if (obj.geometry.sps.name == "SPSO") {
        var pIndex = obj.geometry.particleIndex;
        SPSO.particles[pIndex].isVisible = true;
        SPSO.setParticles(pIndex, pIndex);
        SPST.particles[pIndex].isVisible = false;
        SPST.setParticles(pIndex, pIndex);
    } else {
        obj.geometry.sps.mesh.geometry.updateVerticesDataDirectly(BABYLON.VertexBuffer.ColorKind, colors, startIndex * 4 * 4);
    }

So basically, the scene is always correct at first, it's just wrong when switching alpha/visibility, again thanks for any reply, and if you have a better way of doing what I want to achieve I'll be glad to read your suggestions

(Also replied in the issue)

[GameMonetize]

I'm sure we can do better

We just need a repro in the PG. We have several weapons to tackle this issue (forceDepthWrite and separateCullingPass were added just for that :))

[acarvallo]

Well, it would appear that after trying a bunch of things I messed up . I was basically doing this when adding meshes as particles in the SPS :

// If original mesh is transparent
if (shape.material.alpha < 1.0) {
    SPSO.particles[index].isVisible = false;
} else {
    SPST.particles[index].isVisible = true;
}

So opaque original meshes added resulted in both opaque & transparent particles being drawn, which leads to strange transparency issues.

Just changing previous code, replacing "SPST.particles[index].isVisible = true;" by "SPST.particles[index].isVisible = false;" fixes my issue, but create another one...

Now I just have a "blink" effect when I highlight opaque particles, like this : https://sendvid.com/ogdv46mn

And *sigh* I finally found the issue by debugging, it was because the bounding box/sphere was empty for the SPST in case (as it was in the video I linked just before) there are only opaque objects, I tried setting

SPST.computeBoundingBox = true;

But if I call the following function (refreshVisibleSize();) each time I switch visibility of a particle, it works !

SPST.refreshVisibleSize();

 

So sorry for my mistakes, thanks for the implication, I really appreciate it.

One more question, shouldn't setting computeBoundingBox to true refresh the bounding box or is it because I call setParticles(id, id) (so updating a single particle and not updating whole SPS's particles) ?

[GameMonetize]

This last question is for @jerome for sure

[jerome]

Let's summarize :

- Adding a shape to the SPS only copies the shape geometry (vertices, indices, normals, uvs, colors). The shape material isn't copied. As the alpha is a property of the material of the shape, it's ignored by the SPS. That said, you can then set your own material to the SPS or play with each particle color/alpha (managing the particle color4 property and setting the SPS mesh property hasVertexAlpha to true)

- As its name says, the particle property .isVisible just makes the particle visible or not whaterver other considerations.

- SPS.computeBoundingBox = true jus enables to global SPS mesh BBox internal computation within the next call to setParticles() instead of having another side computation. This is faster and this is useful for SPS whose sizes evolves with the time (an explosion for instance). This is related to the global SPS mesh visibility/pickability relatively to the camera frustum, this has nothing to do with a given particle blink.

NOTE (maybe here's you answer) : when calling setParticle(id, id) the particle loop doesn't recompute everything but only the passed particle range (here only one : id), so the whole Bbox can't be rightly updated and would have an inconsistent value (maybe I could auto-disabled it here, not sure). A forced global computation will be needed with refreshVisibleSize(). You could also disable before the property computeBoundingBox so its last computed value is kept or set a fixed value if the SPS size doesn't change, instead of calling refreshVisibleSize().

- SPS.refreshVisibleSize() forces the immediate the SPS global mesh Bbox recomputation. This is an expensive operation, not needed when using computeBoundingBox. This is useful to compute once the Bbox of a SPS that reached its maximum size and won't evolve then, after some particle init for instance (and thus to disable computeBoundingBox). As formerly, this concerns only the global vsibility relatively to the camera frustum and not a given particle.

Not sure to help.

A PG would be helpful.

In your video, the bug doesn't seem to be related to some SPS visibility (so BBox) issue, because only one particle blinks at once. I would say it has something to do with the call to setParticles(i, i) .... just guessing. Do you have the same effect by calling simply setParticles() ?

.

[acarvallo]

Hello (sorry for the late reply, been a bit busy),

First, finally was able to put part of my viewer in a PG: https://playground.babylonjs.com/#MNZ2VS

Found what was going wrong again, before starting the render loop I was copying the SPS (without their meshes being built), and building the mesh of the copy (but the copy was dropping get/set prototypes...), so setting computeBoundingBox did nothing (not setting the actual _computeBoundingBox attribute).

So all is good on Babylon's side regarding bounding box stuff, and what is depending on it (ray/intersect).

It is now working well with my 2 SPS (but is not optimal for performance).

Then I wanted to try with only a single one (having both opaque & transparent objects in it), using forceDepthWrite=true on the material used for the SPS seems to do wonderful !

Only thing is that for some cases (one opaque object in front of another), setting the front one as transparent won't make the back one visible at all (see the PG for that).

Also, as of now I feel like the separateCullingPass property of a material does nothing, I can't really spot a change when setting its value to false in the PG of that comment : https://github.com/BabylonJS/Babylon.js/issues/2832#issuecomment-331647665

[GameMonetize]

separateCullingPass  first render the back face then the front face to get a bit of face sorting.

Do you have a PG with separateCullingPass used?

[acarvallo]

Hi,

Yes the one from the issue, when you specifically mentioned that it could be used to maybe solve my problem : https://www.babylonjs-playground.com/#JB6H3P#5

As I also said, forceDepthWrite really helps but is not perfect for my real usage example, and setting separateCullingPass doesn't seem to change it as well : https://playground.babylonjs.com/#MNZ2VS#1

[GameMonetize]

Hey

can you try to isolate the issue a bit more? Or point me precisely to the issue you have?

[acarvallo]

Yes I can : https://sendvid.com/3yjykjzf

You don't see the grey box behind, when the front one becomes transparent, and you don't see it as well when looking through one of the yellow-ish ones for example.

That is my issue , not /always/ seeing through transparent particles inside the same SPS.

[GameMonetize]

Yeah unfortunately I see no easy way to deal with a SPS here

With separated meshes, this will not be a problem at all but inside the same SPS it could be complex.

So why not consider that the SPS is not transparent and when you want to highlight a piece of it, you just need to hide it and instead display a transparent mesh. So the SPS is not transparent and the transparent mesh will be correctly displayed

 

(Does it make sense?)

[acarvallo]

It totally makes sense, unfortunately the PG is just a part of my geometry, I can have a lot of objects both transparent and opaque for their initial state.

I used a SPS because it makes picking easier thanks to particles/pickedParticles, but the thing I'm really looking for is performance, may it be using a SPS, a regular mesh or something else.

Maybe you have another idea or magic property (like forceDepthWrite) in your bag of knowledge that could help ?

 

Again, thanks for the answers !

[GameMonetize]

The problem we are trying to face here is clearly the need for face sorting. It is a really old problem in realtime 3d. and there is no good solution so far.

The sps is only one mesh so you cannot have face sorting at all.

Perhaps you could consider having 2 sps: one solid and one transparent?. The transparent one could be set to use forceDepthWrite and separateCullingPass

[jerome]

Maybe this will help for the SPS : https://github.com/BabylonJS/Babylon.js/pull/2961

// create a particle depth sort enabled SPS
var sps = new BABYLON.SolidParticleSystem("sps", scene, {enableDepthSort: true});
// then later only ...
sps.setParticles();   // and the particle are depth sorted each call
// We can skip the sorting at any time (or reactive it) : sps and camera not moving anymore
sps.depthSortParticles = false;  // true by default when enableDepthSort is set to true

 

Documentation and PG to come

[GameMonetize]

YOU ARE FREAKING ROCKING MAN!

[jerome]

Let's start with a simple PG, a SPS populated with transparent solid particles : http://playground.babylonjs.com/#EPBTB7#3

As you can notice when rotating the camera, because the SPS is a standard mesh and because transparent standard meshes don't sort their faces according to the camera position for performance reasons, the transparent particles are weirdly rendered from some points of view. This is unfortunately the expected behaviour when dealing with "auto-transparency" (transparency applied to itself).

Now the SPS provides a new feature called the particle depth sort : http://playground.babylonjs.com/#EPBTB7#2

The internal SPS geometry is recomputed each frame so the most distant particles are drawn before the closest ones.

Note this process is CPU intensive so it's done only at the particle level and not at the facet level.

Note also that, for now, the distance between the camera global position and the particle position is computed only in the SPS local space. This means that if the SPS mesh is scaled, translated or rotated, this distance is no longer pertinent. So, for now, it's better to set up your SPS internal particle positions, rotations, etc, like these were World ones and to let the SPS unrotated, unscaled at the origin (it's designed to do this very well actually).

I need to think about the opportunity to add the SPS global transformation (translation, rotation, scaling) to the current particle location to get a septh sort working in every case. It would be much more computation, so not sure it's worh it.

[EDIT] I just did a PR fixing this at a decent performance. Now the SPS can be rotated or translated and the particle will keep sorted from the camera as expected.

[jerome]

@acarvallo please let us know if this parameter now helps to solve your transparency issue

[acarvallo]

Hm, I should have activated notifications, could have answered earlier :|

So,  thanks @jerome for the SPS particle depth sort, but it does nothing in my case due to all my particles's position being Zero (only using position buffer from the original mesh for each particle).

And you're absolutely right about having 2 SPS @Deltakosh, it's indeed what I have in my actual project, but I can't seem to have a perfect (or decent) looking render.

Here is a playground to see what's going on : https://playground.babylonjs.com/#BQLAVV#2

And 2 screenshots, camera looking from one side, then the opposite one : 

I would basically like to have the render on the right be of the same type as the one on the left.

Tried to play a bit with depth/culling options and alphaMode (as you can see starting from line 48), but I couldn't find a really good combination.

 

[jerome]

maybe this one then : http://doc.babylonjs.com/how_to/how_to_use_facetdata#facet-depth-sort ?

[acarvallo]

Hi, thanks for the link, I wasn't aware of this feature !

Updated PG : https://playground.babylonjs.com/#BQLAVV#3

It's for sure better, but there still seems to be some artifacts :

It's strange because they should all be similar elements,  so I would think they will render the same. I guess it's because the faces touching each other have the same positions.

Not sure if there is much more doable with facets data, another possible option could be to depth sort particles, but use, for example, the center of the bounding box of each particle instead of its position.