layerMask not restricting by camera

[JCPalmer]

Am playing with a 2 camera playground that I did not originally write.  Made it so there was a layerMask on one of the cameras that matched that of the 2 meshes.  Thought I should have only gotten 2 visible meshes, but both cameras saw the meshes & got 4.  What am I going wrong?

Thanks,

 

Jeff

http://www.babylonjs-playground.com/#1IG874#14

[GameMonetize]

Nothing wrong but by default camera has a mask of 0xFFFFFF which works with all meshes because the selection is done with a binary and:

 

if (cam.layerMask & mesh.layerMask) then render mesh

[JCPalmer]

Ok,

So as far as making a dialog system goes, dialog meshes can be excluded from the application's / scene camera by setting the layerMask of the scene camera with no bits lining up to the dialog meshes.

 

I also have a requirement to implement a modal stack of dialogs. Dialog panels not on the top of the stack can have layerMask of 0, and no one can see them.

 

Still have an issue of the dialog system's camera drawing scene meshes too, since default layerMask of Mesh is also 0xFFFFFF.  Perhaps the camera.viewport setting could help (not familiar with its use in BJS yet).  What if I positioned all dialog meshes near (0, 0, 100000)?  Then the dialog sys camera would have to be so far away from scene meshes, as to not be visible.

 

If a way cannot be found, will just have to switch off scene camera.

http://www.babylonjs-playground.com/#1IG874#15

[JCPalmer]

Thinking about it, switching the scene camera off will not get the scene meshes out of the dialog system's camera.  But looking at that 0xFFFFFF.  It is way too small to include everything.  Max of  signed integer is 0x7FFFFFFF.  I tried a mask of 0x10000000, but did not work.  Seems like the mesh default is so aggressive.   Could its scope be backed off by a few bits?  Or am I just missing something?

[GameMonetize]

Can you repro on the playground? I did not get why 0xFFFFFF is too small. If you want multi-layer dialogs, you can set 0x000001 to layer 1, 0x000002 to layer 2 and so on

[JCPalmer]

The issue is not the dialog system's meshes.  Want to be able to add this as an extension into a scene written by anyone.  I can restrict the extensions meshes, because I explicitly assigned them a layerMask.  Extension meshes only show up in the extensions camera, by temporarily setting the game camera's layer mask to something that will not show them.

 

The games meshes are assumed to be defaulting on layermask.  What can I do so that they do not show in BOTH cameras?  In repo, changed game camera mask to 0x000001. Repo has 3 meshes:

1. Colored ground representing game mesh, default mask (shows in both cameras)
2. Box representing dialog on stack, mask 0, seen by none
3. sphere representing current dialog, seen only by dialog camera.

http://www.babylonjs-playground.com/#1IG874#16

[GameMonetize]

For this specific case, the only solution I can see is to browse all meshes beforehand and set their layerMask

[JCPalmer]

I agree, except for trying to build the Dialogs far from & pointing the camera away from (0, 0, 0).

 

But cannot 1, 2, or 4 bits be carved out the Mesh default?  Defaulting Meshes would still be visible in millions of possible cameras.  Instead of:

public layerMask: number = 0xFFFFFFFF;

In the case of 4 bits:

public layerMask: number = 0x0FFFFFFF;

Think this would allow for 15 "special" cameras:

0x10000000

0x20000000

0x30000000

0x40000000

0x50000000

0x60000000

0x70000000

0x80000000

0x90000000

0xA0000000

0xB0000000

0xC0000000

0xD0000000

0xE0000000

0xF0000000

[GameMonetize]

No problem on my side

[JCPalmer]

Ok.  Just verified by explicitly setting game mesh & camera to proposed new default.  Then I set Dialog camera & current dialog to 0x10000000.  Left suspended dialog  at 0.  Only 2 meshes seen, by the appropriate camera.  http://www.babylonjs-playground.com/#1IG874#17

 

Expect PR with 1 line change to AbstractMesh.

 

Could also change the default to Camera, but is easy enough to change by the extension.

 

Am also thinking that I need a dedicated light.  I played with emissive materials, but did not like the results. Thought I might not want to distort the scene with light the game maker did not create.  Would not matter if the dialog was temporary.  Once I start this part, if I can figure out how to make sizing work, you could up small permanent dialogs.  Like maybe a little scoreboard with LCD digit meshes.

 

How does an includeOnlyWithLayerMask member for light sound?  Actually sounds more scalable than includedOnlyMeshes.indexOf.

    export class Light extends Node {        public includeOnlyWithLayerMask = 0;        public includedOnlyMeshes = new Array<AbstractMesh>();        public excludedMeshes = new Array<AbstractMesh>();        public canAffectMesh(mesh: AbstractMesh): boolean {            if (!mesh) {                return true;            }            if (this.includedOnlyMeshes.length > 0 && this.includedOnlyMeshes.indexOf(mesh) === -1) {                return false;            }            if (this.excludedMeshes.length > 0 && this.excludedMeshes.indexOf(mesh) !== -1) {                return false;            }                        if (this.includeOnlyWithLayerMask !== 0 && this.includeOnlyWithLayerMask !== mesh.layerMask){                return false;            }            return true;        }

[GameMonetize]

Sounds perfect!! Do you want to PR or I can include directly

[JCPalmer]

I already had it written.  PRed