[SOLVED] Wrong texture passed in uniform to my shader

[PeapBoy]

Hi everybody ! 

This week I faced with a strange behaviour. I generally lose time looking for a solution on my own, I did it again, but today I have the unpleasant feeling to be powerless. 

I'm trying to precompute kind of PCSS map (soft shadows) to get nice-looking shadows in static scenes.
For this purpose, I use renderTargetTextures with refreshRate = RENDER_ONCE.
And I use three shaders, called in this order :

1. The one of shadowGenerator which gives me the shadowMap of the blockers.
2. The PCSS one, which uses the shadowMap and gives me a PCSSMap.
3. The material's shader, which simply display the PCSSMap in real time.

After each renderTargetTexture created, I call scene.customTargets() to order the calls.
Okay : This works great ! 

Now, I would like to correct small artifacts and I need to repeat step 1 & 2 for each blocker separatly.
And here comes the drama.

Let's take a look at the new call order :

• 1. ShadowGenerator creates blockerMap with the first blocker.
• 2. PCSSGenerator creates PCSSMap for the first blocker.
• 1bis. ShadowGenerator creates blockerMap with the second blocker.
• 2bis. PCSSGenerator creates PCSSMap for the second blocker AND mix it with last PCSSMap (the first one).
• 1ter. ShadowGenerator creates blockerMap with the third blocker.
• 2ter. PCSSGenerator creates PCSSMap for the third blocker AND mix it with last PCSSMap (the second one).
• 3. Display result.

My issue is : I can't grab the last PCSSMap.

After a few tests, I highlighted when my issue appears and when it doesn't.

So here is a big simplification of my PCSS fragment shader (it only outputs one of the two textures it takes in uniform) :

uniform samplerCube blockerMap;
uniform samplerCube previousPCSSMap;

// This function samples the blockerMap. We don't mind about the result.
float sampleFunction() {
 for (int i = 0.0; i < POISSON_COUNT; i++) {
  vec4 sample = textureCube(blockerMap, direction);
 }
 return 1.0;
}

void main(void) {
 // To fill
}

And here are 4 usecases :

1. It returns blockerMap, everything is OK.

void main(void) {
 //sampleFunction();
 gl_FragColor = textureCube(blockerMap, direction);
 //gl_FragColor = textureCube(previousPCSSMap, direction);
}

2. It returns the previous PCSSMap, everything is OK.

void main(void) {
 //sampleFunction();
 //gl_FragColor = textureCube(blockerMap, direction);
 gl_FragColor = textureCube(previousPCSSMap, direction);
}

3. It returns the blockerMap, everything is OK.

void main(void) {
 sampleFunction();
 gl_FragColor = textureCube(blockerMap, direction);
 //gl_FragColor = textureCube(previousPCSSMap, direction);
}

4. It returns the blockerMap instead of the previous PCSSMap, what's wrong ?

void main(void) {
 sampleFunction();
 //gl_FragColor = textureCube(blockerMap, direction);
 gl_FragColor = textureCube(previousPCSSMap, direction);
}

As you can see, sampleFunction() works with blockerMap and has absolutely no contact with the previousPCSSMap. However, the previousPCSSMap seems to be replaced by the blockerMap and I have absolutely no idea how it's possible.
As it's nonsense, I dare coming here begging for your help... 

Some more info :
- I use shadowGenerator from Babylon.
- I use my own PCSSGenerator. But this one is a CC of shadowGenerator. The unique difference is the shader which is called.
- The last shader (material's one) only displays the result, the issue should not come from here.
- I verified 1000 times, I don't send blockerMap in previousPCSSMap in my code. Maybe it does in a dark side of the library, but I don't think so.
- I systematically empty my cache between each shader modification.
- Of course, my PCSS shader contains a lot of calculations and uniforms I didn't show here. But I really commented a lot my code to obtain something really close to the usecase above. I'm working on isolating the issue in a new project.

Thanks ! 

PeapBoy

[Pryme8]

whoa, um I think we have a way to prebake shadow maps already @JCPalmer might know.  If I had more time this morning I would be on this for you as I need to do it at some point here in the very near future.

[PeapBoy]

Hi Pryme8 !

It could be very interesting, I would be glad to hear about it.
Though, I'm not sure the issue comes from shadows in general, but just in my way to order and use successive shader passes. (Maybe I should take a look to how works PostProcesses in BabylonJS).

 

[JCPalmer]

No, sorry I never needed to do that.  I vaguely recall a performance thing where you can turn it off after done once.  Not a pre-bake, but close.

[Pryme8]

Ohh I get it now, is it because the previous one is being updated faster then the GL can output, and is referencing the wrong one?

Why do you need the previous one? Why not just modify your shader to do the multi passes that your speaking of?

Sorry if im off basis im just glancing at this as I try to get work done this morning.

But nice to meet you btw, looks like your doing some decently high end stuff and we love guys like you

[Pryme8]

Just now, JCPalmer said:

No, sorry I never needed to do that.  I vaguely recall a performance thing where you can turn it off after done once.  Not a pre-bake, but close.

Yeah thats kinda what I thought was how it was done, Im trying to remember who was doing the prebaked shadows in the scene... I thought it was you my bad.

[PeapBoy]

4 minutes ago, Pryme8 said:

6 minutes ago, JCPalmer said:

No, sorry I never needed to do that.  I vaguely recall a performance thing where you can turn it off after done once.  Not a pre-bake, but close.

Yeah thats kinda what I thought was how it was done, Im trying to remember who was doing the prebaked shadows in the scene... I thought it was you my bad.

Are you talking about the refresh rate of the renderTargetTexture which is used for shadows particularly ? :')

 

6 minutes ago, Pryme8 said:

Ohh I get it now, is it because the previous one is being updated faster then the GL can output, and is referencing the wrong one?

Why do you need the previous one? Why not just modify your shader to do the multi passes that your speaking of?

Sorry if im off basis im just glancing at this as I try to get work done this morning.

But nice to meet you btw, looks like your doing some decently high end stuff and we love guys like you

My lack of knowledge about how GL behaves internally is maybe the main problem.
So I'm not really sure to understand your first note.

Doing all the stuff in only one PCSS pass works fine. But I saw some artifacts. And I'm quite sure there is no way to get rid of these artifacts in one pass because these artifacts are totally due by the volonty to prebake PCSS, they don't appear in real time PCSS.
So I'm trying to do something really energy intensive : Bake PCSS for each blocker separately. I would only have to mix all my PCSSMaps together at the end and bye bye the artifacts.
Rather than keeping in memory all the maps and mix them at the end, I just mix immediately the new map with the last one. It remains the same.
I can't deal with each blocker separately in the same shader pass, can I ? If I can do the multipasses in my shader, I would be very interested in how to do it ! 
By blocker, I mean each mesh taken into acount in shadow generation.

Thank you a lot to take time about that haha ! 

I'm still a newbie in general. But I work on PCSS and possible optimisations for two months so maybe I will finally get interesting results. 

[Pryme8]

what about maybe running the shadows once, and then saving the buffer from that, and then maybe do it again and then combine the results of the two somehow?

Im still kinda shaky on what it is your trying to accomplish, you just want nice soft shadows in your scene that are not being recomputed every frame right?

[GameMonetize]

Interesting! Do you mind sharing it in the playground? I would love to help but I need a repro case

[PeapBoy]

I can't really explain why I need to work with each blocker separately. Because I would need to explain all my code.
Prebaking PCSS (instead of using the real time one of nVidia) involves a compromise : a mesh can't be blocker and receiver in the same time. This compromise leads to small artifacts in case of self-shadowing and in case of blocker receiving shadow after all.
I tried some tricks to get rid of these artifacts, and everything failed.
Repeating PCSS computation for each blocker separately, rather than working with the original shadowMap (with all the blockers, hiding each other), is really dumb (too energy intensive, it won't work with a lot of blockers). But it will resolve all the artifacts for sure.

So, if I understand well. Running the shadows once means calling shadowGenerator (for all blockers) once. Saving buffer means saving the shadowMap.
But then I can't combine the results, because I already lost some precious info : the depth of blockers hidden behind the closest blocker (which depth is in the shadowMap).

 

29 minutes ago, Deltakosh said:

Interesting! Do you mind sharing it in the playground? I would love to help but I need a repro case

I'm working on reproducing the case in a separate, really minimal, project. If the problem appears again, I'll try to make a PG ! 
But I'll do that tomorrow, it's night here. 

[PeapBoy]

Right !

Hello guys 

Here is the PG : http://www.babylonjs-playground.com/#17Y5UV#17

At the top of the code, there is the shader I use to receive and output the shadowMap (I called it PCSS, but it absolutely doesn't perform PCSS here )
Then, there is the PCSSGenerator prototype, which simply herits from ShadowGenerator and changes its customRenderFunction to fit my needs.
And finally, the createScene() function.

Look for "CUSTOM PART" words to see relevant code.

So, what am I doing here ?
I'm just trying to chain shaders to highlight my problem.
First I create a shadowMap1 with ShadowGenerator, then I pass the shadowMap1 to PCSSGenerator which does nothing else than outputing the shadowMap1, now called PCSSMap.
Secondly, I create another shadowMap (2) (of another mesh, to see the difference), then I pass it to another PCSSGenerator.
This time, I ask the PCSSGenerator to output the last PCSSMap, rather than the shadowMap2.
Finally, I never get the PCSSMap, but always the shadowMap2.
 

I'm quite disappointed. I'm clearly doing something wrong, since in the PG, we can see it doesn't work at all.
But in my project, I have the strange behaviour I explained above : sometimes it works, sometimes it doesn't, which made me think I wasn't really doing nonsense.

Hope I'm clear. Thank you for your help 

PeapBoy

[Nabroski]

Hello

I kind of scroll trough this post, two things:

1: I would code a very basic shader and see how it works

2: http://www.babylonjs-playground.com/#17Y5UV#18

Also here

 

[PeapBoy]

On 21/10/2016 at 11:39 PM, Nabroski said:

Hello

I kind of scroll trough this post, two things:

1: I would code a very basic shader and see how it works

2: http://www.babylonjs-playground.com/#17Y5UV#18

Also here

 

Hi @Nabroski !

Thanks for your help
I took a look at your link...
I'm really sorry, I should never have spoken about shadows. Because now everybody focus on it and 90% of the thread talks about that. 
My issue is not about shadows but about how Babylon (or just WebGL) references textures when chaining shaders.
And I faced with that issue when working on a specific version of PCSS, so I spoke about PCSS.

Actually, I just get the wrong texture in input of my shader, when calling shadowGenerators one after another and trying to use the result of the previous shadowGenerator in the next shadowGenerator (using a custom shader of course).

If anybody has an idea... I'm sure I'm just missing a basic WebGL internal behaviour or Babylon internal behaviour about texture referencing. 

PeapBoy

[Nabroski]

 

Just to make myself more clear, this rings look for me like you are using the wrong  shadowmap generator. 4 or 5 types in babylonjs are you sure you are using the right one.

[PeapBoy]

Oh.

I just didn't mind about shadow acne in this PG : bias equals 0.
I didn't want to add more code than needed. The custom shader  (without bias) replaces the original shadowMap shader.
In my project I don't have this.

[Nabroski]

Hello

just a question, what is the desired result ? 

[PeapBoy]

Hi,

I just made this to highlight where I speak about the "wrong texture I get".

 

To simplify things, the desired result is just to display an older renderTargetTexture. Generally it works. But in this specific usecase, I always get the wrong texture and I cannot figure out why.

[Nabroski]

You have to use the words customVertexShader, customFragmentShader otherwise you get a conflict with the default one, -- if i remember right. but this is only a playground thing

where is the documentation for this ?
this._scene.getEngine().createEffect("PCSS", attribs, ["world", "mBones", "viewProjection", "diffuseMatrix", "lightPos", "depthValues"], ["diffuseSampler", "blockerMap"], join);

can you recreate the playground with the first shader working ? i just want to see how it looks

[PeapBoy]

3 hours ago, Nabroski said:

where is the documentation for this ?
this._scene.getEngine().createEffect("PCSS", attribs, ["world", "mBones", "viewProjection", "diffuseMatrix", "lightPos", "depthValues"], ["diffuseSampler", "blockerMap"], join);

I'm taking a look at this, maybe there's something I didn't see.
 

3 hours ago, Nabroski said:

can you recreate the playground with the first shader working ? i just want to see how it looks

By "the first shader", do you mean the PASS 1 of my picture ?
It's the original shadowGenerator, so the original shadowMap shader.

And it gives this : http://www.babylonjs-playground.com/#17Y5UV#20

You can comment and uncomment each pass at the end of the PG ! 

[Nabroski]

Hello

can you also modify your PG with the words costom everywhere in the shader
i would do it by myself but i dont understand this this._scene.getEngine().createEffect("PCSS", attribs, ["world", "mBones", "viewProjection", "diffuseMatrix", "lightPos", "depthValues"], ["diffuseSampler", "blockerMap"], join);

like here
http://www.babylonjs-playground.com/#1YHPOU#0

 

[PeapBoy]

Wow ! It's possible to write the shader without the annoying quotes !? 
Sooooooo coooool ! 
I'll remember that for the next time, thank you !

Renaming PCSS by custom, we get that : http://www.babylonjs-playground.com/#17Y5UV#22

42 minutes ago, Nabroski said:

i would do it by myself but i dont understand this this._scene.getEngine().createEffect("PCSS", attribs, ["world", "mBones", "viewProjection", "diffuseMatrix", "lightPos", "depthValues"], ["diffuseSampler", "blockerMap"], join);
 

As you can see in the last PG, I replaced "PCSS" by "custom".

I didn't invent this line. PCSSGenerator is exactly the same as ShadowGenerator. I overloaded some of its functions to fit my needs. And actually, the unique difference with original ShadowGenerator on this line is the name of the shader we call : that's why I replaced "PCSS" by "custom".
This call creates the effect, so I think it means it inserts the defines, compiles the shaders and links the attributes and uniforms.

Oh wait...
I made a mistake in the PG !!!
(THANK YOU, you made me look at this line !)
I forgot to add "PCSSMap" to the sampler list.
Now it works : http://www.babylonjs-playground.com/#17Y5UV#23
But it works in the PG, still not in my project. So, actually I just lost my repro case 

In my project, the issue of "wrong texture referenced" seems to appear randomly (and I was surprised that it wasn't working at all in the PG, now I know why). So maybe I won't be able to reproduce the issue in the PG...

[Nabroski]

Hello

Please zip all relevant code and data (less is better) witch reproduces the issue and upload it somewhere share a link per pn or here in the forum. I also pass few deadlines and hope to get sleep for few hours. I will reply tomorrow.

Best 

[PeapBoy]

I'll try to do that !
Good luck for your deadlines.

[PeapBoy]

GOT IT !!!

Hello ! 

Trying to reproduce the bug, I found the mistake. But let's see the bug in action (yes, I finally could reproduce it !) :
Here is the repro case : http://www.babylonjs-playground.com/#17Y5UV#27

At line 49, comment and uncomment the line and... look at my nightmare.
As you can see, commenting or uncommenting an useless line, totally changes the shader behaviour !
That's exactly what I wanted to highlight in my previous posts.
Note that I slightly changed the PCSSGenerator to be as close as possible to my real project, even though it's harder to understand. Now, it includes the shadowGenerator pass, that's why there are only two passes left.

Now here is my mistake : I put the uniform "previousPass" (containing the last pass in a textureCube) in the uniform list... Instead of the sampler list.
I finally can display my last pass. 

Working PG : http://www.babylonjs-playground.com/#17Y5UV#28

 

Nevertheless, I'm really curious about this behaviour. If anybody understands this... It would be very interesting to understand why WebGL replaces a texture by another instead of throwing an error or a warning. And it would be even more interesting to understand why it links the good texture, or not,  when sampling it (that's what does the line to comment).

Note that my mistake didn't avoid the shader to receive the texture, it was passed as an uniform anyway, and we can be sure of that because when we comment the line, we see the good texture. That's why I still don't understand this behaviour.

Is there a shader parsing or anything like that, which involves a texture to be linked or not, whether it's sampled or not in the shader ?


However, thank you so much, you helped me a lot to find my mistake. 

PeapBoy

 

[GameMonetize]

samplers are actually pretty dumb, they just work in order of declaration and they are statefull

so if you declare samp1, samp2 and samp3, it is up to you to set the right texture in slot 1, 2 and 3

 

if you forgot something then the sampler will just read the currently bind texture