PBRMaterial with noise

[Rodrix3]

Hi there!

I am studying the trends in photorealistic 3d live rendering and I see a trend in applying random noise to the  material  fragment shader to increase the realism, which adds a "grainy texture".
I've seen it, it makes a GREAT difference.

My question is: is it possible to add noise to the current PBRMaterial?
If not,  could we add it? Is anyone interested in helping me modify PBRMaterial and add it as an additional feature?
 

I know it is possible to add a custom shader, but i wouldn't want to do a development that branches off PBRMaterial and miss any PBRMaterial updates;
unless, it is possible to add a shader code on top of the already PBRMaterial by injecting it into the current fragment shader.

@Deltakosh, what do you think of adding random noise feature to PBRMaterial?

P:s: the grain is not applied as a post-process full screen effect (as we currently have in the rendering pipeline). It is a material shader.

PREVIEW:  

[Rodrix3]

P.S.S: I have the shader code from ThreeJs, I think this could be easy to implement if this feature is not yet available. It is using Perlin noise.
If anyone is interested, I can post more information and help with the development.

[NasimiAsl]

hi that is possible but you most change that in fragment shader 

custom material is available for standard material ( some bug under progress too ) but for PBR material is not yet

may be this sample help you to do it in shaderBuilder

http://www.babylonjs-playground.com/#7AIII8#26  without noise

http://www.babylonjs-playground.com/#7AIII8#28

 

http://www.babylonjs-playground.com/#7AIII8#14 other sample of PBR

http://www.babylonjs-playground.com/#7AIII8#29

http://www.babylonjs-playground.com/#7AIII8#32

http://www.babylonjs-playground.com/#7AIII8#33

 

 

 

 

[Rodrix3]

@NasimiAsl thanks for all the contribution; however, my proposal is to add this as a standard feature of PBR Material,
and do a pull request. I am not sure what is the standard way of proposing a new feature and implementing it, so I present the proposal and wait to hear from you guys to know what is the standard workflow and if you think this feature is a good idea or not!  

SPECIFICATIONS:

• PBRMaterial shall have a new parameter called noiseIntensity which shall default to 0 (off by default - backward compatible)

This is the basic implementation:
 
FRAGMENT SHADER: 
uniform float noiseIntensity;

if(noiseIntensity>0){
  float n = noiseIntensity * ( .5 - random( vec3( 1. ), length( gl_FragCoord ) ) );
   gl_FragColor = gl_FragColor + vec3( n ), 1. );
}

 

..what do you think? Is this a good idea? It seems pretty simple to implement, and these are the results (if I interpreted the code right  / I gave a quick look):
https://www.clicktorelease.com/code/spherical-normal-mapping/ (ThreeJs / Author: Spite)

[Rodrix3]

Could anyone point me to where I can find the source of the PBRMaterial so I can make modifications?
I see nothing here.. https://github.com/BabylonJS/Babylon.js/tree/master/materialsLibrary/src

Thanks!

[Guest]

Pinging @Sebavan to investigate

[Guest]

BTW, I think it could be a great idea but I don't see clearly the difference with and without

can you provide both images?

[Rodrix3]

Hi @Deltakosh 

Sure!

Give me some time so I download the original sample from ThreeJs as see if I can turn it off to clearly show the difference.
However, meanwhile, let me give you insight into ANOTHER usecase of grain noise.

Usecase 2: Making low quality textures look realistic or making high quality textures look realistic when stretched accross very big surfaces. This is useful for flight simulators that have gigantic textures
and need textures to look good enough when the camera approaches the ground. The way they do it is they use a gigantic 8K texture for the whole island,
and then add on top random noise so that when the texture is looked up close and lacks resolution - the details are "faked" via the noise:
 

Original texture of whole island at 4K resolution:

 

Close up of flight simulator as it approaches the ground. Left: no grain. Right: grain shader on.

 

Source: https://gamedevelopment.tutsplus.com/tutorials/creating-realistic-terrain-for-html5-games-with-webgl--cms-24539

(includes Shader implementation code)
 

[Rodrix3]

Here you go @Deltakosh

Usecase 1: Photorealistic rendering:

The effect is very subtle but at 1080 the overall impression is BIG. The image on the right looks much more realistic and it is hard to tell why, until you zoom in and see those imperfections.

I suggest you download the images I sent, since the forum resizes them.

The random function used for the noise is calculated via a seed and dependent on the frag coord so it doesn't "move" and makes it a "detail" of the texture instead of a moving noise:

float random(vec3 scale,float seed){return fract(sin(dot(gl_FragCoord.xyz+seed,scale))*43758.5453+seed);}

Full implementation on ThreeJs is here: https://www.clicktorelease.com/code/spherical-normal-mapping/

 

[Sebavan]

Hello, If you are speaking about grain we already support it as a post process to ensure a nice consistency in full screen like your suzanne head picture:

https://www.babylonjs-playground.com/#10D6YT#120

Doc: https://doc.babylonjs.com/how_to/using_default_rendering_pipeline#grain

Now if you would like more a dithering feature as we are doing in the background material I can easily port it into the pbr and standard material ?

 

 

 

[Rodrix3]

57 minutes ago, Sebavan said:

Hello, If you are speaking about grain we already support it as a post process to ensure a nice consistency in full screen like your suzanne head picture:

https://www.babylonjs-playground.com/#10D6YT#120

Doc: https://doc.babylonjs.com/how_to/using_default_rendering_pipeline#grain

Now if you would like more a dithering feature as we are doing in the background material I can easily port it into the pbr and standard material ?

 

 

 

Hi @Sebavan 

Thanks for checking into this.

I was already aware that there is a grain post process.
However, this new suggested feature is a material shader feature, not a full screen, as you mention in your last line.

It is also important that the dithering / grain feature is static and does not move. It should be relative to the frag coord and using a random number based on seeds, so that the grains are always in the same position.
That type of code is on the links I added at the top.

Let me know what you think and if you would like my help in anything I can  
 

[Sebavan]

Being static from the coord is exactly what the grain is doing. As you want it to be subtle is exactly similar to the one we have in the background material: https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/background.fragment.fx#L294

If that works for you I will integrate it in the materials tomorrow as I would like to integrate it as part of our image processing configuration ?

[Guest]

Why not jus using the PP then? It makes sense to have a fullscreen process for the grain no?

[Rodrix3]

@Deltakosh, well it is different.

If grain is on post process then the grain is "on the screen" and if you rotate the object the grain doesn't rotate.
It is like if the grain was on the lens.

However, if the grain is on the material shader, then the grain is on the surface of the object. It is part of the material.
When you rotate the object, the grain moves with the object and the grain becomes part of the detail of the material instead of being a "trick" done to the screen.

In my perspective, the detail achieved with the second technique is far superior and has many uses, while the post process looks "faker".
Of course, both techniques CAN be combined like in the JS demo I sent that uses grain ON material AND on post process.
Let me know what you think

>>If that works for you I will integrate it in the materials tomorrow as I would like to integrate it as part of our image processing configuration ?
Does that mean not being able to control the grain PER material? I think being able to control how much grain each material has would be important for usecase 2.

[Sebavan]

It still means it can be control per material and it uses the world position which is equivalent to the frag coords in term of stability.

The only drawback is it is not stable during animation so I might use smthg a bit different for the input.

Would that work for you ?

[Rodrix3]

2 minutes ago, Sebavan said:

It still means it can be control per material and it uses the world position which is equivalent to the frag coords in term of stability.

The only drawback is it is not stable during animation so I might use smthg a bit different for the input.

Would that work for you ?

Thanks so much for taking my input into consideration @Sebavan. I really appreciate it

I suggest:

• Control per material
• noiseIntensity as parameter (default as 0.0 - disabled).
• Should work for moving or rotating objects so I suggest using frag coords instead of world position.

That would work great for me.

Thanks and let me know in what I can contribute.

Spite (developer) from ThreeJs used this:
float random(vec3 scale,float seed){return fract(sin(dot(gl_FragCoord.xyz+seed,scale))*43758.5453+seed);}
float n = noiseIntensity * ( .5 - random( vec3( 1. ), length( gl_FragCoord ) ) );
 gl_FragColor = gl_FragColor + vec3( n ), 1. );

Source: view-source:https://www.clicktorelease.com/code/spherical-normal-mapping/

This might be useful; although go ahead and use any implemtation you prefer.
Thanks again

[Rodrix3]

Alternatevely, we could use Perlin noise, but I don't think that is the "grain" effect we want to achieve as I am quite sure it is used for clouds, waves, and other procedural textures.
However, if you consider Perlin noise a good idea here is the shader: https://medium.com/neosavvy-labs/webgl-with-perlin-noise-part-1-a87b56bbc9fb

[Sebavan]

I would not rely on perlin noise to prevent the inline cost or the extra texture channel. About gl_FragCoord, the main issue is this will still fill moving depending as it depends on the projection so zooming for instance might change the result. Using the world position as a seed will ensure a good stability except if used with animations but it would be a good enough tradeoff in term of perf and quality.

[Rodrix3]

21 minutes ago, Sebavan said:

I would not rely on perlin noise to prevent the inline cost or the extra texture channel. About gl_FragCoord, the main issue is this will still fill moving depending as it depends on the projection so zooming for instance might change the result. Using the world position as a seed will ensure a good stability except if used with animations but it would be a good enough tradeoff in term of perf and quality.

Ok that sounds good then

[Sebavan]

@Rodrix3 Actually as the values are not perfectly equal due to the rasterization, the noise won t be stable except by relying on a noise texture.

The main issue is here the addition of a new channel which might break the number of supported uniforms/varying on small devices.  Let me see if I can find a trick to workaround it.

You can play with my branch called noise on the main repo.

[Rodrix3]

1 hour ago, Sebavan said:

@Rodrix3 Actually as the values are not perfectly equal due to the rasterization, the noise won t be stable except by relying on a noise texture.

The main issue is here the addition of a new channel which might break the number of supported uniforms/varying on small devices.  Let me see if I can find a trick to workaround it.

You can play with my branch called noise on the main repo.

@Sebavan thanks for the update!
I see... more complicated than I thought. I understand. I will let you know if I think of anything. Thanks

[Rodrix3]

I've been thinking.
On the original ThreeJS sample, the noise acts weird when moving the camera.
We need a method that makes the noise static, so we can do proper animations.

I think that adding a noise texture could be the way to go; and have the option of this being disabled on mobile devices.
I am not sure of the implementation you have in mind if that's the case, and if that would imply a change in the interface of the noise feature (API).

What do you have in mind?