box projected / parallax corrected cubemap with babylon js?

[shater]

Hi,

It is possible create similar like in playcanvas orange room example with babylon js?

https://playcanvas.com/project/345310/overview/orange-room

it use box projected cubemap and pbr shader (fast static cubemap).

an other and similar example with three js (parallax-corrected environment mapping)

https://www.clicktorelease.com/blog/making-of-cruciform

I found only in babylon js PBR dynamic texture reflection example but it use simple cubemap and not static so its slow.

Any example and suggestion welcome.

Br.

shater

 

[Temechon]

Hi, 

Maybe this can help you : http://doc.babylonjs.com/overviews/physically_based_rendering_master#how-to-create-a-gold-material

and here: http://www.babylonjs-playground.com/#1VCUIJ#12 (l8 and 14)

[shater]

Hi Temechon,

The problem is when create a room with simple cubemap and use it as reflection map for example for a floor.The simple cube map not fit well, (see the first picture in post) and with box projected or parallax corrected cubemap is looks good (second picture) . In the video you see the problem too when the camera is moving.

video1

video2

 

and website and pdf paper about this techinc.

https://seblagarde.wordpress.com/2012/11/28/siggraph-2012-talk/

https://seblagarde.files.wordpress.com/2012/08/parallax_corrected_cubemap-gameconnection2012.pdf

or i missed something your link ant possible solve this problem ?

 

Br,

shater

[GameMonetize]

Hello, we are not supporting parallax corrected cubemap (yet :))

[shater]

Hello Deltakosh,

It is on todo list in next release or later?

 

 

 

[GameMonetize]

I would say later. But perhaps you can help? To be honest I did not check how to implement it but perhaps it not a big deal?

[shater]

I'm an artist but can programming some limit so i can't decide this is hard or not  , but i can help with any graphics if need a room demo for this , like in playcanvas demo.

I found a full book for this (planar reflection cubemap:  gpu pro 4 book pdf):

https://vk.com/doc7051544_384516179?hash=dabe180f6e2b1e153b&dl=1048f9e17f9d5bd1d2

Br,

shater

[GameMonetize]

Thanks...I'll definitely take time to see how this works

[BitOfGold]

Well I first tried to make cube map reflections without parallax correcction, and it was not easy.
Not until I realized that I have to remove the reflectionTexture from materials when rendering with reflection probes.
Now it looks like a reflection
As long as I am 1 unit far from everything. 
I think I now understand the math but I  dont know how to do it with a shader.
https://www.bitofgold.com/cubemap/

If you look in the wall mirror, it shows the parallax error.
the reflection probe at the camera is not at the surface point.

Edited March 17, 2017 by BitOfGold

[GameMonetize]

link is not working

[BitOfGold]

Thanks!
It was late at night

[shater]

Hello,

BitOfGold if i understand well you use parallax correction for cubemap? Look very nice!

In this three js examples the parallax correction shader is very fast and run well on old mobile phone (chrome/firefox) too.

secular material

https://www.clicktorelease.com/code/making-of-cruciform/index.html

with smooth material

https://www.clicktorelease.com/code/making-of-cruciform/index2.html

 

 

 

[BitOfGold]

@shater No, sorry, It's fake
Only works because the camera is around the same distance from the floor as the size of the reflection probe's camera cube. Reflection probe is at the camera.
(so it is mathemagically parallax corrected  )

We know the normal at the point of reflection, and need to get point where that normal crosses the reflection probe's cube.
PBR reflections are ony working now, because the reflection texture is infinite far away (like a hdr sky), or the reflection probe is in the middle of the object.

Thanks for the example!
That is the simplest one yet.

I hope I can work on this the next week.

Edit:

I cannot wait until next week
https://www.bitofgold.com/cubemap/

I tried the parallax correction in the @shater 's example above. It works, bit it's far from perfect.
So it's not fake now, the reflectionprobe is in the middle of the room, the "cube size" is about the size of the room.
The reflection probe does not move with the camera now, just sits in the middle of the room.

But... the reflections are a bit "off". They are at the wrong place, and when the probe moves with the camera, they are only good when it is at the middle of the room.

Code (commit in my fork of babylon) is here:
https://github.com/BitOfGold/Babylon.js/commit/b70d8689f882948437c26a2fdd27d0bd48ea972c
 

I think I will try other correction methods too, now I have the template.
If someone knows a better function for this, please help

[BitOfGold]

Or, there is some error. In the original example, this works perfectly

[shater]

Hi,

Im not understand well all the code but, on git found the only parallaxCorrectNormal function and other is missing? Seem the remain codes calculate the relative camera position and other normalize the cubemap size?

remain code:

void main()  {
 
    gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
    pos = vec3(modelMatrix * vec4( position, 1. ) );   
 
    vec3 correctedTangent = m3( modelMatrix ) * tangent.xyz;
    vec3 binormal = m3( modelMatrix ) * cross(normal, correctedTangent.xyz );
    vec3 normal = m3( modelMatrix ) * normal;  
     
    tbnMatrix = mat3( normalize( correctedTangent ), normalize( binormal ), normalize( normal ) );
          
    view = pos - cameraPosition;
 
    vUv = uv * repeatUV;
}

 

void main()  {
 
    vec3 vVec = normalize( view );   
 
    vec4 bumpNormal = texture2D( normalMap, vUv );   
 
    vec3 n = bumpNormal.rgb * 2.0 - 1.0;
    vec3 modifiedTangent = normalize( vec3( n.xy * bumpiness, sqrt( 1. - n.y * n.y - n.x * n.x ) ) );
    vec3 nVec = tbnMatrix * modifiedTangent;
       
    vec3 rVec = parallaxCorrectNormal( reflect( vVec, nVec ), cubeMapSize, cubeMapPos );
    vec3 env = textureCube( envMap, rVec ).rgb;
 
    vec3 diffuse = texture2D( diffuseMap, vUv ).rgb;
    vec3 specular = texture2D( specularMap, vUv ).rgb;
     
    vec3 dN = parallaxCorrectNormal( nVec, cubeMapSize, cubeMapPos ).rgb;
    vec4 diffuseColor = ( 2. / sqrt( length( dN ) ) ) *  textureCube( envMap, dN );
     
    gl_FragColor = vec4( diffuse * ( 0. * diffuseColor.rgb + 1.5 * env ), 1. );
 
}

[GameMonetize]

@BitOfGold IF this works well, I would love to get a PR

[BitOfGold]

@Deltakosh 
OK, but first I have to find a solution to correct the reflections.
I tried "spherical" parallax correction., but it was worse than this "box correction".
Still I cannot find better than this.

Playcanvas uses this same projection:
https://github.com/playcanvas/engine/blob/master/src/graphics/program-lib/chunks/cubeMapProjectBox.frag
So I think it is the right correction, with some error
 

[BitOfGold]

@shater I think the other code (calculation of reflection normal) is a bit different in babylon.
But it is the same thing.
 

[GameMonetize]

Keep us posted!

[shater]

@BitOfGold have you any progress?   I hope someone else can help us.

 

[BitOfGold]

@shaterSorry, no better version than this.
It's working but not perfect.

Reflections are perfect in this example:
https://www.clicktorelease.com/blog/making-of-cruciform

The room's scale is matching the parallax box size, and camera position can change from middle of the box, and reflections are still perfect.
In my code the only scale that works is when cube box size is 29x29x29m (room size is 17x3m)

So no progress, sorry.
I feel that I'm near and there is only one error...
I hope anyone else sees what is wrong, or can make a better correction.