VR rig cameras not separated, no 3D effect

[eugenesia]

Hi! Pardon me as my first post happens to be a bug report. I just learnt Babylon.js last week, am going through the tutorials, and it's a really great framework!

So coming on to the bug report: I am using a VRDeviceOrientationFreeCamera but it seems the left and right rig cameras are in exactly the same position, with no offset distance between them. As a result, although there are two viewports, the images are exactly the same. Using a Google Cardboard, the image appears flat and not 3D. See attached screenshot below.

Please also refer to Babylon.js playground http://www.babylonjs-playground.com/#ZTRL5#30

In the Playground and the screenshot, you can see that the line of vertical sticks along the Z axis are aligned in both viewports. If there is an offset between the 2 cameras, the sticks should not be aligned.

 

Screenshot - both left and right VR cameras produce the same image:

 

Bug in code

The error seems to happen where the rig cameras are updated in the ancestor class TargetCamera::_updateRigCameras() in https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/babylon.targetCamera.ts#L265 . Both left and right cameras have the same position and rotation, thus producing the same image:

284 case Camera.RIG_MODE_VR: 
285 case Camera.RIG_MODE_WEBVR:
286   if (camLeft.rotationQuaternion) {
287     camLeft.rotationQuaternion.copyFrom(this.rotationQuaternion);
288     camRight.rotationQuaternion.copyFrom(this.rotationQuaternion);
289   } else {
290     camLeft.rotation.copyFrom(this.rotation);
291     camRight.rotation.copyFrom(this.rotation);
292   }
293   camLeft.position.copyFrom(this.position);
294   camRight.position.copyFrom(this.position);
295
296   break;

I can attempt a pull request to fix this, but I'm not sure if I should just position the 2 cameras apart just like for a stereoscopic camera. Appreciate some comments please, thanks!

[GameMonetize]

Hello thank you for reporting..This is really weird. We will check asap!

[GameMonetize]

Actually it works..If you move using arrows left/right you'll see the 3d effect.

Postion and rotation are the same because the differentiation between left/right cameras is done here:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/babylon.camera.ts#L621

[RaananW]

Hi,

Both cameras should have the same position. It is the position of the body, which has two eyes. the position correction is done later, and can be controlled with the VRCameraMetrics that can be provided in the constructor.

The 3D effect is very hard to see when you look straight at all objects. But when moving a bit to the side... http://www.babylonjs-playground.com/#ZTRL5#31

So, I am not sure it is a bug. I promise to check further thou.

[eugenesia]

Hi everyone,

I managed to create a function to fix the bug in this Playground: http://www.babylonjs-playground.com/#ZTRL5#34 . The left and right images are the same, which is wrong, because our left and right eyes don't see the same images as they are in different positions.

The function below changes the View Matrix of the right rig camera, moving it to the right slightly to simulate the right eye.

Code in Playground: Change right rig camera's View Matrix slightly so it is displaced to the right.

// Shift the right camera view a bit to the right to simulate the right eye,
// to create illusion of depth. So it doesn't give the same projected image as the
// left camera view.
var camAdjustInterval = setInterval(function () {
    // Both rig cameras have the same view matrix component, this produces same projected
    // image for both eyes, hence no illusion of depth.
    if (camera._rigCameras[1]._viewMatrix.m[12] === camera._rigCameras[0]._viewMatrix.m[12]) {
        // Move the right camera a bit to the right, to create illusion of depth.
        // This adjusts the "View Matrix".
        camera._rigCameras[1]._viewMatrix.m[12] -= 0.2;
    }
}, 500);

 

The result is an image that has better illusion of depth. See screenshot.

Screenshot: Correct projected images. The right camera is slightly to the right of the left camera, so has a different projected image. This gives illusion of depth.

[RaananW]

Hello my friend,

when I am wrong, I am wrong. Thanks for being persistent  

http://www.babylonjs-playground.com/#ZTRL5#37 - here is the fix. 

The view matrix' drift WAS calculated, but was never used. how clever!

Having said that - once the fix is integrated you can play with the distance between the two views using the metrics variable that is being provided (optional). default is ca. 0.064

I will commit the fix pretty soon, the minute I will make it look nicer - reusing code, formatting typescript, sprinkling some magic powder and all that jazz.

Update:

PR merged, fix is there. It will soon be available in the playground as well.

[eugenesia]

Thank you @RaananW this works now! I tried to debug it myself but got lost in all the matrix-related code.

I hope to understand more of the code so I can contribute next time!