Omni‐directional Stereo (ODS) Content

[paleRider]

Hi everybody:

We are starting a VR project and, as we want it to be as cross-platform as possible with a unique base code, I'm thinking on giving a chance to BJS (maybe this is not the best case of use for this awesome engine).

This is, indeed, not a genuine 3D Engine development, in terms of 3D scenes with different assets into it (characters, props, effects, ...), but a theatrical experience where the user sees an omni-directional stereo (ODS) pre-renderized CG animation, walking on the streets of a medieval village, going downhill from the cathedral to the palace. We need this approach in order to have high-quality CGs. As said we are not looking for a pseudo-VR, consisting in a planar projection of a 360º video on the inner side of a sphere, but a real VR video experience.

Under this assumption we'll need a different video stream for each eye, in order to give the user a sense of depth as you look around in every direction. Near things look near, far things look far.

Of course we'll be rendering texts and other CG effects overlays on the video streams.

I'm searching through the docs and this forum for a starting point on all this, but I'm only finding pseudo-VR approachs.

Is this type of "dual-camera" planned to be incorporated to BJS in future versions?

Best regards.

[RaananW]

Dual cameras are integrated since version 1 probably

I am not sure what you refer to as "pseudo-VR", but screen-split can be easily achieved using the activeCameras array and modifications of the viewport. We do have many types of camera to support that, which you can also change to your own needs. Check a simple demo on the website, and click the camera on the right to switch between them - http://www.babylonjs.com/demos/spaceship/ (The Device-orientation requires you to click and move the mouse once, if you don't use mobile). The distortion compensation can be adjusted.

Multi-Views - http://doc.babylonjs.com/how_to/how_to_use_multi-views

You can also check what we do with the rig-cameras in the camera class for better understanding on how we solved the multi-camera issue in VR and WebVR (and the anaglyph camera as well )

 

Oh and - Obviously, Babylon is the right choice for your project  

[paleRider]

Thanks for your quick answer RaananW:

What I'm talking about is the type of dual-camera contained in this doc: ODS content

Best regards.

[RaananW]

Hi @paleRider,

what is the difference between that and the VR camera without connecting it to the motion sensor? I didn't read the entire thing (I read it like I read books - look at the pictures and try figuring out what they meant). I saw something about HUD and vertical stacking (which is configurable - instead of setting the width to 0.5, set the height to 0.5 in the viewport).

[paleRider]

Dear RaananW:

Ok, as said this is a project in its starting point.

Anyway, the difference is that we don´t want this:

But this:

Said this, obviously I need to read all about BJS VR Cameras and try to match its behavior with the specifications of OSD approach.

I'll return to this thread as soon as I have a clear image of the whole thing.

Best regards.

[RaananW]

It seems like the projection matrix needs to be different. So it is all about changing viewports and the calculation of the projection matrix. A hint can be provided in the WebVR camera as well (there is a different projection and view matrix function to the rig cameras (child-cameras). Start at the rig-camera configuration in the camera class and take it from there. It can be a bit confusing at time, so if you need any help let us know.

[SvenFrankson]

Are you sure you need ODS from BabylonJS cameras ?

We did something similar last month (bake an HD scene to run it in VR), and our workflow was the following

 

- Create scene in Blender

- Generate two spherical textures, it uses your ODS-generated projection. Blender takes care of it.

- Apply the textures on two spheres, in Babylon.

- Use a VR Camera from Babylon (it uses you first projection +- eye crossing) from inside the spheres. Each sphere is rendered on only one camera.

- Then you can look around (but not move !), the HD scene appears in 3D with the quality of a baked render.

(repeat many times to get an animation)

 

It worked pretty well. As far as I know, the ODS projection is used to generated a texture from a 3D Scene. There no need to use it in real time when watching your result.

(our demo is not for public use, but if it's unclear I can try to make a light demo so you tell me if we're talking about the same problem)

[paleRider]

Hi SvenFrankson:

Well, the approach you followed is admissible if the spheres rotate when the head does so, in order to not to have both eyes in the same line of sight, doesn't it?

Best regards.