VR other devices

[brianzinn]

I have my scene working great in GearVR, but fuzzy in Cardboard on OnePlus.  I think what I need to do is calibrate the VR metrics for my viewer:
https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/VR/babylon.vrCameraMetrics.ts

google cardboard is:
Sreen to lens distance 39.3mm
Inter-lens distance 63.9mm
Screen vertical alignment: Bottom
Tray to lens-center distance 35mm
k1 distortion coefficient 0.33582564
k2 distortion coefficient 0.55348791

There is a cardboard calibrator (https://wwgc.firebaseapp.com/) where you enter your PPI and will generate a QR code - using the cardboard clicking button.  Would be cool to have something built into BabylonJS.  I think I need to work my way from those numbers to those use by camera metrics - does anybody know how? Otherwise maybe I'll try to start a PG using GUI + gaze to alter the metrics - except it looks like the metrics are a constructor parameter for the camera, so doesn't look like a good option.

result.hResolution = 1280;
result.vResolution = 800;
result.hScreenSize = 0.149759993;
result.vScreenSize = 0.0935999975;
result.vScreenCenter = 0.0467999987;
result.eyeToScreenDistance = 0.0410000011;
result.lensSeparationDistance = 0.0635000020;
result.interpupillaryDistance = 0.0640000030;
result.distortionK = [1.0, 0.219999999, 0.239999995, 0.0];
result.chromaAbCorrection = [0.995999992, -0.00400000019, 1.01400006, 0.0];
result.postProcessScaleFactor = 1.714605507808412;
result.lensCenterOffset = 0.151976421;

@davrous - Got some ideas - maybe something that could be a VR Experience Helper option to choose popular viewers using GUI + gaze like your demo (ie: cardboard/gearvr/etc.)?

[RaananW]

one of the main differences between cardboard and WebVR is that a WebVR device will provide us with view and projection matrix, so we don't need to make the calculations on our own. 

You can initialize your VR camera using those values (a VR matrices object is the 5th variable in the VR camera's constructor - https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/VR/babylon.vrDeviceOrientationCamera.ts#L4). So technically, you could ask the user to provide those values (in any way the user sees fit) and initialize the camera using them. I am not sure we want to integrate a QR reader just for that, but it would be a great external module!

[brianzinn]

I did find that parameter (my first link), but was saying that since it is a constructor parameter that maybe it's not a good way to get the user to adjust them at runtime - I didn't see a way to change them post construction, so would require creating new cameras with each adjustment.  I was thinking to make PG with GUI panels and gazing would move the different values up/down so we could calculate the camera metrics - not sure if that would work creating new cameras so quickly?.

QR reader would be cool and I found a QR decoder, but looks like it only provides the numbers in my post, so my other question was how do we get from my first numbers (screen to lens , k1+k2, etc.) to the numbers required by BabylonJS?  I did try researching it a few months back, but never got it.

Adding protobuf definition with perhaps useful comments:

/**
 * Copyright 2015 Google Inc. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/**
 * Message describing properties of a VR head mount device (HMD) which uses an
 * interchangeable smartphone as a display (e.g. Google Cardboard).
 *
 * While some properties are certain (e.g. inter_lens_distance), others
 * represent nominal values which may be refined depending on context (e.g.
 * viewport_angles).
 *
 * Lengths are in meters unless noted otherwise.  Fields are _required_
 * unless noted otherwise.
 *
 * Some context on why this set of parameters are deemed necessary and
 * sufficient:
 *    * FOV scale can be reasonably approximated from lens-to-screen distance
 *      and display size (i.e. knowing lens focal length isn't crucial).
 *    * Lenses are assumed to be horizontally centered with respect to
 *      display.
 *    * The display is not necessarily vertically centered.  For interchangeable
 *      phones where the device rests against a tray, we can derive
 *      the vertical offset from tray-to-lens height along with phone-specific
 *      bezel and screen sizes (supplied separately).
 */
message DeviceParams {
  // String identifying the device's vendor (e.g. "Google, Inc.").
  // A device's [vendor, model] pair is expected to be globally unique.
  optional string vendor = 1;

  // String identifying the device's model, including revision info if
  // needed (e.g. "Cardboard v1").  A device's [vendor, model] pair is
  // expected to be globally unique.
  optional string model = 2;

  // Distance from the display screen to the optical center of lenses.
  optional float screen_to_lens_distance = 3;

  // Horizontal distance between optical center of the lenses.
  optional float inter_lens_distance = 4;

  // Four-element tuple (left, right, bottom, top) of left eye's view extent
  // angles relative to center, assuming the following:
  //     * eye is aligned with optical center of lens
  //     * display screen is equal or larger than extents viewable through lens
  //     * nominal eye-to-lens distance
  //     * mirrored field of view will be applied to the right eye
  // These values are essentially used as an optimization to avoid rendering
  // pixels which can't be seen.
  repeated float left_eye_field_of_view_angles = 5 [packed = true];

  enum VerticalAlignmentType {
    BOTTOM = 0;  // phone rests against a fixed bottom tray
    CENTER = 1;  // phone screen assumed to be centered w.r.t. lenses
    TOP = 2;     // phone rests against a fixed top tray
  }

  // Set according to vertical alignment strategy-- see enum comments above.
  // NOTE: If you set this to CENTER, see special instructions for the
  // tray_to_lens_distance field below.
  optional VerticalAlignmentType vertical_alignment = 11 [default = BOTTOM];

  // If the phone is aligned vertically within the device by resting against
  // a fixed top or bottom tray, this is the distance from the tray to
  // optical center of the lenses.
  // NOTE: Due to a bug in initial versions of the SDK's, this field
  // must be set explicitly to .035 when vertical_alignment = CENTER.
  optional float tray_to_lens_distance = 6;

  // Coefficients Ki for pincushion distortion function which maps
  // from position on real screen to virtual screen (i.e. texture) relative
  // to optical center:
  //
  //    p' = p (1 + K1 r^2 + K2 r^4 + ... + Kn r^(2n))
  //
  // where r is the distance in tan-angle units from the optical center,
  // p the input point, and p' the output point.  Tan-angle units can be
  // computed as distance on the screen divided by distance from the
  // virtual eye to the screen.
  repeated float distortion_coefficients = 7 [packed = true];
  // Slots 8, 9 reserved for per-color channel distortion.

  // Optionally, whether the head mount uses a magnet in any part of its
  // design.  Intended as hint as to whether phone's magnetometer is
  // available for tasks such as orientation tracking.
  optional bool has_magnet = 10;

  enum ButtonType {
    // No physical button, and touch screen is not easily accessible.
    NONE = 0;
    // HMD has integrated magnet switch similar to original Cardboard.
    MAGNET = 1;
    // At least a portion of touch screen is easily accessible to user for taps.
    TOUCH = 2;
    // Touch screen is triggered indirectly via integrated button on the HMD.
    INDIRECT_TOUCH = 3;
  }

  // Specify primary input mechanism of the HMD.  Intended for advisory
  // purposes only, to address simple questions such as "can HMD
  // be used with apps requiring a physical button event?" or "what icon
  // should be used to represent button action to the user?".
  optional ButtonType primary_button = 12 [default = MAGNET];
}

 

[RaananW]

They are in the constructor for a specific reason  

The way the rigging system works prevents us from changing those parameters in real-time, hence - the need to have them fixed during object construction. Setting them dynamically should work, but might have side effects that can only be fixed with reconstructing the camera.

About the mapping between the configuration values and babylon's values - there should be a correlation. It was implemented a long time ago (I am actually not sure by whom, I have to say), but was built on top of existing parameters.

[brianzinn]

I'll try a few things this weekend. The distances in BJS look like meters and the distortionK's probably map to k1 & k2.  The other ones may be optional or the same.  Anyway, I will try it out and report any findings.  Can't be too far from default settings as I can see the scene, but it's just blurry.  Cheers.

[brianzinn]

no luck.  I did find a lot of crystal clear 3D threeJS cardboard demos with 1 or 2 settings - usually just eye separation (three stereo).  This is my starting point for next time:
http://tparisi.github.io/WebVR/examples/cube-cardboard.html

Think I will try again in a few months - there's only a few million Cardboard units shipped   ... my eyes are sore from all the different settings!
edit: my last PG looks much better than defaults.  Only FOV and interpupillaryDistance changed from defaults (https://www.babylonjs-playground.com/#N28LXA#17)

[RaananW]

I know the feeling about sore eyes! Makes you feel sick as well, using the wrong settings

What cardboard version are you using? 

[brianzinn]

cardboard V2.  actually the settings from that previous post PG ended up being usable.  Besides not integrating the known cardboard parameters, I would have liked to not have to set the FOV manually afterwards, so just rely on device metrics.  It looks like I can influence the fov by setting the postProcessScaleFactor (https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/VR/babylon.vrCameraMetrics.ts#L21), but have not investigated that properly yet.  the docs don't state, but assume FOV in rads - I used (60 * (Math.PI / 180)).  I will do some more testing for sure.  Additionally, I may try to do some work on vr helper, some options for choosing device metrics for device orientation only VR.  I am using a modified VR helper and will try to add to BJS, if it ends up looking useful (ie: 2nd image with cardboard logo).  My end state would include a cardboard creation method (like GetDefault()) with cardboard parameters for VRCameraMetrics.

[brianzinn]

Just wanted to add that with a few tricks we should be able to show Cardboard VR automatically for any device using the phone camera and an extension, so just sharing some research.

1. Scan QR code. (ie: https://github.com/LazarSoft/jsqrcode) online at (https://webqr.com/).  Webpage did request camera permission, but did not scan!  Instead I uploaded the QR code image from cardboard and was given the URL: http://goo.gl/KS2FEm
2. That short URL from the QR Code redirects to: http://google.com/cardboard/cfg?p=CgZHb29nbGUSCUNhcmRib2FyZB0J-SA9JQHegj0qEAAAcEIAAHBCAABwQgAAcEJYADUpXA89OghX8as-YrENP1AAYAM
3. Input the redirected URL on http://vrembed.org/tools/google_profile_decode.html (use LONG url field, since api.longurl.org no longer works for short url).  Note that this is a full JavaScript implementation to decode that URL into JSON of the cardboard configuration. That website source is on github with license Apache 2.0, although the webpage itself just mashes a few libraries together (bytebuffer, protobuf, quirc) - not much code.  There are many, many other non-licensed examples around, too. I got.

{
 "vendor": "Google",
 "model": "Cardboard",
 "screen_to_lens_distance": 0.03929999843239784,
 "inter_lens_distance": 0.06390000134706497,
 "left_eye_field_of_view_angles": [60,60,60,60],
 "vertical_alignment": 0,
 "tray_to_lens_distance": 0.03500000014901161,
 "distortion_coefficients": [0.335825651884079,0.5534878969192505],
 "has_magnet": false,
 "primary_button": 3
}

Also, got additional device information:

{
  "name": "Google Cardboard",
  "renderMode": "VRRenderModes.STEREOANAGLYPH",
  "icon": "VRIcons.logoAnaglyph",
  "hfov": 60,
  "ipd": 63,
  "ipdAdjust": 0,
  "k": [0.335,0.553] 
}