Significant delay on device orientation camera

[AB95]

Hi everyone,

I am currently working on a 3D scene that using VRDeviceOrientationFreeCamera and DeviceOrientationCamera, I experienced serious delay on both cameras. Based on what I have seen so far, it takes at least half a second for the camera to start rotate after I rotate my device. I try on other demo scenes with vr features enabled on babylonjs.com, there are also significant delay on the camera. Is this issue solvable by tweaking around the camera input or is it something we need to bear with if using the babylon default vr camera?

 

 

[AB95]

fyi, I am using samsung note 3 and htc one m8 to do the testing

[GameMonetize]

ping @RaananW who may  know

[RaananW]

Half a second sounds quite a lot, but don't forget that the phone needs to start broadcasting events before the camera starts reacting to it.

You can test it externally - connect a callback to the device orientation event and see when it starts broadcasting events. the cameras (should ) start rotating one frame after this callback will be called.

 

[AB95]

@RaananW

Hi, 

I need to update my question a bit:

the DeviceOrientationCamera is actually running smoothly, it does react soon after I move my device which more or less match your description: start rotating one frame upon the broadcasting events. However, the VRDeviceOrientationFreeCamera is the one that has significant delay, I haven't test it yet, have something else to work on, once I have the testing result I will post here again. Thank you for the suggestion :). 

[RaananW]

VR cameras tend to be a bit slower, mainly due to the double-rendering. The camera should (being a key word here ) start orienting itself one frame after the first event broadcast. But since a frame might take longer to render it seems slower.

Would be great seeing your test results!

[AB95]

@RaananW

I had done a few rough testing(as shown in the partial console.log result at the bottom), the vr camera rotation is updated 1~2 frame after the device orientation event triggered, so the problem would be slow rendering .

Considering my scene is not a complex scene(for testing purpose the scene only consists of a ground mesh and skybox, a single directional light and of course, the vr camera), the slow rendering of vr camera really bothers me as it can affect the user experience significantly. 

Besides optimising the scene, I also think about speeding up the rendering times such as disable scene.render() in runRenderLoop when device orientation event is triggered, and let the device orientation event handle scene-render(), although I am not sure about how reliable this approach can be. 

Any idea of how I can improve the rendering speed when using vr camera will be much appreciated! I am also trying to figure out by myself at the same times.

partial console.log results:

Quote

current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 150
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 151
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 152
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 153
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 154
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 155
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:270 current frame: 156
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:607 device orientation event triggered, alpha:359.945621907525 beta:0.2986429160215867 gamma:-1.208710286255823
app.js:270 current frame: 157
app.js:271 current camera rotation quaternion: {X: -0.5000000000000001 Y:-0.5 Z:0.5 W:-0.5000000000000001}
app.js:607 device orientation event triggered, alpha:359.9941907293401 beta:0.1048322364611232 gamma:-1.6672456676958671
app.js:270 current frame: 158
app.js:271 current camera rotation quaternion: {X: 0.4931444304495314 Y:0.5041954210489265 Z:-0.4962474274525176 W:0.5062944191442977}
app.js:607 device orientation event triggered, alpha:359.9797369794789 beta:0.0012798887173048191 gamma:-2.0526725512842723
app.js:270 current frame: 159
app.js:271 current camera rotation quaternion: {X: 0.4921833743380011 Y:0.5067963643912972 Z:-0.49316137359377427 W:0.5076463637810527}
app.js:607 device orientation event triggered, alpha:0.02657288881905253 beta:-0.12699661267000126 gamma:-2.9143470718066626
app.js:270 current frame: 160
app.js:271 current camera rotation quaternion: {X: 0.49087128041621153 Y:0.508960271305947 Z:-0.4910562803189598 W:0.508791271401184}
app.js:607 device orientation event triggered, alpha:0.10196416774032856 beta:-0.09898740622270114 gamma:-3.5314964374754343
app.js:270 current frame: 161
app.js:271 current camera rotation quaternion: {X: -0.48780399413133096 Y:-0.5129739694628639 Z:0.4864419955458506 W:-0.5121319702307249}
app.js:607 device orientation event triggered, alpha:0.17298285394534751 beta:-0.15008825591582034 gamma:-4.242009898000271
app.js:607 device orientation event triggered, alpha:0.225241386801106 beta:-0.24167732880310897 gamma:-5.015785037129178
app.js:270 current frame: 162
app.js:271 current camera rotation quaternion: {X: -0.4852311897929403 Y:-0.5151291769085183 Z:0.48347919052644167 W:-0.5152091768180925}
app.js:607 device orientation event triggered, alpha:0.20320685286663484 beta:-0.4533879929955596 gamma:-6.537040808834638
app.js:607 device orientation event triggered, alpha:0.2122916847943328 beta:-0.6361120002706434 gamma:-7.19015671209291
app.js:270 current frame: 163
app.js:271 current camera rotation quaternion: {X: -0.4796769504746507 Y:-0.5213819181183771 Z:0.475599953850218 W:-0.5214169181568742}
app.js:607 device orientation event triggered, alpha:0.20977786694361325 beta:-0.9539299337408338 gamma:-7.834337831094192
app.js:607 device orientation event triggered, alpha:0.1341054900318868 beta:-1.386089221445541 gamma:-8.5346896740945
app.js:270 current frame: 164
app.js:271 current camera rotation quaternion: {X: -0.4714607002967813 Y:-0.5319776812565292 Z:0.463839703583633 W:-0.5287506823636662}
app.js:607 device orientation event triggered, alpha:0.02969286027171025 beta:-1.8241579874176104 gamma:-9.352823560717342
app.js:607 device orientation event triggered, alpha:359.9310356329224 beta:-2.176021138717708 gamma:-10.256742878303594
app.js:607 device orientation event triggered, alpha:359.8070649368146 beta:-2.4052166169151845 gamma:-11.201137478254171
app.js:607 device orientation event triggered, alpha:359.6708467647478 beta:-2.470917840544867 gamma:-12.169989763185086
app.js:607 device orientation event triggered, alpha:359.6416256857499 beta:-2.045127360966608 gamma:-14.36541879912423
app.js:270 current frame: 165
app.js:271 current camera rotation quaternion: {X: -0.4683880076409457 Y:-0.540740008399658 Z:0.45434600872846564 W:-0.5308320091767229}
app.js:607 device orientation event triggered, alpha:359.83839792814274 beta:-1.3506983211511128 gamma:-16.560199491673217
app.js:270 current frame: 166
app.js:271 current camera rotation quaternion: {X: 0.4421323281696226 Y:0.5679612619915928 Z:-0.4249063406229463 W:0.5489932704165352}
app.js:607 device orientation event triggered, alpha:359.8833171669617 beta:-1.1132504334256914 gamma:-17.646876671437518
app.js:270 current frame: 167
app.js:271 current camera rotation quaternion: {X: 0.42884580871439515 Y:0.5725297199342673 Z:-0.416676817440686 W:0.5609647243344157}
app.js:607 device orientation event triggered, alpha:359.87576815691483 beta:-0.9664785399552027 gamma:-18.73309055239836
app.js:270 current frame: 168
app.js:271 current camera rotation quaternion: {X: 0.4224935488867708 Y:0.5753825402639188 Z:-0.41225354973033007 W:0.5661105406572504}
app.js:607 device orientation event triggered, alpha:359.8390215302928 beta:-0.9099491158939335 gamma:-19.83692060024058

 

[RaananW]

3 hours ago, AB95 said:

Besides optimising the scene, I also think about speeding up the rendering times such as disable scene.render() in runRenderLoop when device orientation event is triggered, and let the device orientation event handle scene-render(), although I am not sure about how reliable this approach can be. 

Any idea of how I can improve the rendering speed when using vr camera will be much appreciated! I am also trying to figure out by myself at the same times.

That's the million dolalr question, isn't it?

First, the only way to actually render the scene is run scene.render in the render loop. no other component will do it for you, you are in control. So, taking it out will make everything run very smooth, except for the fact that you won't see a thing.

The first thing you can do is to disable the lens correction (which, if I am not mistaken, is on per default). This is done in the camera's constructor.

The second thing you can do is lower the resolution of your scene. Play with the hardware scaling facor (engine.setHardwareScalingLevel) and see if it helps.

Other than that - use a better device / browser!  Sadly, not all devices and mobile browsers are prepared for VR using WebGL. The second render every frame is an overkill for most devices I have tested on. On the nexus 5x for example, the simplest scene renders at ca. 25-30 fps, and that's a best case scenario. 

[NasimiAsl]

i think about that before and make some parameter for that in my project

1.  you most define 3 mode quality ( sd mobile ,hd mobile & sd PC  , PC  ) (manage by resolution + (on or off effect like pps , god ray, rain ,clouds effect , shadows , ...  ) )

2.  2 mode Textures some tile and large unnecessary texture can be painted by color in mode 1 and render fully texture

3.  if you can detect ram overflow stuff  you can have stable fps in your scene

and something important behind the face (when you have big mesh in there take big time for that ) i don't know (yet) promise i find that soon

that is not happen  in native version

 

and define this  parameters like  flag for default shader + custom shaders and manage that when you need more Cpu 

and you can manage render time  by make delay function ( you need manage render loop in worker processor and render per worker event for that)

and about check collision and update camera method :  please stop control  this stuff on render loop that kill your fps 

 

[AB95]

@RaananW OMG it works! I didn't expect to be so easy but engine.setHardwareScalingLevel(value) results in a rather instant reaction of the vr camera! Thank you so much for suggesting that, I definitely need to explore more on BJS documentation. Just one thing that made me confused... is the default hardware scaling level = none? since I set engine.setHardwareScalingLevel(1) and it already run smoothly.

@NasimiAsl Thank you for all the tips! Will try on them as my scene will eventually become complex and definitely need to be optimised.

[RaananW]

@AB95 - the default hardware scaling level is the browser's pixel ratio factor.