solved Anti-Aliasing in WebVR - Let's settle the debate once and for all

[dsman]

I have read many threads but have found no concrete steps to ensure we don't have Aliasing in webVR ( Oculus/Vive not Cardboard based VR using  VRDeviceOrientationCamera). Most threads are about anti-aliasing in VRDeviceOrientationCamera using default render pipeline and FXAA post-process.

So let's conclude it in this thread. 

Can anti-aliasing or the level of it (4x/8x) be forced by BabylonJS while using webVR?

As my GPU is very capable, while I am using other Oculus experiences the scene is perfectly anti-aliased. But while viewing our scene in BabylonJS in the browser there's little aliasing everywhere. 

I have also figured running the browser in full screen while in WebVR (split screen view in browser), there's less aliasing. Also if I reduce browser window size, the aliasing is strikingly more. That makes me believe, it has something to do with resolution on which the scene is rendered in primary display. That brings me to another question. Since Oculus screen resolution is larger than FullHD, is it like if I had a bigger monitor than FullHD, the aliasing effect would have reduced? 

[trevordev]

From my understanding, anti aliasing should be possible in 2 ways.

    • FXAA is a post process and is toggled using the default pipeline. See playground below. This is done by babylon's fxaa shader and should be supported by most browsers.
    • MSAA is not a post process and is performed by webGL automatically when number of samples for a given texture is specified to 4, this is on by default when no pipeline is enabled. This default pipeline provides a control to toggle this that is false by default. In webGL versions < 2 (eg. Edge currently) msaa is only supported on specific types of textures that are not compatible with post processes so enabling any post process will disable msaa and enabling msaa in the default pipeline will fail. (With a message in the console)

I think the resolution should not play a role in the quality of aa. If it does it might be an issue of re-scaling after aa has already been applied.

Try out this playground which shows usage for both msaa and fxaa:

http://playground.babylonjs.com/#DNUPI2#27

    1. Enter vr
    2. Observe msaa is enabled so it looks smooth
    3. Press a key
    4. Observe pipeline has been enabled for the headset's camera with image processing enabled so msaa is now disabled and jagged edges are visible
    5. Press 1 to toggle fxaa
    6. Observe smoothness has improved
    7. Press 2 to toggle msaa (Not supported in edge)
    8. Observe smoothness has improved 

I just tested this behavior in edge + mixed reality headset. Let me know if I missed any of your questions or if anyone has issues with a playground repro and I can try to take a look.

[dsman]

Thanks for the elaborate answer. You mentioned that re-scaling after AA is applied could be the issue. I think this might be the issue in our case. 

Also is there any way to increase the MSAA quality?

[dsman]

Also in normal cameras (No VR), the rendering is full anti-aliased but for some materials which are highly metallic (PBRMaterial with high metalness and less roughness and reflectionTexture from a probe with refreshRate of -1), it appears aliased like this. http://prntscr.com/ir5aru 

 

[trevordev]

To increase the msaa quality, the samples set on the first post process can be increased eg. setting it to 8 will be 8x aa. Currently this value is not exposed by the default pipeline but could be added if desired I created this https://github.com/BabylonJS/Babylon.js/issues/3976 to track this potential work.

For the metallic issue you are describing, would you be able to provide a playground example that reproduces it so it can be investigated further?

Thanks.

[trevordev]

@dsman FYI msaa is now controlled via pipeline.samples = 4 or whatever level of msaa is desired instead of msaaEnabled to be consistent with the rest of the API before these were added a stable version.

Updated in: http://playground.babylonjs.com/#DNUPI2#27