Perf on mobile

[GameMonetize]

Hello, I saw this on twitter and a guy keeps telling me that it is faster on three.js on his mobile:

Can you guy check on your devices?

 

http://worldsnews.org/webgltests/babylon/

http://worldsnews.org/webgltests/three/

 

I see no difference on my side

[jerome]

absolutely no difference on my laptops ...

Maybe could he have added a FPS counter somewhere so the felt difference (if any) could be really shown and measured

[GameMonetize]

And on your phone?

[jerome]

huu... this my phone : https://mobile.free.fr/assistance/16.html

really

 

 

(no app, just a phone, well)

[JCPalmer]

I just tried on Sony's Xperia Z3 Tablet, Android 5.1.1.  It is noticeably faster on Three.js.  Wish it was more quantifiable.  Not a major improvement, but clearly visible.

 

Also have iPad Air 2, ios 9.1.  There is no performance difference, but the three.js looks crappy.  The background is white & lines of boxes grainy.

 

My phone is Samsung Galaxy SIII.  Too old to care.

[GameMonetize]

Oh for now the babylon.js version is using antialising but hoepfully the author will update it soon

[GameMonetize]

After looking deeply, rendering are not the same...Looks like Three.js does not take in account the hemispheric with intenisty = 0.3 because bottom faces are fully lit.

 

Definitely not the same scenes....

[G'kar]

Hi

Checking on chrome/android 5 on my samsung GS5, with fps display flag activated in chrome.

Animation looks faster on Three. (rotation speed is about ? *2 or *3, no metric to quantify). fps is 60 fps (stable)

On BabylonJs, I only get fps in range about 17 to 20 fps.

Another difference is the zoom factor. I see twice as many cube on Three (scrollbar cover limited range 80 % of full view)

On bjs, scrollbar are selecting only 50% of full view, add to scroll a lot to view rest of scene. zoom in possible, but not zoom out to same view as in Three.

[JCPalmer]

Yea, I thought you would like those 2 data points.  They are 2 radically different devices.  The Sony isn't even sold in the U.S.  You have to figure out how to get it out Japanese into English mode (10 minutes), but great for coverage / triangulating against iPad.

 

If you are going to mobile things with webGL, having only 1 device is insane.

[jessepmason]

Hello!

 

55 - 59fps on threejs

30 - 40 fps on babylon

 

on Galaxy s6, browser chrome, fps flag enabled in chrome

 

same as what G'kar said about the zoom factor

[jerome]

@DK

I guess that the 3JS demo bottom faces are lit by the HemisphereLight because he set the ground light to white (second argument of the hlight constructor).

In the BJS demo, his hemispheric light has no groundlight, so I guess it's black by default.

[Stvsynrj]

50 fps for the 3JS version, 18 fps for the babylonjs version on a Galaxy Note 3.

btw, i did the same scene using scene.registerBeforeRender() instead of requestAnamationFrame(), don't know if it have something to do with that be i've got 15 fps more, just to say it.

[dbawel]

As DK advised, the scenes are not the same, so this is not any type of reliable benchmark test. However, I'll report my findings, as using on my laptop and desktop, there is no difference in rendering speed or quality, running Windows 8.1 on both systems. However, on my Sony Xperia Z2 tablet running a quad core 2.3 Ghz CPU and a GPU with 2Ghz dedicated video RAM, there is a dramatic difference in the framerate between BJS and 3JS; where 3JS is running much faster than the BJS scene - however I added no code to display reference to fps, so I cannot tell you the precise difference in framerate - although there are no obvious differnces in rendering quality between BJS and 3JS. But again, these scenes are not identical, so I would personally run testing again with identical scenes if this is something you need to benchmark.

[chg]

Nexus 7 (2012 model) on Chrome 47 and Android 5.1 - Three scene smooth framerate, Babylon screen really bad framerate (too lazy to actually measure it, but it looks like 7-8 fps if I'm generous... it's possibly 3-5fps)

 

As DK advised, the scenes are not the same, so this is not any type of reliable benchmark test.

Is it intended to be a benchmark though? I get the impression it's more just a comparison of how things preform out of the box if you don't dig into the engine and try and optimise things... perhaps defaulting to more performance intensive but better looking effects (at least in this case with the hemisphere light as noted by DK) is random Twitter guy's point (even if he didn't intend it to be) 

[GameMonetize]

 bjs scene uses antialiasing and render on a bigger canvas.... no way to compare them...

[chg]

 bjs scene uses antialiasing and render on a bigger canvas.... no way to compare them...

I'm confused by this... I'm not seeing AA in either and to me both canvases seem to be 1366x677

[GameMonetize]

OP has changed bjs version to remove antialiasing but on my device canvas for bjs is twice bigger than for 3js

[GameMonetize]

All running android version: do you have a way to get canvas size displayed?

[GameMonetize]

what I can see is that on 3js version, OP force the canvas to window inner size which is not the case on bjs version.

[GameMonetize]

ok I know what is going wrong.. by default the engine is enabled with adaptToDeviceRatio = true so the canvas buffer could be really bigger than actual screen size.

 

I will change that as this leads to many perfs issues.

[GameMonetize]

Ok update sent. So now on low end devices we won't adapt to DPI by default.

 

Note for the record: to turn it on, just call:

var engine = new BABYLON.Engine(canvas, antialiasing, null, TRUE)

[JCPalmer]

Thanks guys, I did not know about chrome://flags.  I got pretty much the same difference in values on the Sony Xperia Z3 after I got it turned on.

 

I would not get too worked up about benchmarks.  FPS is not the sole qualification.  The iPad results were pretty bad in looks for three.js.  Though I do not think this comparison tried to make a test where three.js would easily handle it better on Android, benchmarks are still a pretty terrible idea.  

 

If you were going to rig one, you would do it on mobile since the machines are less powerful.  Doing something which worked differently the 2 on the cpu side is easiest way to generate difference.  Having a different color for a material causes a different shader to be built in BJS.  Might that only be a different attribute passed in three.js?  I do not know or care, but having 200 materials in a scene is a significant for BJS on a small machine.  What happens when the cubes are all the same color?  The difference in quality of AA does not show up until you run on iPad.  Might not that be it?

 

If I sound cynical, I remember benchmarks past, like with SQL databases, where it was thought vendors were specifically tailoring there DB's to score well on benchmarks.  That would be harder with open source, but it is just a waste of every bodies time.

[GameMonetize]

@JCPalmer: it looks pretty bad in iPad because DPI is high but Three.js does not take it in account by default

[GameMonetize]

Final result:

http://www.catuhe.com/trash/perf.html

 

Please let me know if this is better

[JCPalmer]

about 10 FPS better than before, low 30's.  Still lower than mid 50's of other.