[WIP] gles.js - OpenGL ES bindings for JS (aka fast WebGL for Android)

[Boris van Schooten]

I wrote some WebGL games, but was unhappy with WebGL performance on my cheap Android tablets (as in, Chrome and CocoonJS gave me like 5-10fps, even for games with only 10 sprites or so).  Therefore I wrote my own WebGL implementation, based on the V8 engine and direct OpenGL ES bindings.

 

I just got a nice proof-of-concept sprite demo up and running, showing 10,000 sprites (and running at 60fps on my 57 euro tablet).  I am almost ready to port my WebGL games to Android. 

 

Note that the demo has not been tested on many tablets yet, and may contain crashing bugs, memory leaks, etc.  Check out the demo here:

 

http://tmtg.net/glesjs/

 

You can also find a list of known problems on this webpage. I'd like to hear how it runs on your device!

 

EDIT: gles.js is now released on github! Get it here:

 

https://github.com/borisvanschooten/glesjs

[PAEz]

Wow, thats seriously cool, well done!

I tried your demo on my tablet so I could tell you the frame rate it got but theres nothing showing that.

Is there some way on Android to do that or something?....Im new to Android, only recently got a tablet.

Anyway, runs smooth as silk on my Tegra Note 7.

Add something to tell me the fps and add more sprites, lets see how many it can handle

 

EDIT: Oh and small tip.  For your shaders code use the multiline function technique, makes them alot easier to edit....

https://gist.github.com/PAEz/8211784

...theres a thing on github that does it a little nicer, cant be bothered finding it right now

[Boris van Schooten]

Glad you like it!  Meanwhile I went to the local phone shop to test it on some phones.  I was happy to see it ran on all the cheapest phones, about half of them more than 30fps, so the armv7 architecture thing may not be such a big issue. 

 

This week I'll have time to work on the project.  My first step is to get my games up and running.  Then I guess I should try to get the pixi.js bunnymark running on it on something

 

About the fps display, the problem there is fonts.  I don't have them.  In WebGL you typically use 2D canvas to create text textures, on Android it's the same thing.  So, I will probably make a simple canvas emulation based on Android canvas.  Or find something faster for fonts.

 

That multiline string trick will be very useful, thanks so much for that!  Up till now I coded my shaders in HTML elements.

 

[AshleyScirra]

10fps with 10 sprites is surely a fault and not WebGL's normal performance. I can get 10,000+ sprites on a Nexus 5 with WebGL, so I'd be curious to see your other demo and why it performed so poorly.

 

AFAIK CocoonJS's implementation is identical to what you describe, so should perform no differently. Chrome's renderer is actually more sophisticated and can parallelise WebGL calls, making it even faster, so I think your approach is actually a de-optimisation compared to what Chrome does.

[GameOptimal]

Your webgl demo must have done something wrong while compiling with cocoonjs, there's no way 10 or so sprites would have such an impact while using cocoonjs as the wrapper, even on lower end devices. Considering a group of professionals have worked on this for quite some time. And as Ashley stated, more proof is needed before you conclude your statement. Were you basing your tests solely on the 10,000 sprites? 

 

I'll try running some tests this week comparing the two on a lower end android phone, mainly because I find this interesting.

[Boris van Schooten]

OK guys, I guess I just have anecdotical evidence, but I have a growing impression that WebGL on Android is still in a sorry state.

 

When using Chrome on Android, I often had to click past the part that says "here be dragons" "careful, these experiments may bite" before I even got WebGL at all.  That's not just on my crummy tablet, but also on top tablets like the Samsung Galaxy Pro Tab 10.1.  After I do, performance ranges from bad to reasonable.  My experience comes mainly from games at html5games.com and playzap.net, and my own games. Most run at unusably slow frame rates (like 5fps) on my tablet. It seems that, as soon as you have a screen-filling background, performance drops through the floor.  On my tablet it runs the 10,000 sprite demo at like 10 fps (relatively good actually, though with hiccups).  For what I've seen, gles.js runs consistently faster than Chrome.

 

It appears that the only WebGL alternative is CocoonJS.   That is, PhoneGap and Ejecta-X don't seem to support it.  Please correct me if I'm wrong.

 

I previously tried CocoonJS on a couple of my own apps, and I was disappointed with the performance, though it was faster than Chrome.  But why is performance so much worse than the Java OpenGL apps I made?  You seem to find this puzzling, so did I.

 

I also tried the 10,000 sprite demo on CocoonJS.  I waited the 4 or so minutes that it takes to compile and download, and I got a 22 megabyte APK.  All it does however is crash right after the splash screen.  I searched for the error for a bit and found this:

 

http://support.ludei.com/hc/communities/public/questions/200762527-Not-Able-To-Get-Past-Ludei-Splash-Screen-On-The-Cocoonjs-Complier-For-Android

 

Not very encouraging. Neither is the 22 Mb file. I remember I used to try and keep APKs below 1Mb, because device space used to be so limited. And the google play apk size limit is still only 50Mb.

 

Anyway it seems I am not the only one with issues.

 

http://www.html5gamedevs.com/topic/7478-cocoonjs-game-performing-worse-in-webgl-than-in-canvas-mode/

http://www.html5gamedevs.com/topic/3980-common-phaser-cocoonjs-issues/

 

OK, enough talk for now, back to coding!

[Boris van Schooten]

Finished coding basic support for DOM, input, and sound, and a few more GL bindings.  I just got my first game running on it!  Check out the APK on the website.

[tumira]

@Boris van Schooten

 

Can you add support for webrtc , admob and IAP , Google Play services ?

[Boris van Schooten]

@Tumira,

 

I am planning to add support for the most popular monetisation services.  About WebRTC, I'm a bit skeptical about its maturity.  Can you point to a couple of killer example apps/games that use it?

[tumira]

@Boris van Schooten

 

Well the technology is still new, but Construct 2 game already support it for developing multiplayer game. I hope you can reconsider this.

 

Also any plan for Webaudio openal binding ?

 

The game demo apk that I downloaded from your website seems to have input bug when I tested on my galaxy s3. Kinda like touch offset bug.

 

I`m impressed with the progress.

[joannesalfa]

@Boris van Schooten

 

But you didn't tell what's wrapper you have used to compile APK from your Nyah cat vs Flappy bird game... Isn't phonegap or cocoonJS?

[Boris van Schooten]

@Tumira,

 

I'll take a look at Construct 2 examples if I can find them.  Audio is currently based on the audio element, but Web audio API is in the pipeline if OpenAL cooperates.  Do you have a particular game/app that makes good use of the Web audio API that actually has added value over the Audio element?  As with WebRTC, I need use cases to see if it's meaningful on mobile and how it should be efficiently implemented.

 

The input bug is actually a feature, unless you see something else than I think you see.  That is, it's a relative control scheme to make the game more playable.  Movements of the finger (on any part of the screen) are translated to amplified movements of the mouse/cat.

 

@joannesalfa,

 

You mean, if my Nyan Cat APK somehow depends on another package like Phonegap or CocoonJS?  No, it's just the V8 engine and a small piece of C++.  Phonegap seems to depend on WebView (I haven't tried it yet), which means you won't even get WebGL on KitKat, and CocoonJS wrappers are huge, so I don't think that's even possible. I remember getting a 10Mb APK out of their cloud compiler, but last time I tried it was 22Mb (both with the "WebGL/accelerated canvas" option).  I guess a release is due so you can play around with it yourself.

[joannesalfa]

@Boris van Schooten

 

It looks complicated, I have no experience how to embed any HTML5 game which is generated by Construct 2 including gles.js inside on V8 engine and a small piece of C++ to compile APK.

 

 

Besides, a lot of HTML5 developers are really excepting that, because some old devices with android 4.0, firefox and chrome blacklist GPU so they can't run in browsers until compiled APK.

 

 

I've tested your first game on my shitty generic tablet with android 4.0, it runs 60 fps, so awesome.

[Boris van Schooten]

@joannesalfa,

 

Great to hear it also works well on other tablets.  I can tell you that not only old cheap tablets have blacklisted GPUs in Chrome, but also new expensive ones. It seems to miss a "gpu reset notification" feature on many tablets.  This is some security or robustness feature that it insists on, which needs override through the ignore blacklist option. 

 

I can explain a bit about how APKing works.  The APK contains an Android app that is basically a wrapper around the V8 Javascript engine.  V8 can execute JS, but misses the browser functions.  No graphics, no DOM, not even a console.log function.  The wrapper interfaces with the V8 internals through (fairly convoluted) C++ calls.  Thus it creates new JS functions inside V8 that lead back into C++ functions that do the graphical stuff / browser emulation.  Then it feeds JS source code into V8 for execution.

 

The APK is just a glorified ZIP file (you can probably unzip it with you favourite zip tool).  It contains the app binaries (the "browser") and a data directory (assets/) that holds the "website" .  When started, the app looks for a file assets/index.html, and will use assets/ as the base directory for loading scripts and resources. Creating an APK from a html5 game is done by simply putting your own game files in the assets/ directory and APKing it.  Unfortunately that can't be done with a regular ZIP tool, because it needs to be signed with a personal key (and aligned), so you need the Android SDK for that.  I think CocoonJS uses the exact same scheme, only they provide a convenience cloud compiler for APKing. 

[joannesalfa]

@Boris van Schooten

Thanks for explaining! I guess no need to compile APK again, only can replace asset via zip editor and sign APK with Android SDK.

 

Everyone loves high performance games, no choppy performance. I thought HTML5 has ruined my dream until you showed us your investigation, it would be awesome if you make your own CocoonJS, the biggest problem is implementation of memory management. Ludei team gave up implementation of memory management, so they dropped it. 

 

Construct 2 includes memory management using webGL per layout to layout, which it's designed to make big games. I would like to request to update your investigation, can you compile your first game using Intel XDK with Crosswalk for Android?

 

We need to compare the performance between your own wrapper and crosswalk, I can test this game with my generic tablet. Let's see who wins.

[Boris van Schooten]

Memory management can mean a lot of things, but I think you are referring to image loading in C2?  I looked it up.  It appears C2 creates textures at the start of a level (layout) and deletes them when a new level starts (layout-by-layout loading). At least, when WebGL is used.  See the link here. For Canvas2D, dynamic loading of textures is left to the browser/display engine. In all cases though, C2 pre-loads all images at startup by defining and using all Image objects in the entire game once. This is done to hasten the slow process of loading from the internet to local (disk) cache (something not really needed for apps running from a local filesystem, I think).  Again, they rely on the browser not to load all the images uncompressed into memory, and either store them on disk or store them in compressed form.  Apparently, regular browsers have memory management that handle this well, so they stick to this approach. See the link here.

 

I think the image pre-loading causes CocoonJS to load all assets into memory at startup of a C2 app. This will also happen if WebGL is used.  But CocoonJS can do dynamic asset loading, and there is even a CocoonJS-specific Image.dispose() function that prioritises unloading an image.  See the link here.  This suggests C2 can do better in terms of Canvas2D dynamic loading for apps that run on the local filesystem (do layout-by-layout loading by defining images only per layout), but they choose not to implement it, because they believe this is the browser's problem.

 

A similar problems seems to occur with other engines like Ejecta, which apparently load images into memory as soon as they are defined.  Gles.js has no such problem, because it does not load anything into memory unless on demand.  In gles.js, an Image is just a dummy object that stores only the filename.  Image data is loaded into memory only when a texture is explicitly loaded with texImage2D, and the original data is immediately discarded again, so that only the texture remains in memory.  This ensures minimal memory usage, at the possible expense of extra load time (it currently loads from local filesystem only, for actual URLs a caching scheme would be desirable).

 

About Crosswalk, afaik it's just yet another Chromium wrapper, so I don't expect it to be much different from Chrome.  Except that you can disable the gratuitous blacklisting behaviour in your wrapper.  But I will try it when I find the time.  Maybe there's someone here who already uses it who can try to compile my game?

[AshleyScirra]

When we talk about "memory management" we only mean lazy-loading image texture memory, which you say you've already got covered. We added support for lazy loading in Ejecta with some help from Dominic, and all existing browser engines already do that. CocoonJS's dispose() function is useless because by the time you call it it's too late - it already loaded the image data in to memory, it doesn't help stop that up-front cost.

 

Still I've said this before and I'll say it again - we don't think non-browser wrappers have a future, and they are far too difficult to integrate with a mature engine like Construct 2's and even then they always miss a ton of features. And these days by the time you finished catching up the iOS 8 and Android L webviews will be proliferating with WebGL support, so people can just PhoneGap their HTML5 games.

[Boris van Schooten]

@ashley,

 

Sure, I hope we won't be forced to write our own homebrew wrappers for html5 for long.  You can wait for 1 or 2 years for someone else to fix this problem, but I certainly won't wait.  I've already made a release of my first Android html5 game, so for me the wait is over.

 

But since it's mostly C2 users that take a keen interest in my little project, I was suggesting that maybe you can cater to your users better by implementing a layout-by-layout loading option for canvas2d.  That is, set Image.src only at the beginning of a layout, and drop the reference at the end.

[AshleyScirra]

No, we have run down the rabbit hole of modifying our engine for non-browser engines and it's a nightmare we don't want to repeat. You could implement lazy-loading with LRU eviction yourself, which is a better solution anyway.

[Boris van Schooten]

Update!  I got most of pixi.js examples and the first phaser example game running in gles.js.  The pixi and phaser engines run without any hacks in the code. Check them out on the website.

[tumira]

@Boris van Schooten

 

Impressive work. Hopefully you can make it work for Construct 2 users.

[gnumaru]

Even though the “easy way” would be bundling and html5 game with a full web-browser, like intel xdk/crosswalk do on mobile (and like node-webkit does on desktop), there is great value in you initiative.

 

It is certainly very annoying to make an html5 game with less than 1MB in size when zipped and see it grow to almost 20MB when packed in an apk with crosswalk. And certainly a full browser is not needed when all you want is only webgl. Also, I don't like the idea of uploading my game for some other service to bundle it inside an apk using some obscure process which may not always be available.

 

I installed all your examples in my tablet, a Genesis Skyworth GT-1240 (the cheapest 10 inch tablet I could buy), and every one of them performed very well.

 

I think gles.js can be very valuable for those planning on making webgl only games. It may not be suitable for games that relies on DOM or canvas, but probably sounds the best option for pure webgl. As you stated yourself at your website, you just need to define an easy and simple workflow for packaging the webgl apps for android (maybe an eclipse plugin? Or cli tool?).

 

Keep up the good work, and I hope to see gles.js released in the wild soon.

 

By the way, I played Tsunami Cruiser. Nice game =)

[eldog]

This is what I've been waiting for. Looking forward to a release. 
 

On my Nexus one (ancient 4 year old device), I get around 5fps on the 10,000 demo. However, on the balls demo I get something more like 30fps and the dragon spline is near 60fps. I am surprised any work at all, very well done.

 

Am looking forward to its release. An open source, backwards compatible (e.g. not requiring the latest Android), non-cloud powered way of putting html5 games into an app that I can share with friends/world is what I've been yearning for.

 

P.S. I too loved Tsunami Cruiser, one of my fave Ludum Dare entries in that compo (was great it worked on Android too).

[joannesalfa]

He stopped posting here since one month ago, I don't know why

[BdR]

He is probably too busy with some of his game releases, see here -> http://indiegames.com/2014/10/freeware_pick_tsunami_cruiser.html

 

I'm also really looking forward to any updates, because gles.js looks really promising. It might even give CocoonJS a run for their money..