demo webworkers for Babylon ?

[Dad72]

Hi,

 

Is it possible to have a demo or documentation for the use of webworkers ? How does it work? 

 

Thanks

 

 

[jerome]

Davrous did one here : http://blogs.msdn.com/b/davrous/archive/2011/07/15/introduction-to-the-html5-web-workers-the-javascript-multithreading-approach.aspx

same in french : http://blogs.msdn.com/b/davrous/archive/2011/07/08/introduction-aux-web-workers-d-html5-le-multithreading-version-javascript.aspx

 

A simple primer here : http://www.html5rocks.com/en/tutorials/workers/basics/

another in french : https://developer.mozilla.org/fr/docs/Utilisation_des_web_workers

 

Something important you should know about these two articles : they talk about data serialization/deserialization in JSON before/after receiving messages between two different threads. However almost all browsers implement now something called transferable objects.

This means quite all js data type are directly transferable from a thread to another without needing this JSON translation. And this transfer is usually fastest than the JSON way.

So they are good articles to quickly learn about webworkers... then ugrade your knowledge to transferable objects 

[Dad72]

Thanks Jerome.

 

I'm talking about how it is used in Babylon. has recently been added, but there is no documentation.

 

I speak of a presentation in the playground this functionality.

[RaananW]

Hi,

It was just added two days ago and still ongoing modifications. It is in general transparent to the user and the developer. The scene has a new variable, workerCollisions. It's a Boolean. False, the default value is the collision system as it always was. Setting it to true will start a worker and run the collision in a different thread.

I still need to write the entire documentation. It's on the to-do list :-) If you have a question regarding a specific feature or how it actually works I'll be more than happy to answer!

[Dad72]

So just do it:

scene.workerCollisions = true; //active use of workerCollisions

Ok, this is easy. Raanan thank you. I'll wait until the documentation is written for more.

[jerome]

BTW, my fusing mind gives me ideas for the next BJS 8.0 release :

A full multithreaded engine (orthogonal with use of SIMD) ...

 

The main UI thread, clocked on the RAF, will just render things (GPU calls) and collect DOM events, maybe abstract them.

The Engine thread (BJS), clocked on an arbitrary frequency (adaptative to the CPU would be great) will  receive event messages  from the UI thread, will treat them if needed (cam rotations, etc), will compute everything related to the 3D world and pass it back to the UI thread.

The User thread, evented or clocked the same way than the Engine thread, will receive event messages from the Engine Thread (some passed originally from the UI thread, a bit like the event bubbling mode in the DOM), will compute the game logic and will pass back objects to the Engine thread.

 

In short terms :

 

• Engine Thread is BJS except the rendering part. It is evented and/or looping at its own frequency.
• UI Thread is the pure WebGL part : only drawing at 60 fps + DOM event handling/passing... well everything that can't be done in a worker because of the DOM inaccessibility
• User Thread is what we code in the PG createScene() : the game logic only. Evented and/or looping.

 

Ok, ok, I stop dreaming ...

 

 

Other advice from a webworker tester with BJS ($bibi) : it is really worth using webworkers if you need to get data with xhr (or websockets).

The main thread for BJS, the second to get your data from an external source at a given frequency.

[dbawel]

Hi Jerome,

 

We use websockets to coonect to our node.js server.  Do you have any examples to use webworkers for event data?  This sounds almost to good to be true.

[jerome]

I use webworkers to separate in their own thread all high latency I/O like network I/O. [EDITED]

This means BJS runs in the main loop and another thread besides is in charge of requesting/sending/getting/parsing external data, whatever the protocol : websocket or xhr.

 

example (not really spectacular) : http://logiciels.iut-rodez.fr/proto/weathermap/

[EDIT] (oops sorry, I just realize the public version is not the threaded one, but it doesn't change the explanation underneath)

 

Server-side a daemon Ruby script sends SNMP requests at regular period (30" here) to many network switches inside our university in order to collect raw data : nb of bytes counters on each switch device port. Then it aggregates all these data in a collect result what is a simple json file (so a file rewritten each 30") written on a web server.

 

Client-side (browser) :

The main script is just the BJS scene. It listens to messages from the second thread and refresh the display (mesh size, animations, etc) according to what this second thread sends.

This second thread is the I/O thread : it sends a xhr requests every 30" to the web server, gets the json file, parses it, builds internal data for the main thread and passes them to it.

 

So the I/O + collected data logic is in its own dedicated thread and notifies the main thread when something should be updated in the rendering loop.

 

As transferable objects are now implemented in webworkers, it's quite easy to make webworkers have a kind of dialog and behave according to different messages types :

var ioWorker = new Worker("ioWorker.js");// sending message to ioWorker, ex : tells the server who I am and my scene is readyvar message = {type: "clientReady", payload: {pseudo: "$bibi", gameSession: "rg4354cqrfo", device: "iPad", etc: "etc"} };ioWorker.postMessage(message);// treats ioWorker incoming messagesioWorker.onmessage = function(e) {   var msgType = e.data.type;   // treat each message type to do different things   switch(msgType) {   case "spawEnemy":     // handler1 about data payload :     // this only updates shared variables within this thread     // these variables are then used in the render loop   case "updateLights":    // etc   }};

the same symetrically in the i/o thread : mainly exchanges with the data/session/game server and evently dialogs with the main rendering thread

[RaananW]

Let's not forget one little thingy - everything is still asynchronous.

 

If you want to react to use input, you must make sure it will take less than one / two frames to actually return an answer, as the main thread will run the worker's (or, in your case workers' :-) ) answer after rendering the current frame, if they came on time (which is uncontrollable). 

 

JavaScript, in its nature, is async. Adding async threads to it won't make it magically work better... It is wonderful for the rendering when separating complex tasks such as collisions, but I wouldn't use workers to react to the mouse rotation. This must occur in real time on the main view-thread. Otherwise the user will have an annoying delay (if its computer is not the best). I would, however, move the simplification implementation to a worker. This is a perfect async task.

 

Maybe, one day, the workers' implementation in all browsers will be so super fast, that while the view thread is waiting for the next animation frame, the answer can be returned already. That would be "real time"  . But Firefox's transferable-implementation (for example) is ... not the best?

[jerome]

yep you're right

DOM event treatments shoud definetly not happen in a worker (without saying the workers don't access to DOM)

But high latency  inputs can  [EDITED]

 

I hope FF transferable implementation will reach the Chrome one soon (didn't try FF ww). They surely will improve this.

[JCPalmer]

I was pretty sure this thread was about application / game level web workers, but thought I would speak up about the one being built into the frame work.  It is important that they have a switch to do them in-line, as I see is being done above.  Canvas+ does not support web workers.  Got an email yesterday per-annoucing a new version of CocoonJS, but pretty sure web workers was not on the new features list.

[RaananW]

Yep, the first question was about a new feature in 2.1 .

If no worker exists, the feature will be disabled, and legacy support will be turned on. There is also a build that doesn't include the worker at all, so cocoon users could use that as well.

Shame they don't support workers, but I can understand why it would be a bit difficult for them...

[JCPalmer]

Well, the closer I look at that email on a screen bigger than 5", they say they are rebuilding their whole technology, but give no details.  Those "features" were just some bug fixes from Feb.  So maybe.

 

I submitted something about SIMD to them a while ago.  Hopefully, something on the web audio on all platforms and / or web workers are in it.  They have certainly been requested, as I have seen them.

[dbawel]

Thanks for all of the input in understanding webworkers,  as I haven't been threading processes outside of my main thread.  But this should speed up some scenes by launching threads for multiple complex operations, and/or for computationally heavy functions which are continually running.

 

As for simple I/O, I might guess that this would cause delays in retrieving and passing server events outside of the main.  But soon we will be streaming elements such as new meshes and complex bone animation, and webworkers should be a huge benefit; either to call these elements when requested or to potentially cache them in advance.