Object state update in special conditions

[tejveer]

Hi, I'm new to HTML 5 game dev and maybe the next question is very easy for the experienced but I can't seem to find a good solution, so don't shoot. The question isn't strictly related to HTML 5, it is more like a generic question for every platform.

 

For the sake of simplicity, we are given: one update timer that runs 4 ms in its own thread, one render timer that runs 10 ms in its own thread. Usually, game's loop clears the entire screen before the render operation, but we all know this is not always a solution for games when going javascript and/or mobile. So, there are games (e.g. tetris) where we need to handle states of objects (that may move or may not move). The most efficient loop for such a game is to: draw the current moving brick without redrawing the entire screen, wait a second and DO NOTHING, then delete the area of the old brick ONCE, update the brick position ONCE, draw the brick ONCE. I have written the capital words because I can't find a solution for handling these situations. And now the questions start.

 

How do you do such things once in a multi-threaded environment without spamming these operations ? Assume that a new brick is created at uf 1 (uf = update frame), i send it to be rendered, so I do nothing on uf 2 (I have to wait a second). Since rf 1( = render frame) happens after uf 2, how do I remember to draw from uf 1 rather than uf 2 (because uf 2 contains no changes). You may say to keep them in a list of "drawables" and just render that list.

 

But what if uf2 actually contains changes ? Using the above list, i would have a list like [delete uf 0, draw uf 1, delete uf 1, draw uf 2] sent to render. This is not very efficient. You may say to reduce this list by removing opposite commands like "draw uf 1" and "delete uf 1", and only have "delete uf 0" and "draw uf 2". This may also take some serious time to compute every possible combination of reduction, but i may be wrong here. Besides that, doesn't that constitute a danger to other objects that may overlap my "state" object ? like not being displayed properly or being cut but by the "delete uf 0" which is an old state when render happens. Do you have an efficient solution for this ?

[rich]

What you're describing is effectively a 'dirty rect' system. I.e. each sprite is responsible for clearing up its previous location on the canvas.

 

However no modern browser needs to work in this way. The renderer is not decoupled from the JavaScript, they don't run in separate threads. If you tell the canvas to clear, then draw a sprite, it will happen in that exact specific order every single time, no matter what. There's no technical reason these days why you'd ever need to create a system where the sprites are responsible for destroying their backing states imho.

[tejveer]

I specified this question is more generic but anyway, JS has web workers. Sure you can say about browser support and stuff, but all the games have some kind of limitations. For example, the only browser that is the problem is IE and I would never support it lower than IE10. But then again, this question is not about this. I know about the dirty rectangle solution but I never said that the sprite SHOULD delete themselves. That was just an idea and I know it's not even close to good. Since almost all the games have backgrounds, there is no need to delete something, you can just draw the (eventually moving) background again and then render objects normally.

 

What I'm looking for are those states from capital letters. How do you send an object to the renderer ONCE and only ONCE ? Because if you use the dirty rect or background solution, you will always have to send the entire batch of objects every update frame, even if only one of the objects is moving. And in JS this takes time. Mobiles are even slower. And that without minding the battery.

[Sebi]

Ummm.. how long do you think does it take to iterate through an array in javascript?

Maybe I don't get your question, but iterating through 1 million array items takes like what? 3ms?

If you are talking about webgl, same thing, only upload items flagged as dirty to the GPU.

Also not sure what you want to do with WebWorkers.

The time it takes to send data around between the main thread and the worker is only worth it for heavy calculations that actually do take a long time to compute to avoid blocking the IO. Other than that, I didn't find a use case for Workers yet.

There are many things that you can do wrong and that will slow down your app, but iterating through an object and drawing the sprites is not the issue in most cases.

Also take a look at requestAnimationFrame. There is no point rendering all 10ms if the screen can only update at a rate of 60 fps.

Or even simpler: Try out pixi.js or phaser (if you are programming a game). Those libs already handle all that stuff for you.

[Chris]

Hello tejveer - welcome to the board!

 

From reading your op text, I am guessing that you are coming from a development place outside of the browser world, is that correct?

There is a lot of things you don't have (or even can't) take care of in the browser world.

 

About the rendering:

The browser handles the page rendering with a frame rate of 60fps. You are given a API to use that update cycle to draw to your canvas in the same interval.

The API I'm talking about is called requestAnimationFrame. Its basically a callback function that is being called at the same framerate as the browser window itself.

Bonus: The callback is stopped being called when the window looses its visibility (browser minimized or switching to another tab). This also helps with saving battery.

 

You could now call your game logic as well from the RAF callback. You could also set up conventional javascript timers, but they have two drawbacks: they may "stutter" (increasing and decreasing their call rates slightly) - and they continue to run when the user switches away and the RAF callbacks stop.

 

The next thing is: Javascript runs single threaded. You have web workers, thats right - but the main thread coordinates everything, including the rendering. You cannot break out of that. The only thing you might use web workers for are very complex operations like some AI calculations or background stuff that can be calculated asynchronously.

[Sebi]

The browser handles the page rendering with a frame rate of 60fps. You are given a API to use that update cycle to draw to your canvas in the same interval.

Almost. The requestAnimationFrame happens at the screen refrehrate. If you use a 120Hz screen, you will get 120 fps.

60Hz is mostly used tho.

[rich]

I specified this question is more generic but anyway, JS has web workers. Sure you can say about browser support and stuff, but all the games have some kind of limitations. For example, the only browser that is the problem is IE and I would never support it lower than IE10. But then again, this question is not about this. I know about the dirty rectangle solution but I never said that the sprite SHOULD delete themselves. That was just an idea and I know it's not even close to good. Since almost all the games have backgrounds, there is no need to delete something, you can just draw the (eventually moving) background again and then render objects normally.

 

What I'm looking for are those states from capital letters. How do you send an object to the renderer ONCE and only ONCE ? Because if you use the dirty rect or background solution, you will always have to send the entire batch of objects every update frame, even if only one of the objects is moving. And in JS this takes time. Mobiles are even slower. And that without minding the battery.

 

Web Workers are not designed to split-up rendering and cannot be used in that context. The browser process is single threaded. Ignore anything you may have learned in other development environments where threads are common-place, because they don't apply here.

 

If you want to render an object once and once only, then you draw it once. It's as simple as that. If using canvas you context.drawImage once. If using WebGL it gets more complex and performance gains can be made by batching sprites and textures and smart use of shaders. But effectively the number of times an object is rendered is entirely under your control. Nearly all game html5 frameworks work on the basis of clearing down and redrawing the entire display list every single frame, because in most cases the cost of working out which parts of a display has changed and updating only that area is more expensive than drawing the whole lot again. And if you enter the realm of post-processing effects, filters or any kind of mask or overlay then you often need to refresh the whole display anyway, because it's no longer as simple as checking "what has moved?".

 

For some games though you may see benefits of only updating the display when something has changed. I'm thinking turn-based or board games. In these cases you could benefit from a dirty rect implementation, or more simply just don't render the display until something, anything, moves or changes state (animation frame, opacity, blend mode, etc). And if it does you re-render the whole lot again.

[tejveer]

Guys, I am a web developer for almost 8 years (@Chris). I know JS and i know about rAF and what it does and how it does it, i know about web workers, i know about JS being single threaded, etc. The question was simple. Canvas is a like a "stateful" screen, so to speak. You draw something, and you can forget about it, but it still stays drawn. I want to take an advantage out of this without erasing it on every render frame. And since an update frame can execute more than once for every render frame, the question was simple. I reached the same conclusion as Rich's 2nd post before creating this topic. I thought it's better to always erase and redraw the entire canvas than trying to find a way to render a specific update frame. But this wasn't good enough for me and that is the reason for this topic. I thought experienced game developers have gone through this and have some ideas I cannot see for the moment. This is a question of game performance rather than a js or canvas one.

 

@Rich's 2nd post: It's simple to say "draw it once", because the drawing is one operation. How about deleting it once ? How do you do that ? If the update operation says to draw it, then you actually have to wait till rAF triggers, it's not that simple. It's like if there were 3 update frames for 1 render frame, and object was changed on update frame 0, then rAF needs to search through all update frames to see what needs drawing. Because if object updated once on update frame 0, it will no longer appear as changed on update frame 1 or 2 for example. So render frame cannot render only the last update frame because it will miss older "update" operations.

 

@Chris: Calling logic from rAF is a bad idea to me. Since execution is single threaded, if rAF spends too much time doing logic and display, there is no time for other parts of the game like user input to trigger. And then, they will be postponed. This leads to lag. And i hope you don't give me examples like point and click games. Stopping the rAF execution on tab change is not an answer to the battery question. The games are made to be played by looking at them, not by switching tabs. I find the games to be the most intensive application that can be run on computers, no matter if they are c++ or browser ones, so personally, i find web workers good enough to do the things behind the curtain and let the ui thread only handle rendering itself, user input and asset loading and whatever else is DOM dependent.

 

@Sebastian: web workers are fast enough, my guess is that they are even faster than ui thread, but I haven't tested that yet. Even exchanging messages with ui is fast enough and probably this will be enhanced in the future.

 

At one moment I had an idea about moving updated objects from one update frame to the next one, till the last one before render. This way, render can only render the last frame and the older updates will still be considered. But that raised other questions like order of operations, what happens with other objects if my update objects is not the top most, etc. Anyway, thanks for the feedback and if some moderator thinks this topic can not be expanded further, he can close it.

[Gio]

Hi tejveer

 

We do have an (optional) dirty rect system in our WADE framework, and I can safely say that it can be a massive win with canvas-based games. Not so much with WebGL, since that is not a stateful screen (i.e. the browser doesn't necessarily preserve the front buffer between frames), and you have to create and maintain a separate backbuffer and draw it to the front buffer every frame, and that pretty much nullifies any advantage that you may get from a dirty rect type of optimization. But it depends on each specific scenario, which is precisely why our dirty rect system is optional.

 

Anyway, there are many ways to handle it and this is what we do:

 

1) We have the option to have layered canvases. So if you have a static background, that normally goes into a separate layer (a separate canvas), and never gets redrawn.

 

2) When sprites move or change (because of animations, etc), and they are on a layer where the dirty rect system is on, they tell their layer that the area within their bounding box is now dirty. If you delete a sprite, exactly the same thing happens: the sprite marks its bounding box area as dirty. This happens in each update frame. The layer keeps a list of these dirty rects that is only cleared at the end of a rendering frame.

 

3) When it's time to render, the layer merges the dirty rects to determine what area of the screen needs redrawing. We'll call this the "dirty area" of the layer. This "dirty area" is clamped to the visible screen size.

 

4) The layer checks to see if there are any more sprites in the dirty area, even though they haven't been flagged for redrawing yet. These sprites will need to be redrawn too, so their bounding box areas are also marked as dirty.

 

5) If we added any new sprites in step 4, then we go back to step 3.

 

6) The dirty area is cleared with clearRect, the sprites that have been flagged for redrawing get redrawn, and the list of dirty rects is cleared.

 

I'm oversimplifying things a bit, but that's pretty much what we do.

 

Having said that, there are a couple of things to keep in mind:

 

- Like others have said, the render and update frames will never overlap, unless you do something really weird with WebWorkers. Don't do weird stuff and you'll be fine.

 

- Saving processing power on mobiles is a big deal. It's not just about saving the battery. A little known fact is that on many mobile chipsets (e.g. most Samsung ones), the temperature of the processor is extremely important, because if the processor overheats, it automatically and silently switches to low-power mode, so your 2.4Ghz processor may suddenly become, effectively, a 600Mhz processor. So avoid unnecessary operations like redrawing things that you don't really need to, because that may slow down your game in more ways than you might think.