Replay system (early stages) for Canvas games

[MichaelD]

Hello all, lately I have been trying to find a way to save progress or part of a game session that was made with canvas. What I came up is explained in the following tutorial-like blog post of mine:

 

http://nightlycoding.com/index.php/2014/01/replay-system-for-kineticjs-and-html5-canvas

 

Also there is the Codepen where it all started: http://codepen.io/netgfx/full/DLrCy

 

In a nutshell I'am using canvas toDataURL function along with animationFrame only decoupled by underscoreJS debounce (to save some CPU cycles). Playback is made with the excellent Greensock TimelineLite.

 

Let me know what do you think. Should I consider refining it and made more universal? Could some game frameworks be interested?

[Chris]

This is surely a safe way to crash even modern browsers on decently powered machines: constantly taking full screen (okay full canvas) screenshots, converting them INTO A DATA URL (so just add +30% space, jay) and then storing them.

 

This blows up your memory in no-time! Hooray!

 

Honestly - the idea is a trainwreck. Sorry to be so harsh but you will get nowhere with this. There is no way to create an actual replay video out of the captured data, nor will you ever be able to transfer the replay to a server since it will grow HUGE even after a couple of seconds. Okay, you COULD implement a JavaScript based H264 encoder that you feed the images to and convert them to a mp4 video on the fly. Yeah. But always remember you are trying to RUN A GAME on 60fps at the same time.

 

 

If you really want to create a decent replay system to be integrated into game engines, then create a system thats able to store canvas states - "keyframes" - every couple of frames. That means NO image data but position, rotation, scale and what not of all objects.

 

You could then use a library like greensock to tween between the values when you want to play back the recorded replay data. When playing the data back, all values have to be feeded back to the engine.

 

That task is huge, complicated and surely a hell to develop. But its the only way to make replays available on all kinds of devices while keeping the captured data on a reasonably low amount.

 

And if you manage to create such a system, I'm sure there will even be developers out there that will pay you real, hard money to license it.

[MichaelD]

Well, first of all thanks for the input, your points seem valid. However I don't quite agree that there will be NO way to create viable videos as playbacks with JS. As JS engines keep improving and tools like workers and AnimationFrame exist. 

As you can see here, there is little we cannot do: http://www.techrepublic.com/blog/australian-technology/decoding-vp8-and-h264-video-with-javascript/#.

 

I am certain a better solution than mine will emerge, but saying "it cannot be done" is not a way to push technology further.

[Chris]

I already mentioned that you could h264 encode your images with javascript in my previous post

 

Recording video while a game is played back just doesn't make sense. Not even native games you are playing on your pc or console are doing this. It just takes up too much resources that are needed otherwise.

[MichaelD]

KineticJS already implements (although not officially supported) a nodeJS implementation that allows Kinetic to run server side, this has already been utilized to create image parsers, so with a single ajax call every time a "screenshot" is needed, the game or server hardly uses up any resources.

 

Besides a utilization with "canvas-shots" can be created to "film" a game of checkers where an image is taken on-player-move and not all the time. Then these images can be created into a filmstrip and the whole game would be viewable.

 

I think there are games that don't need 60fps all the time to run, for example board-games or trivia-games. 

[jpdev]

In those more static games it would also be relatively easy to just record the game-moves into a list, and store this list for a replay.

 

In addition to require so much less space than storing canas-screenshots for every move made it also allows for (re)playing the animation for each move the game might have.

[b10b]

I also suggest recording the input stream, not the output stream.  Even for realtime games using a fixed-interval-update means replays become trivial and easily compressed to either a keypress stream or touch / mouse coordinate stream.  "Playback" can be done client side through the same rendering system that was used on initial recording, or server side to a validation routine.  5 minute game sessions can be stored in Bytes, not Gigabytes and represent smart data / telemetry which is typically more valuable from a big-data perspective than encoded video.