[EXPERIMENTAL] HTML5 Game Compressor (SWF equivalent)

[austin]

This is something I've been tinkering around with for the past few weekends. One of the reasons Flash games spread so well across the web is the fact that they are self-contained in a single SWF file. HTML5 obviously doesn't have this advantage - zips are straightforward enough, but a lot of the Flash game portal owners I speak with don't typically like it...

 

HTML5 games technically can be packed into a single html file by using Data URIs instead of URLs - basically packing all of the JavaScript, image and audio data into the files themselves. In theory, it's pretty easy to do, but with so many game engines out there loading resources in different ways, there ends up being some weird use-cases to deal with.

 

I've built something to go ahead and do this - it works pretty well with a few game engines right now. If you want to try it out with a working game, use "http://slime.clay.io/index.html" as the URL.

 

Try it here.

 

It lists the advantages on the page - the virality/distribution part is the obvious one. Another interesting thing this allows for is encrypting the Data URIs with the host name as the key, and decrypting on the client-side. This just makes it so all of your assets aren't readily available for folks, and hopefully is a bit better of a site-locking mechanism. Of course, being JavaScript, it's still fairly easy to work around it.

 

With some game engines, it's incredibly difficult to work in the data URIs (GameMaker is the first to come to mind). For others like Construct 2, audio is loaded in via an xmlhttprequest, so that's pretty much a no-go (the audio...images work fine). In fact, for most engines audio is difficult to work with since they don't have strings for "audiofile.mp3", and rather do something like "audiofile" and tack on the extension based on the device being used.

 

With that said, I'd love to hear what everyone's thoughts on this are. If you try out your game and it doesn't work quite right, just use the "Report Problem" link. There's a lot of engine-specific code to work around, and I'll likely have to do the same for the popular engines.

[soybean]

Wow.. this is utterly brilliant . Very nice idea to pack the pngs and everything to a single file!

 

Edit: Btw, I've tried compressing one of my game made with Construct2, with audio, and it runs well on desktop browser, even the audio!

[Chris]

Hm, honestly - I don't like it. For a number of reasons:
 

• Its not compressing
  You are packing everything inside the HTML file, sure - this reduces the number of HTTP requests required to load the game, but at the same time you need to base64 encode every image, which blows all images up by around 60% in filesize. Sure - when your HTML page is delivered GZIPed this will reduce the size again by around 30% but you got the numbers...
   
• Browser caching destroyed
  Whenever you load a game this way, the browser HAS to completely download the game over and over - there is no speedup in loading time after the first run of the game since there are no assets for the browser to cache.
  Also, even if you manage to load your package via AJAX and inject it afterwards (browser will cache this), a player still needs to re-download the WHOLE package if you change the smallest sprite inside it.
   
• May cause trouble on mobiles or older pcs
  Browsers are able to do much more efficient memory management when assets are divided into different files. When you pack everything into one big HTML file, especially mobile browsers will crash at some point.
   
• Site locking?!
  Come on, a simple search-and-replace disables this...

[austin]

There are definitely some drawbacks to it.

 

The base64 issue is valid, can't do too much about it. 

 

Browser caching isn't completely destroyed - the static HTML can be cached. Granted, changing a single sprite will require the entire package to be re-downloaded, but that's an issue SWF's deal with as well.

 

To be honest I'm not sure of the effects of a large single file like this on mobile and older devices, so I'll trust what you have to say on the matter.

 

Site locking is a bit more difficult than a search and replace, they'd have to decrypt each of the encrypted data URIs with the right salt (which for now is just the window.location.host) before replacing. But yes, it's definitely not bulletproof by any means. It just ups the bar a level so your standard portal owner who knows a bit of JavaScript probably won't be able to figure it out.

 

I think where this is right now, it's a "nifty" tool with the easier distribution and even being able to download a single self-contained file to run a game. Of course, nifty does not equal viable. I'm still a bit skeptical myself on whether or not this is worthwhile, I wanted to get it out to the community for their (your) thoughts and input on maybe improving this to the point where it's a viable solution. 

[Chris]

Dont get me wrong - I really like the idea! I experimented with some scripts that pull all resources through a single ZIP file and then extract it client-side, too

 

I can't say where the border is where mobile browsers start to crash. There was a HTML document sbdy posted on hacker news some time ago that made mobile safari crash on the iphone but I can't exactly say when this happens in "real life".

At least it is more processing work for the mobile devices since separate resource loading is certainly optimized and the in-line resources need to be extracted and decoded, first. But again: I have no measures how big the performance impact of this is.

 

I just did the following to remove the lock (game was locked to the 'clay.io' domain):

 

String replace all: window.location.host => 'clay.io'

 

The Game worked on my localhost like a charm after that

[austin]

Whoops. Yeah, I guess that's way easier than I'd thought I'll have to think of another way to beef that up a bit.

[rich]

I'm not 100% convinced that an equivalent of a SWF file is needed these days. They were a product of their time really. Portals grew up around the concept because it was easy and existed, but I wonder about the true value of trying to shoe horn all games into that format in 2013. I hear the argument that it helps games to spread, but the lack of a shareable format has never seemed to hurt the virality of YouTube videos for example.

 

Getting them embedded and working in the portal site is definitely more of a headache, but that could be overcome with a unified packaging format (something the guys at Adult Swim were working on) rather than an asset globbing system.

[Ezelia]

I think it'd be interesting if we work on a "standard" HTML5 package structure that can be used for games.
if a standard structure is defined, we can then imagine tools to build it and scripts to deploy it.

what do you think ?

[rich]

Yes I agree, like I said there is/was an initiative already running to work on this but I've no idea what the current status is. I ought to find out really.

[austin]

I like the idea of a standard package format, but the 'big' companies seem pretty stubborn in this regard. Really all that's needed for a standard packaging system would be the files and a manifest file. The problem is, Microsoft, Mozilla and Chrome have all done this, but all of the formats differ.

 

If we were going to try and create a standard structure, I'd say we should pick one of those three, rather than creating a new standard. http://xkcd.com/927/ 

[Raiper34]

Hmmm very interesting

[Paul-Andre]

What you can do to prevent having it worked by search and replace (but this is totally crazy) is to have the window.location.host​ used to unlock a script that is then evaled to decompress the other resources, using window.location.host​ again.

[austin]

This is pretty cool and the better approach for this. Zynga released something similar too: https://github.com/zynga/grunt-cruncher

[dhaber]

I like what you are doing here, but base64 greatly increases the file size.   I experimented with something similar, but I haven't had the time to do it properly.  The proof of concept did work though, and I think it may be a good approach.  My technique was to store all the assets in a single PNG file as data. (Meaning, not an image you'd want to look at.)  This gives you a single file that is compressed and has everything you need for the game.  You do need a javascript loader for it as well, but each game is stored in a PNG, and the same loader can load any game.  You can do all sorts of obsfucation and encryption as well, but since the files do need to be unpacked to be used, I don't think there is any way of really keeping the assets secure.