How to achieve Hot Code Replacement (without Browser reload)?

[Nodragem]

Hello, 

I am playing around with webpack and I discovered the Hot Module Replacement (HMR) functionality.

Do you think that it could be used to hotswap code as in this PlayCanvas video:

 

I mean, that is quite fascinating, right?

If we can find a way to make it work in BabylonJS (and hopefully other HTML5 engines such as Phaser), I am happy to write an article in the documentation.

Useful References:

- doc of Playcanvas: https://developer.playcanvas.com/en/user-manual/scripting/hot-reloading/

- doc of webpack: https://webpack.js.org/api/hot-module-replacement/

- monkey-hot-loader, a library that abuses webpack HMR but I did not understand clearly what this is doing: https://jlongster.com/Backend-Apps-with-Webpack--Part-III 

[Guest]

Hey!

this is what we are doing for the playground :). Everytime you hit Run, we are calling engine.dispose and we recreate a new one: https://github.com/BabylonJS/Babylon.js/blob/master/Playground/js/index.js#L541

 

Then obviously there is the wiring that needs to use webSocket to control all different targets from VSCode

Pinging @Sebavan as I think he did something to hot reload from VSCode

 

 

[brianzinn]

HMR you can do right now, but you will be brought back to your initial state, so it is not that useful.  HMR is an essential part of my workflow.  I have had limited success (mostly because of physics on moving objects) with HMR and restoring an in-progress game and 3D level builder by replaying state changes on init, but I have a simple puzzle game.  I'll try to put up an example tonight.

[Sebavan]

Hello,

Are you trying to hot swap during development inside the Babylon code or within your application built using Babylon ?

Anyway @RaananW might be the man for that..

For me I have just added full page reload to prevent some of the state issues I was having or shader recompilation and so on to be broken.

[Nodragem]

@Deltakosh If I understand, you use HMR to reload the engine without reloading the page? but now, let's say I animate a cube, moving left and right forever. Can I change the color of the cube without reloading the engine, and hence, without interrupting the animation of the cube?

@brianzinn yep, that would be very nice if I could see an example of how you use it. Can HMR be used for the example I gave the line just above?  i.e. changing the color of a cube in real time (while it is moving). That would be very useful to make menus for instance.

@Sebavan during development, I would like to modify my code and see the change in real time, without reset of the engine (see example I give above).

[brianzinn]

@Nodragem I've made a demo for you.  I think you will be underwhelmed with this example, as I'm currently not able to, for example, maintain an animation.  One reason for doing it this way, is that I'm able to  change mesh (ie: box) size, which requires me to rebuild that part of the scene.  Nevermind the wierd markup in the demo, as I'm using React as well to wrap the BabylonJS objects, but this gives you an idea - you can super easily do it without React.  You can change the cubes color, move them around, move camera, change lights, change cube rotation axis/speed.  I have started on a reactive project using vue that does similar, but I think I will write an intermediary representation like proxies to share between the projects for full HMR even resizing meshes by updating vertices data - then you could do things like change cube colors during an animation.  I just am really low on time these days to pursue it at all

13 hours ago, Nodragem said:

That would be very useful to make menus for instance.

I am changing same instances of menus on the fly without HMR, too.  You can see that on the Panel (+2D UI) page.
https://brianzinn.github.io/create-react-app-babylonjs/with2DUI

The HMR was I just added in this changeset:
https://github.com/brianzinn/create-react-app-babylonjs/commit/6d9f6ba0db70b7ebd6f568ba1ea4ae8954d8fd48

I do some more involved HMR on the 3D level editor I'm working on, but it's just for development not end-user.  I'm managing with external state, if you are wondering.  I think that once you get started that it you will see it's quite easy to get at least what I have there - it took me more time to make that .gif than to add HMR - it's my first screenshot gif!! yay   Cheers.

[Nodragem]

Thank you for the demo

I am not very familiar with React.js or vue.js, so I am a bit confused with your examples. My understanding is that there exists a react-hot-loader, but there is no such thing for pure javascript, except the experimental monkey-hot-loader (here), so I am not sure how easy it is to hot swap javascript code ? .

Note that I am interested in HMR for development purposes. I just want to modify my parameters or functions in real time without reloading the entire game. I am going to play with HMR and try to figure out what can be done with it.

[Guest]

Please report back if you can 

[aWeirdo]

Hot swapping is basicly just re-executing a piece of code after it changes..

you can do it in the console, (PG is complicated because of editor, doesn't store as global variables.. etc etc)

Quote

Open console

->Right-click "g" (sphere)

-> Store as global variable

> tempX

-> tempX.material.diffuseColor.b = 1;

> Red sphere is purple.

http://playground.babylonjs.com/#PES2C6

I suppose it would be possible in PG.. look for changes to editor, only execute changed lines.. or something.. complicated..

[aWeirdo]

Hi again guys, /@Nodragem 

I had some spare time & decided to give it a go in the PG, not easy with the current setup, but works fairly well. 

It could maybe use a bit of cleaning.. and i'm sure it's a leaky AF and other issues will show themselves after some testing.. 
Anyway here ya go, I suggest not using setTarget functions as they will re-run..

All meshes, lights & cameras must have name's/id's set.
It will not update changes made to Mesh/Light/Camera creation strings

-Added updateXXXX variables ( lines 35, 36, 37 ), disabled updateCameras by default (setTarget issues)

http://playground.babylonjs.com/#TY8HCN#7

 

[Nodragem]

Hello aWeirdo,

Thank you very much for all your ideas, sounds great!

Can you give an example of how to use/do hot swapping in  the last playground you sent? 

EDIT: I tried again and it works! I just need to change values in the editor!

 

Edited September 28, 2018 by Nodragem

[aWeirdo]

Hi @Nodragem 
there's no networking, as it's in the PG,
so "auto-update" would perhaps be a better description,

but you can change / add properies and see the changes instantly without having to reset the scene (e.g. without clicking run)

I should add that it needs a lot of work. so expect bugs.

[Nodragem]

Hello,

Here is my attempt to hot reload an asset of my game (i.e. the floor texture) without reloading the whole game.

And I can't make it work ?, my code does detect the change of file and it does execute the change in material, but nothing change on the screen. 

 

Here an extract of the relevant code:

import '../assets/2D/dungeons_and_flagons3.jpg';


class Game {

    private _canvas: HTMLCanvasElement;
    private _engine: BABYLON.Engine;
    private _scene: BABYLON.Scene;
    ...
    private _maze: MazeLoader;

    constructor(canvasElement : string) {
        // Create canvas and engine.
        ...
        this._scene = new BABYLON.Scene(this._engine);
        ...
        this._maze = new MazeLoader(10, 10, this._scene);
    
        if(module.hot){
            console.log("Changed Detected!");
            module.hot.accept("../assets/2D/dungeons_and_flagons3.jpg", ()=>{
                console.log("Accepting the change");
                this.AcceptChange();
            })
        }
    }

    AcceptChange(){
        this._maze.floorMaterial.diffuseTexture = new BABYLON.Texture("./assets/2D/dungeons_and_flagons3.jpg", this._scene);
        if(this._maze.mazeFloor)
            this._maze.mazeFloor.material = this._maze.floorMaterial;
        console.log('texture udapted!')
    }

the module section in my webpack.config.js is as follow (the file-loader is the relevant one):

module: {
        // tell webpack what to use to compile typescript (i.e. ts-loader) 
        // and what not to compile (i.e. the folder node_modules)
        rules: [{
            test: /\.tsx?$/,
            loader: 'ts-loader',
            exclude: /node_modules/,
            options: {
                transpileOnly: true 
                // IMPORTANT! use transpileOnly mode to speed-up compilation
                // ALSO very practical when you don't want your build to fail because of typescript errors which aren't detrimental to your javascript.
            },
        },
        {
            test: /\.(png|svg|jpg|gif)$/,
            loader: 'file-loader'
        }
        ]
    }

my npm command is as follow:

"dev:server": "webpack-dev-server --mode=development --open --inline --hot"

 

When I change my file "dungeons_and_flagons3.jpg", the change is detected, and the console displays:

[HMR] Checking for updates on the server...
game.ts:40 Accepting the change
game.ts:49 texture udapted!
log.js:24 [HMR] Updated modules:
log.js:24 [HMR]  - ./assets/2D/dungeons_and_flagons3.jpg
log.js:24 [HMR] Updated modules:
log.js:24 [HMR]  - ./assets/2D/dungeons_and_flagons3.jpg
log.js:24 [HMR] App is up to date.

Hence, it seems that my game does detect the hot module reload and does execute the lines of code that updates the material...

What worries me is that log.js:24 announces the update of the module after my code accepted the change... that may explain why my texture is not updated.

Please feel free to clone the git repo here: https://github.com/Nodragem/Maze-Demo

[aWeirdo]

Hi @Nodragem 
try this and see what it says;

    AcceptChange(){
        this._maze.floorMaterial.diffuseTexture = new BABYLON.Texture("./assets/2D/dungeons_and_flagons3.jpg", this._scene);
        if(this._maze.mazeFloor){
            this._maze.mazeFloor.material = this._maze.floorMaterial;
            console.log('texture udapted!')
        }
        else{
            console.log('texture NOT udapted!', this._maze.mazeFloor)
        }

    }

 

[Nodragem]

it looks like that it says: texture updated!

I made a simpler example here, with just a plane and a sphere, and I tried to hot reload the texture of the plane:

https://github.com/Nodragem/babylonjs-webpack-boilerplate

You can see that the message is "texture updated!".

I've got the impresssion that AcceptChange() get called before the actual change, so when reload the texture, it reload the same one.

Or... maybe it is a cache issue?

[brianzinn]

I was thinking caching once you have made it that far. Does the browser attempt to redownload?  Did you try something like:

this._maze.floorMaterial.diffuseTexture =
  new BABYLON.Texture("./assets/2D/dungeons_and_flagons3.jpg?" + Math.round(new Date().getTime() / 1000), this._scene);

If that doesn't work - are you sure that your webserver is serving the updated asset?  I will take a look tomorrow.

I would also add this to disable offline:

if (module.hot) {
  BABYLON.Database.IDBStorageEnabled = false
  ...
}

 

[brianzinn]

I would also try this:

engine.clearInternalTexturesCache();

I am interested to help with the HMR stuff.  I have started already on some intermediate representations.  The factory methods ie: CreateBox(...) are not leaving me with something that I can easily update (ie: size), so I have resorted to this.  What you have with textures looks promising, so let's see if we can get that working

[aWeirdo]

I don't think caching should affect anything when you're creating a new texture with a different image file.

However, What are you changing the image file to?
It seems unclear from your code where you write-in the file change.
Aswell as the logs you posted, they say the file is the same as the standard one, e.g. dungeons_and_flagons3.jpg

// MazeLoader
this.floorMaterial.diffuseTexture = new BABYLON.Texture("./assets/2D/dungeons_and_flagons3.jpg", scene);

// AcceptChange
this._maze.floorMaterial.diffuseTexture = new BABYLON.Texture("./assets/2D/dungeons_and_flagons3.jpg", this._scene);

[brianzinn]

14 minutes ago, aWeirdo said:

I don't think caching should affect anything when you're creating a new texture with a different image file.

haha - now I have to download the github repo.  I expected an image with the same path would be cache optimized by either the browser or BabylonJS.

[aWeirdo]

@brianzinn true, if it has the same path & name, caching will mostlikely mess with it,
but speaking of hot-swapping code, i expect the path/name to be changed to a different image file, not the image file to be replaced ? 


And as i mentioned, i don't see the file name being changed anywhere, the logs even output the same file name, if you update imageA to imageA.. it might reload it, but it'll still look the same heh
I'm afraid i don't use webpack or TS so i can only comment on what i read in the code.

Let us know what you find

[brianzinn]

46 minutes ago, aWeirdo said:

I'm afraid i don't use webpack or TS so i can only comment on what i read in the code.

I use TS and Webpack and also HMR every day.  Except for some battles with webpack, I mostly let them magically do things behind the scenes. I regularly use module.hot.accept for maintaining state and changing data processing logic, so it's really cool to see this working in WebGL in running code (my screen example above rebuilds entire scene with HMR)!  I can confirm that my first suggestion of adding a cache buster that the texture image is swapped!!
https://github.com/Nodragem/babylonjs-webpack-boilerplate/blob/master/src/game.ts#L30
change to:

let imageSource = "assets/2D/dungeons_and_flagons3.jpg?" + Date.now();
material.diffuseTexture = new BABYLON.Texture(imageSource, this._scene);

I see it's only being swapped once and then the acceptChange() isn't called the second time.  I will try to investigate more, but am really busy these days.  Great project idea.  I'll definitely make some time here as this is an area of interest for me and a crucial part of my workflow.  Next step for me is to look at texture internals of BabylonJS and see if we can swap there without using sledgehammers like "engine.clearInternalTexturesCache()" - maybe some PRs for engine are needed for this access.  Also, the base example should have two meshes to ensure the texture swap is scene wide.  Cheers.

[brianzinn]

In case anybody is wondering what hot reloading textures looks like, I have made a screen capture.  This saves you the trouble of downloading @Nodragem's repo, adding that one line patch and running it yourself, even though that only takes a few minutes   Chrome on top, Edge on the bottom - on the right dragging a different image up into our webserver /assets/ directory and having it hot reloaded into WebGL + BabylonJS.

The huge advantage with this technique (over the HMR you see in my screen capture above) is that we are replacing a running instance (not reloading cameras, lights, other assets, etc.) and this is where you want to be if you are ie: making a game, whereas my first screen capture (while it does use HMR) is rebuilding the entire scene.  Some advantages of scene rebuilding is that I can add/remove to scene, replay state (in my game, I can fully restore from state on startup) and change things like constructor variables and have them take effect, etc.

@Deltakosh - Reporting back.  I am about to start digging through the engine cache texturing to see if that is how we could/should be replacing textures for entire scene.

[Nodragem]

@brianzinn Whahou, that is amazing! that is exactly what I wanted to do. Thank you very much!

Concerning your suggestion of integrating hot texture reloading in the engine:

- wouldn't it hinder performance to have a watcher watching for texture change?

- could we switch it off when we build the production version?

- can we make a general method that works for any game assets: mesh, animation, sounds, fonts, etc...?

- next step: reloading js code? if we had the concepts of scripts (as in BabylonJS editor http://editor.babylonjs.com) we could reload them as assets; integrating the "hot-reload's accept change" in the Script class. But for now, we can test if we can reload any modified js file.
 

If I try to summarize my game developer's dream dev-tool, it will be in 4 parts:

- reactivity as in React / Vue.js: no need to make a GUI to control data, you would have a debug mode where you can tweak all existing data (see https://youtu.be/ZCToaOpId0w).

- hot asset swapping/reloading: the game updates changes in sound, 2D, 3D, animation assets without need of restart (as we just did with a texture)

- hot code swapping/reloading: the game updates changes in game logic / scripts without need of restart (see https://www.youtube.com/watch?v=SmXi0sLNWQw)

- no performance cost when disabled: all these dev-tools could be disabled and wouldn't hinder performance once disabled.

 

@aWeirdo I was not trying to make hot code swapping yet. I was just trying to swap a texture. Now indeed, the next step is to hot swap a js file that animates an object; so that I can change my animation in real time. To swap between existing files (texture1.jpg and texture2.jpg) could be done with a drop-down menu in-game, but here the idea is more of updating the game with the most recent assets/scripts without restarting the game.

To illustrate how that can be used in game development, you could:

- play the game and write the IA of the enemy at the same time. You would tweak their behaviours while testing their behaviours.

- test different sounds, change how your code tweaks these sounds while playing the game; until you think your sound FX feels good.

- tweak your jump method in your new platformer without having to exit and restart the game every 1 second!

- position and animate menu items in your GUI without restarting the game.

 

 

[aWeirdo]

What i understood from the beginning of this thread, was code hot swapping, maually re-writing or changing code and live execution, thus the confusion as i was expecting a different image path or name (to be written in to the code), not a different image at same path/name ^^

But either way, i think it's pretty cool  

[Guest]

@brianzinn This looks REALLY promising