Redux, a global scene and scene recreation

[promontis]

I'm currently building an interior design app using BabylonJS. The app is build with React and Redux and contains a BabylonJS canvas. When I add interior elements in the UI, they are also placed in the 3d scene. 

By adding an interior element, an action (stating what has changed) is published from React to Redux, where the redux store/state is being updated in a reducer using the action. I've tried updating the 3d scene according to the updated state in Redux, but it gets messy pretty quickly since you have to figure out what has changed. Since then I moved to updating the 3d scene in the reducer by using the action, since that describes what should change. 

How I update the scene using Redux:

I'm using the Babylonjs React component as described here: https://doc.babylonjs.com/resources/babylonjs_and_reactjs

When the component is mounted, I also set a global scene variable, so that I can access the scene in my reducer. In the reducer I map the action to BabylonJS calls for creating meshes/materials/etc. The interior elements all have ids and I set those ids to the meshes/materials, so that I can later update the meshes/materials when an interior element is updated. This seems to work great, and also be in line with what I've been reading in this forum, as people are recommending to use plain vanilla BabylonJS calls.

Since then, I've got a requirement that the scene should not always be visible in the UI (it currently is); it should be in a separate tab. Now the problem is that the canvas isn't loaded until I visit that part of the UI, thus the global scene variable is never set, and the reducer can't update the scene (it is null). 

Now I try to actually formulate a question around this problem, but I find it hard to do so. I am unsure if my current setup on how I route Redux actions to BabylonJS is the correct one; it is plain vanilla BabylonJS, but I've seen implementations on this forum and Github where people use a more declarative/ReactJS way of constructing their scene. I've also read various posts on this forum stating that you should not use ReactJS with BabylonJS, because of performance reasons as you need to recreate the whole scene; best would be the update the already created meshes/materials like I'm doing now. Furthermore, I've also seen people recommending to implement an Entity-Component-System (ECS) around BabylonJS, but from what I understand, that recreates the whole scene as well.

How would you tackle this problem?

Thnx,

Michel

 

[dbawel]

@promontis

I don't know how familiar you are with working in react or preact, however, it sounds like you know what you're doing. You must be declaring your canvas in the index.html file, however, there's allot of declaration that need to happen in your 'app,js' files. Also, there could easily be an issue with styling - which sounds just as likely. Without viewing the full code, as I hope you know - it's impossible to debug. In my opinion, forget about the documentation for using react.js with WebGL, and simply create your canvas, make certain it appears when you need, and then built the WebGL code using the virtual DOM in a seperate file. Always works for me... so far.

DB

[brianzinn]

To answer your question, all of your scene state needs to be in the reducer and you need a way to restore/hydrate a scene from that state.

I maintain the whole scene in the reducer, so I can even do/undo.  I have a 3D level builder that maintains dual state - in Redux and also in the BabylonJS scene.  That is not ideal as they can get out of sync with bugs, but since I need to be able to save state then go backwards maintaining the state only outside of the reducer did not make sense for my scenario and I did not want the reducer to need access to objects in my scene, so my objects generate state to be able to restore themselves as well.  Sort of follows the memento design pattern.

Even though my BabylonJS game that uses Redux is a turn-based game - the fact that physics is involved meant with this design a reliance on two-way communication.  That's why I needed a redux middleware to intercept redux events in my BabylonJS scene.

I like to think of it more as an Event Sourced system where the state of changes can be restored into a scene. Like hydrating a series of events in CQRS view model.

[promontis]

@brianzinn  do you perhaps have a small code snippet available? Seems like you already figured this out! :)

If I understand you correctly, would it work if I were to create or use some kind of redux middleware (perhaps https://www.npmjs.com/package/redux-action-replay-middleware) that stores all the actions. Then, when the scene is mounted on the screen, I would replay the actions to restore the scene. Indeed, much like CQRS. 

Did you build the redux middleware yourself or did you use an existing one? 

[dbawel]

@brianzinn, @promontis,

As often is the case, I simply glanced over the description - and didn't fully digest the issue. Obviously an issue with Redux, and as was mentioned above; I'd also like to see any available code from @brianzinn which demonstrates the capabilities of the dual state and the structure which allows do/undo. This would also help me greatly; if he's able to share enough code to make sense of the unique approach described. My questions are exactly the same as the post above. I'm guessing he doesn't have time to write descriptions for code snippets, but I'll take what I can get. This is one of the most interesting posts I've read in quite a while...

Cheers,

DB 

[promontis]

@dbawel yeah, I was wondering if my question was clear enough  

I'm mostly a backend developer, so when @brianzinn made a comparison with CQRS, I could visualize how that would work. I think most people struggle with how to hook up the scene state with external state or actions/events. I've searched the whole forum for an answer, but couldn't find a concrete enough example. I did find the recommendations as posted earlier.

[dbawel]

@promontis

I've only been working in React/Redux for about 6 months. Delivered projects at Sony, but still learning. I hope @brianzinn is able to post a few code bits. Sure would help me...

DB

[dbawel]

@promontis

Actually, I shouldn't have told you to ignore the babylon docs for react, as that's the middle-ware I've used since I've been working with React. You had said you read the docs and still needed answers. I'll attach a bit of code with good commenting using the 'react-babylonjs' middle ware, and hope it might provide some insight. 

DB

electron_bck2.txt

[brianzinn]

4 hours ago, dbawel said:

I'll attach a bit of code with good commenting using the 'react-babylonjs' middle ware

Yes, that's my npm, but your code sample isn't using the middleware for propogating state to BabylonJS.  I did go through many iterations making different mistakes before coming to my current conclusions, so am excited to see how these fare against other applications.  My original inspiration was largely the "Time Travel" seminal talk by Dan Abramov.  If you come up with a use case then I'm happy to build a repo - maybe a TODO app; I can explain here, but there are some caveats better shown in code.  I'm away from computer for the next couple of days.  cheers.

[dbawel]

@brianzinn

Do you ever make use of 'react-thunk'? I'm considering a test, but not sure what type of project/scene might take advantage of this. Also, any experience with preact? I'm wondering if I should apply a component such as 'preact-compat'. Any experience with this?

I checked out your Git-Hub Repro, and now I know where I'd like to be as it relates to React/Preact experience level one day...

Thanks,

DB

[promontis]

@brianzinn If you can create a repo for a todo app, that would be awesome! I would imagine that you'd have a React app where you can add todo items. Every todo item is just a string, and prints in react via an unsorted list. In BabylonJS we could create a box for each todo item, and stack it on top of each other. That would probably be enough to show your solution, right?

[dbawel]

@promontis

Have you checked out @brianzinn's Github Repo for using React/Redux? It's excellent!

DB

[brianzinn]

9 hours ago, dbawel said:

Do you ever make use of 'react-thunk'?

No, I always use react-saga instead.  Thunk to me has side-effects, which breaks the rules of writing reducers in a "functional" way.  I can also write tests for redux-saga (ie: mock return data from an ajax call) - it's a bit of a hurdle to learn how to test generator functions, but I have written some fairly complex react apps using sagas.

8 hours ago, promontis said:

If you can create a repo for a todo app

I will.  Give me a bit of time as I am quite busy this week. Looking forward to your feedback.

[promontis]

@dbawel I've also dropped redux-think in favor of react-saga. Same arguments.

@brianzinn no problem... take your time. Was thinking about this today in the car... the use case doesn't show how your re-hydrate the scene, right? Should we add the requirement that there should be a button that removes the canvas and add it again? Feel free to add/remove requirements if needed... I think you know best what requirements will result in the best code communicating your solutions. 

 

 

[dbawel]

@promontis

I didn't want to take away your your answer as a top tier on the forum under your name, however, you made it clear that you would be away from the forum for a couple o days... and still you're willing to help others. My respect goes out to you 100 fold.?

I hope you don't get inundated with React/Preact questions...

By the way, I LOVE this forum!

DB

[brianzinn]

On 6/26/2018 at 12:53 AM, promontis said:

the use case doesn't show how your re-hydrate the scene, right?

There will be multiple routes, so the component will be mounted/unmounted.  It's the reducer that maintains minimum state - I'm assuming you maintain your "Single Page Application" throughout and that state is available when switching tabs in your app.

[brianzinn]

I created an example that shows an unloaded scene being reloaded with the todos, which is your original request.  Make sure you are using the NavLink from 'react-router-dom' and not from 'reactstrap' - that briefly stumped me.   The react and babylonJS are kept in sync.  There are bugs lurking if you use a filter other than "show_all", but wanted to share as it's 2am!  It's a copy of 'create react app' + react_redux simple example + babylonJS, so 3 commits; 1 for each phase to make it easier to follow.  I ended up going a different way in many respects with my 3d level builder (http://robolo.co), but want to share this simplified way.
https://github.com/brianzinn/create-react-app-babylonjs

[promontis]

Awesome sample! Thnx so much for this! 

Basically you are using the normal Redux React flow (mapStateToProps) to pass the state and build up the initial scene, but only when the scene is mounted. New todo's are still being mapped to props, but the scene is not mounted again, so those won't be recreated, which is a good thing. Since new todo's won't be processed, you subscribe to the todo actions, and process them yourself.

Your 3d level builder looks super cool btw! Do you have some pointers on what you end up doing differently for it? I'm especially interested in the undo/redo functionality. I'm thinking of using https://github.com/powtoon/redux-undo-redo for this, what do you think? It is publishing undo-actions when you undo, which will end up both in the store (for when the scene is mounted) and in the subscribers. Should work right?

 

[brianzinn]

I haven't looked into that project before.  There is also a basic sample of undo in the redux repo.  I implemented my redux routing to be able to use something like this or redux dev tools (even docked):
https://github.com/reduxjs/redux/tree/master/examples/todos-with-undo

With your interior design app, you are faced with the same kind of issues that I have thought through.  When you, for example, spin a couch 90° clockwise - when you undo that you really want a compensating action in BabylonJS.  Same as a compensating entry in accounting.  I like how Event Sourced systems are write only, but I don't think we can/need do that - we are allowed to forget the actions we have undone once we "do" another action, right?  Combinations of translations and rotations are very order dependent!!

What I ended up doing differently mainly was add complexity by maintaining the entire state using redux-saga and pushing some state into my objects (for serializing the scene for "save" functionality).  You can save 3d levels you create and load them on another computer later.  I'm adding and organizing objects in a scene, so we share a similar goal.  I've spent more time rebuilding than adding features!

Well, that was a bit of blabbering.  I'm happy to work with you on building the sample out to a useful undo application - we may need to change from TODO to a more real world concept.  Not sure if it will be useful as a reference application as there are lots of use cases.  I'd like to have a more useful scenario project and GUI Slider for time travel as an end goal.
 

[promontis]

Quote

With your interior design app, you are faced with the same kind of issues that I have thought through.  When you, for example, spin a couch 90° clockwise - when you undo that you really want a compensating action in BabylonJS.  Same as a compensating entry in accounting.  I like how Event Sourced systems are write only, but I don't think we can/need do that - we are allowed to forget the actions we have undone once we "do" another action, right?  Combinations of translations and rotations are very order dependent!!

From what I can see, this is how https://github.com/powtoon/redux-undo-redo works. It works differently than other redux-undo libraries in that it throws compensating actions like you described!

Anyhow, if successfully integrated your sample in my own interior design app... it works great! Thnx for the help!! 

[brianzinn]

Thanks for sharing - I hadn't actually taken the time to look at the repo, so my bad there!  Did you end up using redux-undo-redo?  I am trying to think of a simple scenario for updating that sample github babylonJS repo with undo/redo functionality.  Maybe just a simple mesh that can be rotated and translated with React and BabylonJS buttons - seems like a TODO app is too trivial as the scene can be generated too easily from state.

[promontis]

I've not yet used redux-undo-redo, but will try to use it in a couple of weeks. I currently have one problem with (re)calculations and the order in which the subscribers and the reducers are being called; I'm going to handle the actions in redux-saga, and publish 'success' actions. These 'success' actions will be handled by the subscribers, so that the order of handling is correct again (reducer need to do work before the subscriber).

Will let you know if it works.

[promontis]

To correct the order of actions, I now do:

export default function* rootSaga() {
  yield all([
    modelingSagas.watch(),
    calculationSagas.watch(),
    previewSagas.watch(),
  ]);
}

- the modelingSagas fetches and initializes the initial model.

- the calculationSagas can recalculate the model when values are changed 

- the previewSagas listens on actions on the model and dispatches one action named 'updatePreview' which contains the last state of the model and the originalAction. As in:

export function* watch() {
    yield takeEvery(modelingTypes.MODEL_INSTANCE_CREATE, updatePreview);
    yield takeEvery(modelingTypes.MODEL_INSTANCE_UPDATE_PROPERTY, updatePreview);
    yield takeEvery(modelingTypes.MODEL_INSTANCE_UPDATE_ARRAYPROPERTY, updatePreview);
    yield takeEvery(modelingTypes.MODEL_INSTANCE_ADD_TO_ARRAYPROPERTY, updatePreview);
    yield takeEvery(modelingTypes.MODEL_INSTANCE_UPDATE_COMPONENTPROPERTY, updatePreview);
    yield takeEvery(modelingTypes.MODEL_INSTANCE_DELETE_PROPERTY, updatePreview);
}

function* updatePreview(action) {
    const state = yield select();
    var house = state.getIn(['modeling', 'instance', 'house']).toJS();
    yield put({ type: 'updatePreview', house: house, originalAction: action });
}

Notice the yield select() which gets the last updated state. I tried to do it via props, but it doesn't update when I want it to update. With redux-saga I have a lot more fine-grained control over what happens when.

Then this is my editor code:

 this.actionHandler = (action) => {
      if (action.type === 'updatePreview') {
        ComponentReducer(action.house, action.originalAction, this.scene);
      }
    };

So, I only handle the updatePreview action. The ComponentReducer is updating BabylonJS. Note it is not actually a redux reducer... I might need to change that name

The ComponentReducer basically switches on the (original)action and update BabylonJS accordingly. action.house is used for calculation.

[brianzinn]

I don't have a ComponentReducer concept or direct communication from my sagas to BabylonJS.  I ended up communicating with events instead.  I think I started your way and ended up with lots of race conditions.  ie: maybe the "request" runs an animation or is slow to load the model/material and is created later, so not available for modifications.  Also, I am generally following the redux-saga advice and avoiding Select as your updatePreview() uses:
https://github.com/redux-saga/redux-saga/tree/master/docs/api#notes-12
Preferably, a Saga should be autonomous and should not depend on the Store's state.

ie:
modelingTypes.MODEL_INSTANCE_CREATE_REQUEST (listened to by BabylonJS, ignored by Saga)
modelingTypes.MODEL_INSTANCE_CREATED (generated by babylonJS, listened to by saga and reducer - ** you can add BabylonJS "id" to action for hash lookups/dictionary/maps - even when rehydrating your scene).

Tense has significance to help me figure out what is going on when I revisit the code - so xx_REQUEST is present tense and xx_CREATED is past tense.

I never rebuild my scenes as it is too slow, but this may be my downfall as I do have a complex DUAL state sync going on!  Anyway, the main thing is that you have it working, which is great.  Managing dual-state is very error prone, so I am not recommending it - it's a pain point that I am searching for a solution to.  Thank-you for sharing your progress.

 

[promontis]

Yeah, I stumbled upon the same advice of redux-saga saying I should not depend on the Store state. Still figuring out if I can adhere to that advice. 

The problem I currently have is that I need to do calculations. Not only simple ones, but also over the whole model tree. To do recalculations I need to have a dependency graph for the calculations, which can't easily be expressed by default redux/react ways; AFAIK normally calculations are done in selectors, but those are mainly localized calculations. Furthermore, and probably the most important reason, selectors do not go hand-in-hand with what BabylonJS wants; it needs actions/events, as in it needs to be updated in an imperative way, not a declarative way.

From what I understand in your example with the xx_REQUEST and xx_CREATED is that you send actions (commands, your xx_request) to BabylonJS. BabylonJS then processes those actions and publishes created actions (events, your xx_created). 

I don't think that will work in my app, as I have both a 3d and a 2d scene (which would publish that it is created?). It might be different for your app. In my app the model is where commands are send to, as it also is responsible for denying invalid commands. When the model is updated, it sends events to my view/read models, which are the 3d and 2d scene, but also other textual representations of that some model (like a financial overview, and a planning).

I don't also don't think we can work around not having dual state. Heck, I even have three states. However, the model is always in control (via reducers and saga). Calculations and recalculations also only apply to the model. Ofcourse, it will eventually update babylonjs (and the 2d scene), but I don't see any other way except than having babylonjs wrapped in react and build the scene in a declarative way. Otherwise, we will probably always have more than one state.

I've found that I have a lot more control when I push all logic to the saga. I think I can do all animations based on the action data, but I'm unsure yet. The nice thing is that you can create a specific saga for handling BabylonJS, and dispatch custom babylonjs actions, on which only babylonjs will react. With this I seem to have a lot more control on when BabylonJS needs to update what. But like I said, I'm unsure on how animations fit in this.