What's next?

[GameMonetize]

Hello dear community, we are about to ship 3.1 (early december). I'm really proud of what we did so far but I want more for 3.2

SO I'm wondering what could be useful for you regarding ease of access and simplicity.

Please use this thread to give me any idea / feedback. I cannot guarantee that everything will be developed but I can promise that everything will be considered seriously

[brianzinn]

There is an interesting comment recently from max123 regarding cameras (http://www.html5gamedevs.com/topic/33942-threejs-or-babylonjs-for-room-design/?do=findComment&comment=194879).  I find myself doing awkward checks on cameras especially in VR.  I don't know if unifying the cameras even more is something worth considering.  I like the work you did adding behaviours recently - perhaps that is one way to go.  It is already done a lot like behaviours the way inputs are dynamic.

Also, AR.js as a supported plugin  

[Dad72]

- For cameras, can be created a class that groups all cameras using a unique camera name (CustomCamera for example), but in the signature add the type of camera that you want to use. After the properties, can be done as for "MeshBuilder" with options {} in the signature too.

- A tutorial part very detailed step by step on the shaders would be appreciated by many, because it seems really complicated, but we know what they can do, it would be cool to learn them gradually starting from 0, in explaining how it works, what is the purpose of each property. in short, a lot of details.

 

 

[jerome]

As workers coupled with sharedArrayBuffers seem to be a good plan to speed things up while keeping JS (no wasm not asmjs needed), maybe a way to integrate/name them easily would be nice ... knowing they usually require a separate file, a way to start them, to stop them, etc. Well a bunch of new things that need to be thought before.

[Raggar]

57 minutes ago, jerome said:

As workers coupled with sharedArrayBuffers seem to be a good plan to speed things up while keeping JS (no wasm not asmjs needed), maybe a way to integrate/name them easily would be nice ... knowing they usually require a separate file, a way to start them, to stop them, etc. Well a bunch of new things that need to be thought before.

Do you know where workers could be useful other than physics?

Me and @BitOfGold did some basic testing of sharedArrays and CannonJS, but I wonder what else could be "off-loaded" to separate threads.

https://jsfiddle.net/gak8320L/15/

[jerome]

here for example :

or anywhere where we need either to iterate over dozens of thousands objects (standard particles) or where we need long or intensive computations (maybe texture generation from capture devices, collisions, path-finding algos, etc)

 

[GameMonetize]

@brianzinn can you share what kind of weird code you had to do?

[JCPalmer]

Do not really have a "what" at this time.  A "how" might be to keep an eye on how big the main file is getting.  No specific breakouts in mind outside of maybe VR/AR, since it seems to be an area of expansion.

[brianzinn]

@Deltakosh - My first weird experience came here (http://www.html5gamedevs.com/topic/33478-highlighter-not-working-in-vr/). But to answer your question - I have my game using a combination of code from VRExperienceHelper and the excellent 'zero to hero' PG from davrous (http://www.babylonjs-playground.com/#E0WY4U#18).

In the PG to cast a ray:

if (!camera.leftController) {
   ray = camera.getForwardRay();
} else {
   ray = camera.leftController.getForwardRay();
}

In the PG to create a VR camera:

if (navigator.getVRDisplays) {
  camera = new BABYLON.WebVRFreeCamera("WebVRCamera", new BABYLON.Vector3(0, 2, 0), scene);
  scene.gamepadManager.onGamepadConnectedObservable.add((pad) => onNewGamepadConnected(pad));
  camera.onControllersAttachedObservable.add(() => onControllersAttached());
} else {
  camera = new BABYLON.VRDeviceOrientationFreeCamera("WebVRCamera", new BABYLON.Vector3(0, 2, 0), scene);
    }

Like how you gracefully degrade to webgl1 if webgl2 context isn't available - so maybe camera.getForwardRay() could check for camera.leftController and use that, if available.  Or WebVRFreeCamera would downgrade to VRDeviceOrientation.  I am imagining how this could be easier for somebody that doesn't have both kinds of devices to test with to make these VR scenarios easier.  Something like a VRUniversalCamera that maybe auto-connects controllers?   

The VRExperienceHelper has encapsulated a lot of the logic to help me understand how to accommodate a proper VR controller, which I don't have - which is great (to have this helper)!  I am trying to build for Cardboard and GearVR, so using a gaze approach.  I am probably doing it wrong, but I am listening for full screen to exit to detect when camera should change back (assuming we only want VR in full screen).  Is checking currentVRCamera instanceof the way to detect if controller is in use - nothing else is public?  I want to wire up code once VR is entered, but onEnteringVR is too early, so ideally I would like an onEnteredVR with maybe extra info like 'this._webVRready').  I sometimes have issues when I change the device orientation that the VR button is off screen and the canvas is not switching from portrait to landscape.  I *do* realise this is an open source project and I can contribute.  I am more waiting to see what comes out of 3.1 as those recently assigned will probably fix many of these and then to contribute from my own experiences, if anything is missing.

So, my request for 3.2 was to make it easier to target VR.

[GameMonetize]

Hello some comments:

-The WebVRCamera already automatically connects the controllers. But I agree with you we should overload the getForwardRay. Good idea. Fancy doing a PR?

- Gaze support will be added to the VRExperienceHelper: https://github.com/BabylonJS/Babylon.js/issues/2498

- For issues not addressed, I think it is a good idea to create github issues so we can track it and discuss them. the are all good ideas (feel free also to update the issue #2498)

[brianzinn]

Yes, I can do a PR for that and more.  I feel better doing a PR with actual hardware, so let me first buy a proper headset on Thursday when I'm downtown Vancouver to listen to your talk .

[SpaceToast]

I'd love to see any of the following:

• LOD enhancements:
  • LOD blending. Not sure what the "cheapest" method to reduce pop-in is. 1-bit pattern masks?
  • Load and dispose at LOD. A "set it and forget it" method to load a model/scene file into the current scene when needed, then dispose of it when no longer needed. This would be quite helpful for building large/streaming type worlds, but useful even for small scenes to reduce load times. (Everyone hates a loading screen!)
• More user-friendly shaders:
  • Built-in primitives. Rather than Road, Fire, Brick, etc. a set of option-heavy primitives like SimplexNoise, CellTurb, Gradient, Tiles, SphereMap, PlanarMap, etc.
  • Primitive combiners. In prerendered 3d, nesting nodes like gradients, noise patterns and falloff types allows for the creation of extremely rich non-repeating textures from very short descriptors. Could something similar be done to generate pixel shaders?
• A simpler method to remap keys and other input methods (as opposed to just adding and removing keys).

(On background, I'm working on a virtual museum of Earth history, in which every step you take represents a million years. So, a mesh-heavy scene that's almost 2 miles long. I'm an old hat at animation, but new to realtime 3D. What I'm mainly finding is that the bottleneck isn't the amount of content to be rendered, but getting that content to the user's browser in the first place.)

[mr_pinc]

Node based material editor

Node based logic editor (ala UE4 blueprints)

A boy can dream can't he

[GameMonetize]

Node based material editor: http://johann.langhofer.net/PBRComposer/

Node based logic editor: http://doc.babylonjs.com/resources/using_actions_builder

[GameMonetize]

@SpaceToast: I really like the LOD blending and Load and dispose at LOD ideas. Fancy creating issues for them?

[JCPalmer]

What about a method of storage for skeleton poses, and a library system to load them?  This would allow load independence from the skeleton itself.

A Pose, as illustrated below, would contain all bone matrices which have a value which is different from Rest / Basis, rather than a separate, frame based, animation for each bone.  Organizing by pose is much easier for key-framing.  If you wish to remove unimportant frames, then there is only one animation to pull it out of.  Pose based is how every other skeleton system I have looked at is:  Blender, BVH, FBX, & Acclaim.

    new QI.Pose("idle-1", lib, {
        pelvis: _M(0,0,1,0,.8467,-.5321,0,0,.5321,.8467,0,0,7.8706,.7904,-.1289,1),
        upperarm_l: _M(.3482,-.9374,-.0096,0,.9197,.3396,.1972,0,-.1816,-.0775,.9803,0,-.0001,1.2863,0,1),
        lowerarm_l: _M(.9937,-.0384,-.105,0,.0585,.9789,.1957,0,.0953,-.2006,.975,0,.0001,2.2665,0,1),
        hand_l: _M(.6098,.2075,-.7649,0,-.2595,.9642,.0547,0,.7488,.1652,.6418,0,0,2.0521,0,1),
        upperarm_r: _M(.3575,.929,-.0953,0,-.8932,.3104,-.3254,0,-.2728,.2014,.9408,0,.0001,1.2863,0,1),
        lowerarm_r: _M(.9777,.141,.1558,0,-.2089,.7337,.6466,0,-.0231,-.6647,.7468,0,-.0001,2.2666,0,1),
        hand_r: _M(.4462,-.0972,.8896,0,.3368,.9392,-.0664,0,-.8291,.3293,.4518,0,0,2.0521,0,1),
        thigh_l: _M(.904,.1781,-.3886,0,.0417,-.9415,-.3345,0,-.4254,.2862,-.8585,0,.9176,-.0551,-.1072,1),
        calf_l: _M(.8972,.0707,-.4359,0,.1451,.8851,.4422,0,.4171,-.46,.7839,0,0,3.614,0,1),
        foot_l: _M(.9937,-.0901,.0661,0,.0848,.2223,-.9713,0,.0728,.9708,.2286,0,0,3.6967,0,1),
        thigh_r: _M(.9106,-.1027,.4004,0,.0003,-.9685,-.249,0,.4134,.2269,-.8819,0,-.9176,-.0551,-.1072,1),
        calf_r: _M(.8968,-.0674,.4373,0,-.1376,.8969,.4203,0,-.4205,-.4371,.795,0,0,3.614,0,1),
        foot_r: _M(.9885,.1229,-.0876,0,-.1239,.3298,-.9359,0,-.0862,.936,.3413,0,0,3.6967,0,1)
    });

Looks a lot like my QI extension, oddly .  The real reason I bring this up is:  I saw the that there is a javascript interface for Kinect-2.  I currently have a very similar skeleton, coming out of MakeHuman, which I think I could make exact.  I would be shocked if Kinect generated 25 separate animations for each capture.  I have absolutely ZERO interest in writing out a .babylon file with a skeleton and bones with 25 animations inside, one for each bone, for each capture.  Not to mention, it is FRAME based, which is hostile to WebVR frame rates.  Consequently, there would be no trying to jam it in PR.

To be consistent with my other comment about file size, perhaps QI is a better fit for Kinect.

[BitOfGold]

Nearly all my problems are and were shadow related... 
So I'm hoping some of theese will be easier to use, or better working, or working at all in 3.2
The best would be "fire and forget", just set casting shadows or not, set shadow quality, and it is working.

Like:
- cascading shadows
- no shadows on instances
- hard to set bias (and other settings like shadowMax, ) correctly when light and camera is moving
- exponential shadowsmaps and blurred shadowmaps self shadowing are just not working for me. I searched for many days but could not find the right settings... self shadowing just does not work for me on a big scale scene. poisson shadows always work.
- same with point light shadows: with some special settings it is working, but not generally looks ok.

Maybe I should make some realy good playgrounds to show how hard to make multiple direct and point shadows working in a simple FPS environment, with walls, floors and objects.
All is working well if you light a small object with some light, and fine-tune bias and other parameters - but not at all when the scene size, lightmap size and light distance is different or changing.


 

[BitOfGold]

About the problems above:
https://playground.babylonjs.com/#UZQ8TN#44

(Sorry if this is not the place of bragging about my shadow problems...   )

So, just look at that, how hard it is to set up shadows, in a pretty simple scene/map.
Many shadow settings (with lots of experimentation to get them right), and some of them are not working, some has many ugly artifacts.

 

[Wingnut]

Hi guys.  DK, thx for the "let's hear wishes/dreams" -thread.  I love it.

var butt1 = new BABYLON.GUI.Button.CreateSimpleButton("butt1", "");
butt1._children[0].text = "test";

Does anyone (besides me) think that a BABYLON.GUI.Button could/should have a .setText("some text");   ??

*shrug*.   Perhaps late-setting of button text is discouraged, because the button doesn't have an .updateFitAndWrapAfterTextChange()  ?

Anyway, butt1.fontSize = "whatever" works. 

I wonder why it doesn't need butt1._children[0].fontSize = "whatever".

Perhaps butt1.text = "whatever" is SUPPOSED-TO work? *shrug*

Maybe this belongs in a different thread. 

[JCPalmer]

@Wingnut, like the "start wind" to "stop wind" here?  If so, I just set re-assigned the text, here.

The only thing I did was, do this in the click.

edit: you need to do it to children[0].text.  It is not doc'ed as I remember.  I just read the source code.

[Tomm Huth]

I would love to see more modularization of the code, right now you end up with everything regardless of what you actually use. Not everyone needs VR, physics or audio etc. Their own NPM modules? Or someway to do proper tree shaking?

Any reduction of the resulting bundle would be great (we’re on the web after all).

[Wingnut]

Hiya Tomm!  Perhaps https://www.babylonjs.com/versionbuilder/  ?

Your idea was SO GOOD... that somebody went back in time, and coded-up a solution. 

There's a whole thread about it.  Lots of action has been taken toward modules... and it's still happening. 

Bloat is really a sign of high-activity/power-inventing, so it's good... and bad. 

All in all, you're thoughts are good ones.  thx!

A note to @JCPalmer:  Did you see the butt1._children[0].text in the third line of my post before yours?  No skimming allowed.    I just think there should be an easier way... like button.setText() or just button.text = "whatever".  If button.fontStyle works, why not button.text?  Seems inconsistent... but there's things I forget to take into consideration.  Thx for reply just the same.  Other ideas welcome, thx.  If longer discussion needed, perhaps Wingnut Chronicles thread.

More-thoughts:  Perhaps (late-) setting .text... after the render... is a no-no, eh?  I think I heard Deltakosh once say "It's better to dispose old button and make a new one with the new text."  (loose quote, of course).  Those GUI things are sort-of non-updatable, perhaps.  Replacing text on an already-rendered button... might make many of the measurement properties... dirty/stale.  Not sure.  Perhaps wrong thread. 

[Tomm Huth]

@Wingnut

That sounds promising, but how does that fit in with a Webpack workflow?

It seems to me Babylonjs is still very much aimed at having it exposed as a global variable, but considering the potential complexity of a game, and where JS is headed in general, it makes very much sense for Babylon to jump on the ES6 modules bandwagon -- TypeScript added support for ES6 modules back in version 1.5. That could also make plugins easier: CannonJS is already playing along nicely with modules and Webpack. Using import/export syntax, Webpack can do tree shaking automagically, I believe, too.

I realize of course that this would be a fairly substantial rewrite, but it does not need to be much more than a restructure of code, dropping the namespace, and exporting and importing the right stuff (famous last words).

As BabylonJS is great stuff already, is would be cool to see it take it even one step further. 
 

[JCPalmer]

@Wingnut, I did not.  A setter at a button class level is basically a one liner, so I assumed you were saying "there was no way to" for a future plans topic.

I also thought of this during a long thanksgiving drive.  It is not really related to 3.2, but Turn Playground into a Cordova app.  All the stuff it does like intellisense is great, but loading a pg seems to be turning into almost an IDE install every time.

The Cordova building process is actually in node/NPM, so there is probably some familiarity for someone.  Might be a good project for someone who wants learn about Cordova.  The "application" is already developed & requires no plugins to use, so you would not be building the app at the same time as porting it.  I have just started to do both at the same time, along with building a custom plugin, and this has been a big lift so far.

Putting all the JS files for the last stable version right in the app would be incentive to switch the default build to stable to come up quick.  Would also probably reduce someones bandwidth bill too.  This meta tag would need to be removed to download the latest JS files, if switched to:

<meta http-equiv="Content-Security-Policy" content="default-src 'self' data: gap: https://ssl.gstatic.com 'unsafe-eval'; style-src 'self' 'unsafe-inline'; media-src *; img-src 'self' data: content:;">

The browser url field could be replaced with just the number, & "Go" button.  People could then, just use the # for the title of the <a>, eg. UVQWEI#5, so if you had the app on your desktop, you could just cut & paste the #.

[Sebavan]

@Tomm Huth, you can easily create your own distrib including only the part you need with the workload: https://doc.babylonjs.com/how_to/how_to_start#custom-builds

I agree it is not as convenient as npm or webpack but this can help you reduce the size of your dependency.