What are my options for UI in Babylon.js?

[Wingnut]

Hi everyone.  About Nockawa's Canvas2d system... I really MUST point out my ridiculous Canvas2D (world space version)... 4-menu thing.  http://www.babylonjs-playground.com/#1KRMMG#12

It's fonts and edges are a bit fuzzy these days, not sure why.  Might be a cachingStrategy issue. (a few tweaks needed)

This may have all been said already, but worldSpaceCanvas2D (wsc) have a node (a mesh) associated with each, and it can be set to anything.  I left mine default, and when I had finished creating all 4 wsc's, I parented all 4 of their nodes... to the camera. 

Reports from the field... say that my menu system is a bit difficult to align/resize... and that's understandable, considering it was designed and built in the playground (never tested by me, elsewhere).  I hope to see percentages allowed in positioning and sizing... someday.  It might be available now, but I haven't heard of it, yet.

Way down in lines 391-401, I'm calling the menu panel-making funcs, and then parenting each node to the camera.

Alternatively, ScreenSpaceCanvas2d (ssc)... does not have a node associated with each, and is more like the "layers" Krum spoke-of.  I like to think of it as a piece of cellophane, stuck to the back of my monitor glass.    It still allows sizing and origin positioning, but it once had a unique issue.  It had troubles allowing mesh-clicking to go THROUGH it.  I think that has been resolved.  But that is the reason I used 4 panels (wsc), in that scene.  I wanted good scene mesh picking/clicking in the center, and GUI on all 4 sides.

Anyway, it is an early-in-Canvas2d-history demo... might show you some things, and it was totally fun to build that scene.  I really enjoy using the Canvas2D system.  I also like Dialog Extension, bgui, castorGui, and absolute-positioned html, too.  I like it all.  If you look at #14 in that playground series, I just started converting it from 4 wsc's, to 4 ssc's (left menu only, so far).  After that, I will try ONE ssc, and check the progress of the "click thru the ssc to the mesh beneath" -issue (of long ago?).  Rumor has it, that issue has been solved, and I would love to do some tests.  I have some projects that need basic GUI controls (now done with absolute-positioned HTML), and one that needs a serious HUD.

@krum110487/others, just in case you need better ways to search than Google, may I suggest our forum search, docs search, and playground search?  Bookmark them well... they will become your best friends.    Party on, gang.

[krum110487]

I do have more questions, but I want to play around a bit and I don't want to hijack this thread, thanks for all of the helpful code and places to get started. 

[Nockawa]

7 hours ago, JCPalmer said:

If you wish your text to be formatted into hierarchical Dialogs / Forms yet still be Mesh(es), you can also use the Dialog extension.  I do not have documentation, but do have a test scene crammed with all features.  It is a bit slow with everything in one dialog, but most do not require such an amount.  Here is html format, and here is repository format of test scene.  Dialog has been updated for BJS 2.4.

If you just want like a sign you can animate in 3D space, there is a .blend file which can make signs, then export.  It is also the .blend I use to make the font letters.

I use DIALOG when I want the UI "in" the scene, and DOM when it just sits beside.

I wasn't aware of this alternative, looks great! can you share some screenshots ?

[dbawel]

I've been testing all of the available GUI extensions for BJS for 2 years now, and I can only provide my personal preferences based upon practical evaluation and usage. I find that canvas2D has great  potential, however in my opinion, the code which drives bGUI is still the best to scale to any device and OS. I also use CastorGUI in every application along with bGUI as I am able to create a button in bGUI and open an HTML form with jQuery support for any additional user functions using CastorGUI. Please keep in mind that I'm not developing games, but apps for various usage.

I have to finally disagree with @Deltakosh on this one (which seriously hurts), as canvas2D is not for the inexperienced user - but for basic interfaces at this time if you have the experience to use it. If you're looking for something that is aesthiticaly pleasing and a no brainer, then Bgui will do the job - however it's a shame that @Temechon gave up on a fantastic foundation for cutomizing beautiful GUIs and @Dad72 has the best HTML GUI extension to date.

OK - now bring on the hurt This is simply my own opinion.

DB

[Nockawa]

2 minutes ago, dbawel said:

I've been testing all of the available GUI extensions for BJS for 2 years now, and I can only provide my personal preferences based upon practical evaluation and usage. I find that canvas2D has great  potential, however in my opinion, the code which drives bGUI is still the best to scale to any device and OS. I also use CastorGUI in every application along with bGUI as I am able to create a button in bGUI and open an HTML form with jQuery support for any additional user functions using CastorGUI. Please keep in mind that I'm not developing games, but apps for various usage.

I have to finally disagree with @Deltakosh on this one (which seriously hurts), as canvas2D is not for the inexperienced user - but for basic interfaces at this time if you have the experience to use it. If you're looking for something that is aesthiticaly pleasing and a no brainer, then Bgui will do the job - however it's a shame that @Temechon gave up on a fantastic foundation for cutomizing beautiful GUIs and @Dad72 has the best HTML GUI extension to date.

OK - now bring on the hurt This is simply my own opinion.

DB

Hello,

As of today, you're mostly right. I did Canvas2d, and it's not a GUI library. It's the 2D Engine that will be used for the GUI Library I'm writing right now. So if you need a simple GUI Lib and you plan to write it on the top of Canvas2D: it won't be easy!

But many people are doing so right simply because they don't have other choices, bGUI is too limited for them and they can't consider HTML.

The GUI lib I'm writing right now will be targeted for simpler use, you won't have to write your own controls and their logic from the ground up and you will have all the basic features/controls a GUI lib generally offers.

Now, concerning the part where you said "bGUI is still the best to scale to any device and OS", can you tell me what is lacking in Canvas2D to achieve that? The feature is big, things are missing, things are bugged, but I work continuously to improve it.

Right now it doesn't support Device Pixel Ratio and I know it's something what must be address in the very near future. The designSize feature help people to build responsive UI, but the feature is quite new and undocumented right now.

[dbawel]

@Nockawa - I must give you the props for what you are building. But most users on this forum still don't find that it is a simple task to build a GUI beyond what they might create themselves in WebGL. I REALLY want to switch to using Canvas2D, and I must say that for GUI elements outside of WebGL - that @Dad72 has in my opinion built the best HTML GUI extension to date; so I'll continue to advise general users to understand the difference between most of the available GUI extensions compatible with BJS. I hope I gave you the proper respect in my last post, and really appriciate all that you are contributing. I'm simply speaking to the average users on the forum. To date I find your extension the most promising of all - in the long run. But for anyone reading this post, please understand the differences betwen the GUI extensions - and between a GUI in HTML and WebGL.

Cheers,

DB

[Nockawa]

3 minutes ago, dbawel said:

@Nockawa - I must give you the props for what you are building. But most users on this forum still don't find that it is a simple task to build a GUI beyond what they might create themselves in WebGL. I REALLY want to switch to using Canvas2D, and I must say that for GUI elements outside of WebGL - that @Dad72 has in my opinion built the best HTML GUI extension to date; so I'll continue to advise general users to understand the difference between most of the available GUI extensions compatible with BJS. I hope I gave you the proper respect in my last post, and really appriciate all that you are contributing. I'm simply speaking to the average users on the forum. To date I find your extension the most promising of all - in the long run. But for anyone reading this post, please understand the differences betwen the GUI extensions - and between a GUI in HTML and WebGL.

Cheers,

DB

No worry, as of today, you're totally right!

Building a GUI on the top of Canvas2D is not an easy thing, I hope to provide a first beta of the GUI Lib in one month with the basics feature and core GUI Controls. From this point I hope people will give maximum feedback to make sure it has the right balance between ease of use and flexibility.

[JCPalmer]

8 hours ago, Nockawa said:

I wasn't aware of this alternative, looks great! can you share some screenshots ?

Here is one from a test scene for 3D rigs.  The form must be part of the scene, because the scene must display in the entire window for 3D.  Yet, you can hit the Hide Dialog button "area" to toggle the form on / off.  I really do not promote DIALOG, but I do have a true demo (not a tool) scene in development, where I will use DIALOG as well.

[Nockawa]

4 hours ago, JCPalmer said:

Here is one from a test scene for 3D rigs.  The form must be part of the scene, because the scene must display in the entire window for 3D.  Yet, you can hit the Hide Dialog button "area" to toggle the form on / off.  I really do not promote DIALOG, but I do have a true demo (not a tool) scene in development, where I will use DIALOG as well.

I've just checked it and it's great! When did you started to code this? Man I wish I knew about this before! Looks like we try to solve the same requirements (and I know I have the bad habit to create a very broad case...)

[dbawel]

@JCPalmer - My apologies, as I didn't mean to overlook the Dialogue extension in my post. It is simple to use and very powerful as well. I find it just as useful as bGUI and CastorGUI - on a case by case basis for all three extensions.

Cheers,

DB

[JCPalmer]

@Nockawa, I started 2/2015.  Took me months to finish.  So long, I was way past putting in document.  Hierarchical layout in multiple directions requires much recursion.  Feel free to look at what I did.  Saw your post about your intentions. 

Think the limit to UI inside the canvas is probably keyboard input.  You would have to implement the concept of "focus" to pull that off.  Unlike cameras & lights which have masks to keep things separate, keystrokes do not.  I bailed on attempting that.  This is a game framework, not a 3D word processor. 

 

[Nockawa]

3 minutes ago, JCPalmer said:

@Nockawa, I started 2/2015.  Took me months to finish.  So long, I was way past putting in document.  Hierarchical layout in multiple directions requires much recursion.  Feel free to look at what I did.  Saw your post about your intentions. 

Think the limit to UI inside the canvas is probably keyboard input.  You would have to implement the concept of "focus" to pull that off.  Unlike cameras & lights which have masks to keep things separate, keystrokes do not.  I bailed on attempting that.  This is a game framework, not a 3D word processor. 

 

I've already developed GUI Lib from Scratch in the past with the notion of Focus, it's not that big a deal once you have implements an event system that is cascading/bubbling. 

I'm more concerned about the truth notion of Focus, which is less used nowadays in GUI, mainly because of Touch input. People don't use shortcut anymore to put the focus on a given UI Element too. Well developers do (I hope, still), but users not so much.

I still find it hard in most of the GUI nowadays to catch which control as the focus and if it's windows is activated or not.

For Visual Studio for instance on a Solution Explorer, you can see which item is selected, if it has the focus it's blue, otherwise it's grey is the Tree Control doesn't have the focus. UI Feedback is very subtle and I wonder if I should implement it.

I guess I will because these are basic mechanics and not doing them would cause several issues for user input.

I understand you comment about "not a 3D Word Processor", but every game UI I see have this notion of Focus when the user that to actively interact with the UI.

[TheTrope]

On 18/09/2016 at 8:52 AM, Nockawa said:

I couldn't have time to test, but I know I did changes for that, did you successfully tested it on VR? (just for my information)

If it doesn't work I'll make sure it is as soon as someone needs it. I've got a Vive just for that!  (well, officially)

It works very well with the WorldSpace canvas, i dit not tried with the ScreenSpace cause it seem's legit to be incompatible with VR (I may be wrong)

[Nockawa]

1 minute ago, TheTrope said:

It works very well with the WorldSpace canvas, i dit not tried with the ScreenSpace cause it seem's legit to be incompatible with VR (I may be wrong)

ScreenSpaceCanvas should also work in VR, it should be "flat", but working. If you have the chance to test, please let me know. Thanks

[TheTrope]

20 minutes ago, Nockawa said:

ScreenSpaceCanvas should also work in VR, it should be "flat", but working. If you have the chance to test, please let me know. Thanks

 

Here is the result of a simple ScreenSpace Canvas  (With VRDeviceOrientionFreeCamera) . I don't know if i need to enable something :/

[Nockawa]

6 hours ago, TheTrope said:

 

Here is the result of a simple ScreenSpace Canvas  (With VRDeviceOrientionFreeCamera) . I don't know if i need to enable something :/

Nope, I'm not surpised you've got this result. I'll talk about that with DeltaKosh, see if I can improve things quickly. Thanks for the test!

[Kreeba]

Just updating this thread as it is well indexed on Google for GUI searches.

 

Babylon 3 has introduced the Babylon.GUI module, which has official support from the Babylon Team. https://doc.babylonjs.com/overviews/gui This should be considered the 'defacto' choice as it will be developed alongside Babylon with VR/AR cameras.