Need help with design of a Graphical User Interface

[Junior]

Hell Everyone,

 

I have created a scene in Blender with several animations and cameras at different positions. I would like help from the community to create a graphical user interface that allows the user to do the following:-

1. activate a camera by clicking on a button in the control panel.
   
2. allow the user to switch the menu of the control panel. e.g. from camera views to paint and body kit
   

 

I would also like to add labels/notations to each mesh so that the user can see a description of what the mesh represents whenever they click on the label. The labels should only be visible when the user click on a button in the control panel.  

 

Please see a mock up picture below of what I intend to do. Thank you for your kind assistance.

 

[Wingnut]

Hi Junior!  Still battling with the car, eh?  Nod.  Keep in mind that very little "design" can happen until you understand the "tools" that are available for designing.

 

Ok, you have this part...  http://www.html5gamedevs.com/topic/7641-i-need-help-to-design-a-camera-control-panel/ done, right?  You know how to do HTML slide-open menus and understand index.html, index.js, and index.css seen in https://github.com/BabylonJS/Samples, correct?  That's step one, and it is not a webGL thing.  It is fancy HTML, and thus need not be discussed on a webGL forum.  Therefore, THAT part of your project... can be labeled "the html side".

 

Now for the mesh labels, which could be webGL, but could also be absolute-positioned HTML.  So, even for the labels, there is a webGL side, and a HTML side.  Primarily, there are two types of labels that you can do with webGL (canvas) in its current state:

 

#1. DynamicTextures (2D text on planes) - We don't have great documentation on this, but you can find what we DO have... here: http://blogs.msdn.com/b/eternalcoding/archive/2013/08/12/creating-a-3d-chart-for-your-windows-8-1-app-using-babylon-js.aspx

 

Babylon.js DynamicTexture is an interface to 2D Canvas Land.  Let me show what I mean.

var label1Texture = new BABYLON.DynamicTexture("dynamic texture", 512, scene, true);

This creates a dynamicTexture object.  If you look at the doc, there are two very important methods on it: drawText and getContext.  DrawText was placed on this object... to make it as easy as possible to put some text on a DynamicTexture.  This method "wraps" (makes easier) MANY much more complicated things involved with 2D Text.  DrawText is a powerful method, but it is also easy, and a bit limited, because it avoids your need to deal with the OTHER important function... getContext.

 

After you made your label1Texture (above code), you could do:

var canvasRenderingContext2D = label1Texture.getContext();

Ahh, the dreaded canvasRenderingContext2D class.  What an interface!  The W3C spec for it... is HERE, the Mozilla version is HERE, and the Microsoft version is HERE.  These classes are the real substance behind the Babylon.js dynamicTexture drawText method.  If you plan on putting those round-cornered borders around your label text, you will need to become proficient with canvasRenderingContext2D.  And, as you have noticed, canvasRenderingContext2D is not part of the Babylon.js framework.  The framework simply provides you with the getContext method... so you can get a "handle" or "reference" to the canvasRenderingContext2D that is about to be used for your new label.

 

So, if you ask for help on canvasRenderingContext2D objects HERE... you may be disappointed, because that object is part of HTML and not part of webGL.  Thus, it is off-topic to a degree.  Nobody is very topic-strict around here, but you might find more help with canvasRenderingContext2D on HTML canvas forums... than you might find here.

 

It is my opinion that small text (like the text on your labels) will not have great clarity and crispness... if you use the dynamicTexture method of making your labels.  Don't let me discourage you from experimenting with it.  Learning to use the canvasRenderingContext2D is a large task, though.  But learning it... is part of the "tools discovery" mentioned earlier, and must be taken into account during the GUI design process.

 

There is another way, and likely much easier.

 

#2.  Use static textures.  In other words, use a 2D paint program and make little pictures of the text you want to put on each label.  You may have used this method in making your mock-up picture.  Once you have some little pictures of the text that goes onto labels, you have bypassed the hassles of canvasRenderingContext2D.  Now all you need to do is use little TEXTured planes that become visible under the right condition, and THIS method is much more webGL and BJS Framework on-subject. 

 

In brief, you would work with standard Babylon visibility settings, and BILLBOARD_MODE or mesh.lookAt() techniques, and would be well on your way to getting the "webGL side" be operational.  Babylon LINES techniques could be used to connect the mesh to the TEXTured label planes, and maybe even the model or camera could be rotated and the lines and planes would adjust their placements properly.  When it comes time to do that, if you run into problems, you can ask about THAT specific webGL framework challenge, and thus avoid asking big sweeping questions like "can someone help me with GUI design?".

 

#3. There is one more method to do your labels.  Pure HTML divs or spans, absolutely positioned with CSS.  You may lose your lines between the mesh and the labels, but keep in mind that you could make the mesh color-flash when a certain label for that mesh... is being displayed/viewed.  This makes it very difficult to have multiple labels displayed at the same time, but maybe you don't need to do that.  And again, this is an HTML situation, and not a webGL/BJS framework issue.  The only webGL issue involved in this method... is flashing the color on the mesh.

 

So, now you have been given help on GUI design.  You have been shown WHAT is a HTML situation and what is a webGL situation.  You have been shown how/where the two (or more) "sides" are divided, and now you can ask Babylon.js WebGL questions here on this forum, and ask HTML questions on HTML forums.  JS questions are entertained here too, but mostly when they involve the BJS framework, and not "general JS" or JS used for HTML operations.

 

Because, to be perfectly frank, Junior, by the wording of your posts, it seems like you want someone to write your GUI interface FOR you.  The only way it will get designed per YOUR liking, is if YOU design it.  And the first task of YOU designing (and building) it, is to get to know your tools and in which forums to ask questions about them (HTML, JS, or BJS Framework).  Keep your "categories" separated, and experiment with them while keeping those separations in mind.  The slide-open panels/menus... are HTML and JS... not webGL.  Those button presses produce events that can be wired to JS event-listening functions (non-webGL), and, in turn, those JS functions can talk to BJS framework functions. 

 

Three layers in that one example.  One HTML (the menu/buttons), one generic JS (the button-event listener functions) and lastly, webGL (BJS) functions called from THOSE functions.

 

I hope I have been helpful.  You'll get better answers to your forum questions when you stay on subjects specific to this forum (specific to the JS / classes used by Babylon.js framework).  Take care, study hard, good luck, party on.

[Junior]

Hello Wingnut,

 

Thanks for your explanations and words of advice. I know that what I am trying to achieve is a huge task, especially for a beginner. (There is a lot of things to learn and problems to solve.) Sometimes a bit of 'naivety' can take you to places that you never dreamed of  .  I would like to finish this project, therefore I am willing to take the challenge, and in the end 'collect my prize'. I appreciate your depth of thought and insightful suggestions as well as your broad scope of knowledge on the subjects involved. I will keep you updated on my progress over time.

[Wingnut]

Hi again.  Junior, I forgot to mention something important.  Have you ever heard of i18n (internationalization)?  i18n essentially means... translation to other languages.

 

IF you use canvasRenderingContext2D along with Babylon's dynamicTexture class to make your labels, then converting that label text to another language... is very easy for both humans and computers.  Likely, 1-30 minutes to convert all the label text. 

 

And, it is easy for a computer to read your labels... to blind people, using voice synthesis.

 

Conversely...

 

IF you use little static pictures of the text, made in a 2D paint program, then conversion to another language... could take hours or days.  Lots of 2D graphics work.  And, computers with voice synthesizers, generally, cannot read text from a picture... to blind people.

 

You probably know all this already, but I just wanted to remind you of this potentially-large ramification of chosen methods.   Ok bye again.

[joshcamas]

I would suggest having another html5 canvas layer, which contains the lines and such. 

If you do this then you can turn a 3D point in space into it's 2D representation on the canvas, and use that for stuff.

[Wingnut]

Josh, then you are still dealing with canvasRenderingContext2D, right?  You are putting text on a canvas that is not really made for text and thus you lose text formatting power such as CSS and wordwrap.  If your labels can somehow remain html, then you get more text wrangling power.

 

I don't know what Junior's design requirements are, but I wouldn't use canvas-based labels and lines at all.  I would open up a div on the sidebar menu telling about a feature... while flashing the color of the mesh of that feature.  Then the user can spin the camera around the car, zoom in and out on a feature, and the programmer doesn't need to constantly track the feature and re-arrange floating labels and re-draw lines. 

 

No need for pointer lines.  An information panel just opened with pretty text on it, and a mesh on the car just started flashing (and possibly the camera automatically flew to the feature being talked-about)... all with the click of a menu button or the click of a mesh.  The user would easily know what was being talked-about, without dealing with pointing lines, and without dealing with canvasRenderingContext2D.  *shrug*

 

I guess it depends upon how much a person enjoys pain. 

[joshcamas]

True, But remember that the power of the canvas is... that you can generate ANYTHING.

Generating html is a HUGE pain, which I know too well- my editor's gui is generated with JavaScript, using data from jsons. But you are generating far less stuff, so meh.

Either way works. It really depends on what you want.

[Stephen Andrews]

Generating html is a HUGE pain, which I know too well.

This. x200. I've gotten so sick of it I now try to avoid it entirely, using a custom JavaScript library to generate the HMTL for me in a Swing/AWT style widget system.

[joshcamas]

using a custom JavaScript library

 

Josh is interested

[Stephen Andrews]

Josh is interested

<script src="lib/jquery.min.js"></script><script>_mvcjs = {	location: './lib/'}</script><script src="lib/mvc.min.js"></script><script src="lib/jscolor/jscolor.js"></script><script src="lib/jque.js"></script><script src="lib/jqdrag.js"></script><script src="lib/jqsimpleslider.js"></script>

^ Base dependencies. It's very poorly coded and kinda hacked together, but it lets you do this: v

var Button1 = new MVCJS.Controls.Button("Text", "purple borderleft");Button1.onclick = function() {   console.log("I'm Button1, I eat everyone :D");}MainFrame.addChild(Button1);MainFrame.open();

But yeah, it's incredibly glitchy, and there's still a lot of controls not implemented, like dropdown lists and comboboxes. (The ones that are implemented though look awesome.)

[joshcamas]

ohh interesting.

 

I've been building a interface generator as well, (Which I plan on recoding) 

 

Here's the HTML: http://steinhauer.x10.mx/editor/0.015/Editor/

 

And here's an example interface object: http://steinhauer.x10.mx/editor/0.015/Editor/interface/leftPanels/Terrain/sculpting.js

 

Sadly the interface is extremely limited, hence the recode. :'(

[Stephen Andrews]

ohh interesting.

 

I've been building a interface generator as well, (Which I plan on recoding) 

 

Here's the HTML: http://steinhauer.x10.mx/editor/0.015/Editor/

 

And here's an example interface object: http://steinhauer.x10.mx/editor/0.015/Editor/interface/leftPanels/Terrain/sculpting.js

 

Sadly the interface is extremely limited, hence the recode. :'(

That's pretty interesting, being able to generate the UI from JSON. However, it doesn't seem to lend itself well to dynamic interfaces which frequently change. (Feel free to prove me wrong) :3

[joshcamas]

Well I've made the switch from json to javascript, with allows for scripting...

 

hence the recode. I need to redesign how each interface section (I call them panels) interact and become more dynamic.

 

I will point out that the code is already somewhat dynamic, as in it changes when you click different buttons. And they can be "updated" (currently just regenerates everything, will eventually be better)

 

With this system the code can be muchh more dynamic.

 

 

An example of something that is impossible atm but will hopefully be added is:

 

There is a list that shows all the meshes you have loaded for your project. These meshes are listed by categories, and even categories within those. These groups can be dropped down.

Each mesh has a little image next to it. When you click the mesh text it will display it. There will also be a 'X' that will "unload" the mesh.

 

As you can see, that is a very complicated interface block. Lots of little chunks. :S :S I'm scared to start coding it.

[DonnyV]

Hey I'll add my 2 cents since a big part of my day job is UI design.

 

Wingnut is right. Don't do it in canvas. You will just end up replicating what the browser is 

already good at. Rendering meta tags into a visual interface.

 

I suggest before typing 1 line of code is to design the interface by drawing it. Its a lot easier to change your interface

on paper or in a design app then to re-code something. Plus its much more freeing to trying new things and iterating your design.

You're not going to get it right the first time or the 5th...maybe the tenth. ;-)

 

I like to use Balsamiq to do my UI designs. Its very easy to use to focus on spacing and layout.

Then you can worry about colors, text fonts and what not. You can use it free on there site or buy a seat which is reasonably priced.

https://balsamiq.com/

 

Theres plenty of frameworks to bind data to your UI that make it super easy too. I like RactiveJS since its template based and has tons of examples. Also it does a great job animating SVG, which could be used for other HUD like features.

http://www.ractivejs.org/

 

Hope this helps.