continued_at_new_forum The Wingnut Chronicles

[Wingnut]

Yeah, I didn't mean to pressure you, @Nockawa... just inviting you to come watch and give us pointers, if you wish.  (and to check if you might have source code for such a thing.... already on one of your hard drives... for us to borrow.) 

We have all the tools we need to do a panel of buttons... right now, I suspect.  If we do it properly modular, we/Thiendv can easily change to the new version, later. 

No rush for anything from you, Nox.  We have clickable text2D buttons and we have rect2d.  I think that's all we need... to get version 0.001 working.

I'm going to try it soon, if nobody else does.  I think it will be easy... but still, I'm pretty lazy.    I just re-learned today that Group2D can have tracked nodes.  Versatile!  Useful! 

Yeah, I think we should do it both ways.  Now, with primitives... AND later, with contextMenu template from your future GUI lib.  Do it all.  Why not, right? 

[Nockawa]

That's the path that many people (unfortunately) are taking right now: building their own GUI based on C2D and hoping they will be able to migration part of or all once the real stuff is out.

I can't give code snippet nor recommend you guys to start using the GUI related classes that are already in the 2.5 beta, it's too early stage and you won't have any value to try to base your code above it and extend it.

[Wingnut]

Yeah, I understand... it won't evolve well.  Yeah, I was thinking... detach the old, snap-on the new.    I don't think we will be getting too advanced... currently.  Nice and simple.

Here's a quickie:  http://www.babylonjs-playground.com/#BUWYF#21

The buttons work! (targeting URL's, not functions, yet).    I borrowed some code from an earlier project.  This stuff is easy... and fun.  I like working with it.  @thiendv spoke of changing colors with the menu, so, I suppose that will be a challenge.  Could try a color picker, I suppose... but... maybe just a "new random color" button, for now. 

I haven't got the "right click again... to close menu" working yet.  If someone wants to activate THAT for us, that would be okay with me.  I haven't done much hiding/disposing of canvas2D things, so if anyone wants to advance that, I wouldn't complain. 

[Wingnut]

http://www.babylonjs-playground.com/#BUWYF#27

Startin' to look and work great!  Buttons parented to rectangle (for border and menu title), rectangle parented to group (for mesh tracking), group parented to ssc.

Ssc covers entire canvas, but no background fill... so we can't see it.  But second right-click closes the menu fine, so actionManagers are seeing pick triggers THROUGH the full-canvas ssc.  YAY! 

I should put LEFT-click-trigger actionManagers on everything... make sure button clicks aren't triggering those.  I'll test that soon. I have faith that it will work fine - with buttons disallowing their left-clicks to pass thru to the actionManagers on the mesh beneath.

I see "Firefox stopped popup windows from opening" - warnings, from time to time.  Upon opening the menu, buttons are pressed automatically... trying to launch webpages in new windows.  Sometimes I get warnings that Firefox stopped 12 popup windows from launching... just from ONE menu opening.  hmm.  It ONLY happens... when you right-click on the mesh origin... and the menu opens beneath the pointer.  Right-click somewhere OTHER THAN where the menu will open, and no probs.  hmm.

I hear my CPU fan speeding-up, too, likely from playing too many videos.  I suppose I could disable the videoTexture while I work on the menu, eh?    (duh, Wingy)

This Canvas2d system is working well... thanks Nockawa and contributors!  I think @thiendv will like this, too, when he/she returns.

[GameMonetize]

Impressive! this Canvas2D feature is definitely a great addition!

[Wingnut]

Uh oh...  http://www.babylonjs-playground.com/#BUWYF#28

I put left-click triggers on the mesh actionManagers (line 186)... to see if LEFT picks/clicks (on buttons) were going THROUGH the buttons... and striking the mesh beneath.  They are.

Soooo...  hmm.

On 9/4/2016 at 4:03 PM, Nockawa said:

skipNextObservers were designed to be able to have a fullscreen Canvas and still be able to select 3D things below. I just didn't have the time to finish coding it (and somebody added a regression that made it less possible it could have been). So now it's supposed to be fixed and thanks to skipNextObservers you can do either: "Canvas2D took care of this event, 3D objects must not be aware of it" or "Canvas2D wasn't interested about this event, the 3D objects might be..."

Nod.  I have other issues, though.  My button observer (coded by Nabroski - lines 158-160) is a pointerUP observer.  To check it for feed-thru to underlying mesh, I have to put a BABYLON.ActionManager.OnPickUpTrigger in line 186.  Here we go with #32.

1.  Open a menu with a right-click on a mesh.  Console says we got a LEFT click on the mesh. BUT, that was the RIGHT button going UP.  That is because line 186 is a BABYLON.ActionManager.OnPickUpTrigger and really needs to be a OnRightPickUpTrigger.  Our current actionManager OnPickUpTrigger watches for ANY mouse button going UP, and it triggered when the RIGHT mouse button went UP (during the menu launch).

2.  Left click CYOS button.  Again, we see "left click" at the console.  This time it was the left mouse button going UP.  This is the actual "feed-thru" that we wish to skipNextObservers.  Not sure how to do that.

I have much to learn about these observers and observables.  See those 'd' and 's' args in line 158?  What the heck are they?  Update:   These are eventData and eventState. They are args for the callback method of Observer Class.  http://doc.babylonjs.com/classes/2.4/Observer#callback-eventdata-t-eventstate-eventstate-classes-2-4-eventstate-gt-void  WOW!

PS:  Those of you who were observant enough to see a rare picture of Nabroski in the link above... might notice that he looks quite similar to this guy.

[Nabroski]

1 hour ago, Wingnut said:

(coded by Nabroski - lines 158-160)

can't remember posted any code here,  which goes beyond line 60

minifed your playground a bit. less readable, more regex at from line 147 

var button1=[],a=[];for(var jk in str)a[jk]=button1=[bgen(butparent,"b"+jk,bpx,21*jk,bw,bh,myfill,mybord,2,2,str[jk],"8pt Arial",fcolor,align,!0)],clickify(a[jk][0],url1[jk]);

http://www.babylonjs-playground.com/#BUWYF#33

 

but its nice work, thank you, fun to play with!

 

 

 

 

 

[Wingnut]

Oh, yeah, you are right, Nabs.  The base playground is yours... but not the 158-160 area.  Hmm, I wonder who I stole that code from.

All in all, thank YOU for the initial scene, and for the code-tighten.  Interesting.  Yep, as you say ... "less readable", but things that repeat... can be done with for-loops.  Reduces code size.

I'm curious why you hard-coded the 21 value (in line 168)  ... instead of using byo variable (from line 143).  If there is a "coding wisdom" reason, I would like to learn it.  (thx)   I need all the coding wisdom I can get. 

[Wingnut]

Ok, continuing with our project... http://www.babylonjs-playground.com/#BUWYF#38

(Nabs, I adjusted your line 191 a bit, using bpy and byo.  The bpy variable is a fine-tuning adjuster for moving buttons up/down within rectangle area.)

Here, I renamed clickify to linkify, but that doesn't matter, because THIS version is NOT for making link buttons.  This time, we are going to call functions with each button.  Thus, we now use funcify and an array funcs.

One of the primary problems here... (besides the bad eval call inside the funcify method @ line 184)... is that each button servicer func... needs to know WHICH mesh.  This means that we SOMEHOW need to "carry" event.source from the menu launch... all the way to the button-servicing function.  For example, b0Service func (line 163) must have event.source (from the menu launch event)... arriving as an arg/param.  It has to know which mesh. 

One of the buttons/servicers will be used to change color of the focused mesh.  Another might be used to rotate it.  Another might launch a "Mesh Info" panel.  etc etc.

I'm not sure how to do this hand-me-down. (handing-down-of which mesh... to the button-servicer funcs).  Multiple MCM (mesh context menu) could be open at the same time.  We won't know WHICH button was pressed on WHICH menu.  SO, each button must "carry" info about which mesh it is used-for.  Erf.  I'm scared!    So is my dog!  

Maybe... in the bgen button-maker func... store the menu-launch event.source onto the rectangle2D button?  Overload the button object so it carries information about WHICH mesh the button pertains-to?  hmm.  Seems clunky.

Ideas/help welcome.  (thx)

[Wingnut]

Moving right along on our right-click-on-mesh menu, here's another version:  http://www.babylonjs-playground.com/#BUWYF#39 

b0Service is operational!  YAY!  Click the b0Service button (bottom button), and the eval in line 173 runs.  It evals a string found in the str array at line 149.  Button 0 servicer (line 151) receives those mysterious 'd' and 's' objects as args (EventData and EventState).

Both are bigtime interesting!  Look at line 153!  Oh my goodness!  I am able to glean (painfully gather) the trackedNode/trackNode from the menu_grp object, and that's exactly what we need!

Sure, the b0Service func sets random color for multiple mesh, sometimes, but that is because Nabroski's scene uses some shared materials.  It is not important.

The important thing... b0Service RUNS when b0Service button is pressed.  YAY!!!  (click on button's text... not on blank area.  More about that, below.)

Oh, you want to know about the 's' object/arg?  Activate line 157 and click on it in your console... to launch the object inspector.  It is an eventState obj.  It is sparsely populated, but... AHEM... one of its properties is...  (wait for it)...  (drum roll please)...  skipNextObservers (currently set to false). 

Okay, okay, I was doing all the talking about observers, observables, and eventStates. 

SkipNextObservers might be what we need... to keep left-clicks on buttons... from "passing-thru" to left-click triggers on mesh behind those buttons (if there IS a mesh behind the button)  (depends upon camera angle, of course).  Tests soon! 

(Wingy dances around... loving his fully-operational b0Service button.) 

Don't those random colors on the mesh... look more vivid and saturated than scenes from lesser frameworks?  Yep, it is because the Gods of WebGL smile-upon we BJS lovers. 

Yeah, yeah, I see the ugly "souceMesh is undefined" error, sometimes.  Don't notice that.  It was just your imagination.  Press the button once more, and you'll get a color change.  I promise.    YAY! 

It is caused by clicking-on the NON-TEXT part of the button 0.  If you click on the non-text part, the d.relatedTarget._parent._parent._parent._trackedNode is undefined.  Perhaps one less _parent is needed... when we click and miss the button text.  *shrug*  Tests ahead.

[Nabroski]

Hello Wingnut

Sorry for the late response, i got a serious ill, 1000000 Fahrenheit.

But i keep tracking what you're doing - very interesting.

 

[Wingnut]

Hiya Nabs! 

Nah, there's no response requirements here, or need to apologize for anything.  This is just a campfire project with no deadlines or clear objectives. 

Well, I guess there IS one requirement.  Get well, eh?  Sick sucks.  Sorry to hear about that, Nabs.  Hope you are feeling better, now. 

(Wingy hands you a mini corn-cob pipe... packed with special herbs and spices.)   (4 and 20 blueberries... baked in a pie.) 

Sweats, then chills, then sweats, then chills?  Yech.  I hope for a speedy recovery for ya! 

You don't suppose somebody made a voodoo doll and put that internet picture of you... on it... and is holding it over the campfire, do ya?  Got enemies?   

[Wingnut]

Okay, we're off and running with version 42. 

Regarding the must click on button text issue, lines 156-158 work-around was added.  Pseudo:  If user clicked on non-text area of button, sourceMesh variable is derived from one less _parent deep. 

Work-around was successful.  Click anywhere on button 0 - good color changes.  But this is an odd and interesting symptom. 

Even though the observer is added to a rect2d called button (line 174), the button TEXT-child (line 76) is "acting" like the observer was added to IT (and not on its rect2 parent). 

Likely cause... d.relatedTarget (line 155) is a different object... depending upon if button TEXT was clicked, or if button blank-area was clicked.  Interesting! 

Likely, if button-click misses button-text, .relatedTarget is probably the rect2d of the button.  If button-click hits button-text, .relatedTarget is probably the text2d object that is the rect2d's child.  I may do more tests.

--------------------------

Regarding click-thru issue and skipNextObservers... it is working well, I think.  The eventState.skipNextObservers was set true in line 175.  With menu open, you can click b0Service button, and no console report from actionManager.onPickUpTrigger (which was added in line 205).  This is good. 

Drop your pointer below the b0Service button and left click on the mesh itself, and console reports a triggering.  SO, our line 175 has stopped the click-thru issue.  Yay!  This is good, too.

So, all is working fairly well.  I think I will call this campfire project... a success... and complete.  Proof of concept... 99.5% proven viable, plausible, feasible, and practical.    And, fun!  I hope @thiendv comes to visit again and is satisfied with our tests and successes.  Meanwhile, party on, everyone!

[Nabroski]

do you already discovered that 

pointerEventObservable

BABYLON.PrimitivePointerInfo.PointerUp

&
BABYLON.PointerEventTypes.POINTERUP

have differences ?  

Im not sure witch actionManager use as Backend, but i would say BABYLON.PointerEventTypes, SkipNextObservers should work and when not

you can also overload Observers, like anything else

button.pointerEventObservable = scene.onPointerObservable and then .PointerEventTypes

 

It confused me at the beginning, due Canvas2d makes no difference, but Babylon does

scene.onPointerObservable = BABYLON.PointerEventTypes
scene.onPrePointerObservable = PrimitivePointerInfo

 

[Wingnut]

Woah, thx, Nabs!  I don't know much about those things, but I know what to search-for, now.    You seem like such a natural with observer stuff.  I hope I get there someday.  You have a Java background, right N?  *nod*  You probably worked-with observer things there.

Maybe we can continue work on the 3 items that are still pending issues with our MCM.  Here is my report from the field...

1.  Remember when Firefox tried to launch windows when the LINKS version of our menu... was opened at a place where the menu launched under the pointer (in other words, when you right click near mesh origin/pivot-point)?  That is still happening with the FUNCS version.  Carefully position your pointer so that... when the menu opens... the pointer is directly over the b0Service button.  You will get a color change upon menu opening.  Reworded once more, if you open our menu in a way that a button is under the pointer... that button will trigger... without left-clicking it.  It triggers upon menu launch.  This is not correct and needs troubleshooting.

2.  Cursor/pointer shape.  Open menu, mouseWheel-out/pan-around until b0Service button has no mesh behind it.  The mouse pointer returns to arrow shape.  We need lots of cursor/pointer shape-control (is that called tool-tips?), eventually.  @Dad72 and I had some recent discussions about this.  I wanted the moon and the stars (a different pointer shape ANYWHERE... different per actionManager, different per Canvas2D button/menu, you know... I wanted complete control of the pointer shape on every clickable in the scene.) 

     I think Dad72 thought I was nuts.    We have our hands-tied somewhat... by renderCanvas.style.cursor setting, I suppose.  But I also know that Sprite2D and similar... can do pointer tracking.  SO... make-your-own-mouse-pointers... with sprite2D and other prims... seems very plausible to me.  Maybe that should be our NEXT experiment with our new menu.  See if we can replace the pointer... with a sprite2D... and make it change shape when it is atop a menu button.  Coooool!  Can we make the pointer on-fire?  (animated)  Bet so.  Sprite2D gives us lots of "replace the pointer" power, I suspect!  Let's do it!    13 cents and my left sock... to the first person who uses Sprite2D as a mouse-pointer replacement.    PARTY!

3.  Flash in the lower left corner.  Sometimes... often... when I launch the MCM (mesh context menu - right click)... I see a flash of graphics in the lower left corner of the screen.  I think this is caused by a programming thing.  The menu's main rect2D (line 117) is turned-ON, and THEN it is positioned at the trackNode/trackedNode.  Our menu (its rect2D) first appears in the lower left of the screen, and then it immediately moves to mesh origin.  If positioning was done FIRST, and then display turned-on, the lower left flash would end.  No biggie... Nockawa can fix that issue with a snap of his fingers, I suspect.

Ok, that's all I have for my campfire report.  Feel free to keep burning, gang.  I'm going to go study some of the things Nabs spoke-of.  Observers/observables are a little weird, but they seem quite powerful and versatile.   "Stephanie... Number 5 need more input!"    Party on.

@Nabroski - when you said you added some "regexp" earlier (lines 183-189)... I didn't get the joke... until now.  hehe.  You were referring back to another post.  FUNNY!   Chicken scratch!   Only scientists understand that fancy coding.  heh.  Lines 183-189 of our menu... have that same look, almost.  It's the stuff of brain tumors.

[Nabroski]

Hello Wingut

No, i'm best in C Code, also know some Java.

so their are both type of HTML DOM Events on the primitivepointerinfo is reduced to a mouseevent and PointerEventTypes is more global

look at your console and make a conclusion, also try to google some of the keys:values
http://babylonjs-playground.com/#1GLQRL#0
 

 

[Nockawa]

9 hours ago, Nabroski said:

do you already discovered that 

pointerEventObservable

BABYLON.PrimitivePointerInfo.PointerUp

&
BABYLON.PointerEventTypes.POINTERUP

have differences ?  

Im not sure witch actionManager use as Backend, but i would say BABYLON.PointerEventTypes, SkipNextObservers should work and when not

you can also overload Observers, like anything else

button.pointerEventObservable = scene.onPointerObservable and then .PointerEventTypes

 

It confused me at the beginning, due Canvas2d makes no difference, but Babylon does

scene.onPointerObservable = BABYLON.PointerEventTypes
scene.onPrePointerObservable = PrimitivePointerInfo

 

I strongly encourage you to rely on pointerEventObservable rather than actionManager when it's about Canvas2D. I did what I could to support actionManager, but it's certainly not as reliable and definitely doesn't have the same feature set than pointerEventObservable.

PointerEventObservable enables you bubbling of events, which is a must for GUI related coding. For instance you can set and observer on the ContextMenu's Rectangle2D, and still get event fired in the primitives inside this Rect2D (your Buttons for instance). Make sure you always check for the "relatedTarget" property in PrimitivePointerInfo, it will tells you which prim fire the event (the rect of a Button for instance). 

This features prevents you from created dozen of Observers.

Make sure you read this: http://doc.babylonjs.com/overviews/Canvas2D_Interaction

[Wingnut]

Well, in this case, both are important.  ActionManager is needed on the mesh itself, used to open/close the Canvas2D stuff.  Thus, it seems important that Canvas2D events NOT collide/interfere with actionManagers (the clicking-thru issues).   (And the opposite, too.  ActionManagers must not interfere with Canvas2D events.)

These two systems need to cooperate with each other... for many projects.  In fact, we might someday have a Prim3D... where a mesh is used as a GUI widget for Canvas2D.  Or how about PrimHTML, where a GUI web-form widget is used as a Prim2D?  Not sure if that is possible, but some crazy inventor will ask about it someday, I bet.  Who knows what demented things people will try... when they are allowed to create their own primitives.  Mad scientists are numerous here in BJS land. 

But yeah, I think I understand what you are saying.  Keep the two systems clearly separated, whenever possible. 

But there is going to be overlaps and crossovers between the two systems (such as the click-thru issue).  It is common for mesh with ActionManagers... to be located atop and/or behind Canvas2D things (not necessarily GUI things).

We can't always choose one or the other, I suspect.  What we might need... is to test/establish protocols/best-practices... for the many times that the two systems will be used simultaneously.  Possibly, we'll be "casting" or converting one type of event... to the other.  We're just beginning to experiment with all the ways Canvas2D can interact with mesh. 

Sprite2D mouse pointers is one such case.  Using Rect2D's bordering features... is another (to select a mesh).  Dashboards of 3D vehicles... might be seriously adorned-with prim2d's.  Menus on the sides of boxes, too.  Could have a GUI menu as a wall texture, transforms just like a texture (instead of lookAt/billboard like our MCM does).  Woah. 

The act of getting real pretty... could get real ugly.  heh.  Agree?  Thoughts?

[Nockawa]

Using the ActionManager for 3D is obviously needed, I'm just saying: pointerEventObservable is working better when Canvas2D is concerned.

Now I've heard multiple times about the click-thru issue and to be honest, I was kind of waiting for someone to yell at me to do it. it's a completely legit feature and I think I'm going to do it today because it should be available to people.

The Prim3D you're talking about is already identified and some dev mere made, but nothing released so far, the scope of this feature and the performances makes it harder that one could have thought.

PrimHTML is GUI Lib as far as I understand what you're saying.

The renderPhase setting I released yesterday (see the main post) enable you to draw 3D content over 2D. Another way to display 3D over GUI to make it "part of the GUI", I still hope a better solution will come up in the near future, but not right now.

Quote

Using Rect2D's bordering features... is another (to select a mesh).  

can you elaborate this one?

[Nockawa]

by the way, in your pointerEventObservable's observer you can set "skipNextObservers" to true in the Event State to disable the interaction event to propagate to 3D. It's supposed to work now.

[Wingnut]

*nod*.  Cool.

Rect2D to select a mesh.... is simply putting a border around the mesh, or possibly bracketing the mesh... showing mesh focus/selection.  Also, possibly using a partially-transparent/colored rectangle atop the mesh... to show it is selected.

I did a skip... in line 175.  That might not be the right place to do it... not sure.  I'm pretty inexperienced with this stuff.  Nice of you to join us, Nockawa... thanks for your input on these things.

[Nockawa]

52 minutes ago, Wingnut said:

*nod*.  Cool.

Rect2D to select a mesh.... is simply putting a border around the mesh, or possibly bracketing the mesh... showing mesh focus/selection.  Also, possibly using a partially-transparent/colored rectangle atop the mesh... to show it is selected.

I did a skip... in line 175.  That might not be the right place to do it... not sure.  I'm pretty inexperienced with this stuff.  Nice of you to join us, Nockawa... thanks for your input on these things.

Are you using the Node Tracking feature to do that? It should work, the only thing that might get in your way is the size of the Canvas doesn't change (it won't shrink if your mesh if going further the point of view or enlarge if it's coming closer), but that's also something that can be evaluated and added to C2D.

[Wingnut]

Actually, I have never used a rectangle2D (its border) to select or mark a mesh.  I just imagined it.    I don't know if it is possible.

Lots of other people use showBoundingBox, or edgesRenderer, outLineRenderer, highLightLayer, or NasimiAsl's thing... to "mark" a mesh.  To highlight it. 

Showing when a mesh is selected or focused... has always been a problem for webGL.  It often takes post-processing (downstream stuff).  Thus, these "focus indicators" sometimes don't display... on renderTargetTextures and screenGrabs (upstream stuff).  But a rect2D's border WOULD display on rtt's and screenGrabs, yes?

It's just another demented idea.  It might be used to track/bracket an enemy aircraft... on a jet fighter HUD.  TrackedNode would work fine, but that auto-sizing thing would be a bit more work... I suspect.   

Thx for asking about it.

Oooh, how about a rect2d "frame" that is X-Ray glasses?  Like a magnifying glass, place it between camera and mesh, and you can see the mesh naked (wireframe).  Cooooooool.    Goofy.  hmm.  I wonder if wireFrame mode shows in rtt and screenGrabs.  I think it does.  It's part of material, I think.  I have SO much to learn.

[Nockawa]

I've just committed the dev to filter events in the 3D Scene when they're occurring above the Canvas, the PR: https://github.com/BabylonJS/Babylon.js/pull/1448

This behavior is now BY DEFAULT, which is a change compared to what you guys had to far, but I thought it would make more sens this way, so the setting is there in case you want to get the same behavior as before.

[Wingnut]

Ok, so, when this goes active in  PG #42, a right-click on the menu... will not close it.  

Oh, way to go, Nockawa... breaking my playground.  heh.  (j/k)  (We can still right-click on mesh... to close the menu.)

Focus... gets involved in all this stuff.  For example, would we ever allow control-mouseWheel to up-scale the menu?  (like control-mouseWheel upscales a webpage in a browser).

hmm.  MouseWheel means ONE thing when a mesh is focused.  It could mean a DIFFERENT thing... when my menu is focused. 

Ow, my brain hurts. 

Later, we might need button focus, too.  When blind people use computers, they TAB thru the choices on a menu (button focus)... and a voice synthesizer announces the text on the button.   We'll need to do that... for our context menus, someday.  Erf.  (Wingy's dog starts whining for some reason.)  That will probably be part of Nock's new GUI Lib.

But we still might need a scene tabbing (focus-advancing) system.  First it tabs thru all the playground HTML form-widgets, then all the mesh in the scene, and then all the prims on the ssc canvas.  Wow.  Okay, okay, maybe not all that.    (Wingy starts power-drinking.)