VirtualJoysticks - The Lost Rings

[Wingnut]

Hi everyone!

     During a quest to solve this post by @ian, I came across some VJ problems. 

A while ago, I worked-on some virtual joystick aux canvas issues, and made this demo... that uses a VJ to adjust the rotation of a box.  It works ok.  After 20 seconds, it releases the VJ canvas and returns the playground to "normal".  It also "brought-in" the entire VJ code from the framework... just to make it easy to mess-around and experiment.

Naturally, I wanted to give @ian a cleaner playground demo... to do arcRotate cam-controlling tests with a VJ, so I first tried to remove the hijacked code (the vj source code).  That attempt is here...  http://www.babylonjs-playground.com/#OORFZ#13

It still releases after 20 seconds, and seems to work (the box rotation is turned-off in #13, so ONLY the vjcanvas and rings are active).  Believe it or not, 10 minutes ago... it didn't work.  It seems that... when working with VJ's, we REALLY (thoroughly) need to "clean up" between tests.

-----------------------

The actual question:  Out on the BJS main website...  http://www.babylonjs.com/Demos/SpaceDeK/

...select the camera icon on the right side, and choose 'virtual joysticks camera'.  The VJ's go active (drag on right side of screen = tilt and turn, drag on left side of screen = truck and dolly)... but NO RINGS.  OH NO!  Anyone else seeing a lack of VJ rings... when 'virtual joysticks camera' is selected on the camera panel?  (thx)

-----------------------

Ian, if you are reading-along, I will begin playing with your quest for virtualJoystickArcRotate... or some kind of kludge that works.  THIS post was supposed to be about #13 not working... having its rings not appear at all, or having them appear when the joystick is first constructed.  #13 had problems.  In the middle of posting this message, it started working.  (weird)  So now I/we can get back to work on your challenge.

Ian, I want to show you something... and we will talk more in your thread.  Take a look here, if you please... https://github.com/BabylonJS/Babylon.js/tree/master/src/Cameras  There is the folder of source code for our cameras.  The inputs folder is part of our new Custom Camera Inputs system.  (don't bother looking for arcRotateCam demo-code in the docs.  Sadly, it isn't there.)

See the folder named "inputs"?  Let's visit that.  In that folder, notice that there is https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/Inputs/babylon.freecamera.input.virtualjoystick.js (VJ for free camera)...  but no parallel file for the arcRotateCamera.  hmm.

What can be deduced from the lack of an input file for VJ's on Arcs?  I'm not sure, yet.  I would say that, for now, we cannot count-on our new inputs system to handle a VJArcCam.  So, we go hacking... trying to kludge something.

You and I (others welcome) might be able to get started on a VJ-Arc... by hacking on demo #13 and kludging something into existence.  I am NOT encouraged by the lack of VJ-Arc inputs file, and lack of ArcRotate demo code in the custom input docs.  Maybe others can shed some light.  VJ's for Arcs could be a tough grind... not sure, yet.  Tests ahead, starting with demo #13 if it remains operational.   

Side note:  bGUI was recently seen interfering with VLS godrays, and with lens flares.  POSSIBLY... the extra canvas (called vjcanvas) used in virtual joysticks... interferes with them, too.  That needs testing as well.  SO much to do, and SO little gumption.  heh

 

[ian]

ok nice to see som info about it.

 

thanx for now

[Wingnut]

Our pleasure, Ian.  I forgot to mention (and I forgot to read) this:  https://www.davrous.com/2013/02/22/creating-an-universal-virtual-touch-joystick-working-for-all-touch-models-thanks-to-hand-js/

I'm going to read it now.  I am starting to figure out when/why the vjcanvas remains un-erased after playground re-runs.  I hope to be able to stop that, soon.  Also will attempt some setVjCanvasAtopRenderCanvas() too.  I want another attempt at that... deriving vjcanvas position and size... by measuring renderCanvas position and size.  The playground's non-full-screen renderCanvas is a nice place to test such a future feature, eh?  nod.  Plus, this is fun.  The more I learn about "the second canvas" of VJ's, the more I can use it for my own demented experiments. 

I am testing a scene.onDispose way to clear-out the VJ debris, on #15 now.  A VJ has no .dispose().  It's not the kind of thing that would.  But that darned canvas... hmm.  I am still experiencing a situation where I get NO rings at first-run of #15.  I change URL to #12... go there, then change url to #15 again, and the rings are there. 

PLUS... at that time... I see a console.log report COMING FROM #12 CODE!  It isn't in the playground anymore.  I am viewing #15.  How the hell can I see logs from #12?  Residual code?  I gotta fig a way to clear-out that old code from wherever it's being run.  If you see a console report that says "hello, hijacked VJ", that is from #12... and you might be RUNning #15 in the playground at the time!

What the heck?!?!   Truly goofy.  Something is caching-up somewhere.    Puzzling fun!  Open your browser's 'inspector' and scroll down to the bottom of the <body>... and you can watch the vjcanvas be created... and then be removed after 15-20 secs (Wingy's patented "Playground-B-Safe" timer at work) 

[ian]

wau #12 is nice.Qul !!

 

qul case #12

I'll need this. specifically for phone or talbet or any touch screen device
Nice

 

greetings

[Wingnut]

#12 has too much code in it, yet.  It has ALL of virtual joystick source-code included.  It's too fat.   

We can do smaller.  Still testing.  Now testing #19.  It still has problems.  I'm watching Roger Rabbit movie while testing, so maybe I am slow. 

[ian]

OK, qul

I look forward to your work

we have a time

Maybe I'll try to include this in my work (if you will allow me).
 

greetings

[Dal]

@Wingnut  I was testing on my phone yesterday with my terrain system, not in the playground at all. The joysticks didn't render. I just assumed I missed something

[Wingnut]

oh goodie.    Thanks Dal.  yuh yuh yuh.  I bet @Temechon is experimenting with the bGUI layer godrays issue... and messing-with the playground's framework version.  Temechon can do that, ya know?  He's like a BJS ninja!  He knows how to sleaze-around in the cracks and crevices. 

It's all good. 

http://www.babylonjs-playground.com/#OORFZ#19

I watch the firefox html inspector... and can see activity.  Scroll the inspector down to bottom of doc (bottom of <body> element)... and you can watch the vjcanvas element get added to the document (when you press RUN). 

Firefox even flashes the element for me... saying... "Hey Wingy... this node just got added". 

VJ drag... works fine (for 15 ses).  Rings... where are they?  hehe.  Gone!

15 secs later... canvas element disappears from inspector view... correctly per my PG safety timer.

What?  Are the rings being drawn on a canvas in Yemen?  On a remote island?  Under a rock at the bottom of the Red Sea?   

I'll fig it, even if it kills me.

Lines 2-37... look at that fancy web-stolen getPosition(anyElement) func.   It finds the top left corner of renderCanvas.  IF I ever get past this "where's the rings?" issue, I'll start with the vjcanvasPerfectlyAtopRenderCanvas() feature... maybe.    If that is ever successful, no more need for safety timer in the PG... when testing VJ's.  Yay.  Vjcanvas will cover ONLY the renderCanvas and no more.  YAY!

If THAT works, we'll peddle it to @davrous and see if we can get him to include it on VJ's.  I don't want to cram new things onto his VJ's without his approval and wisdom.  He can spot/stop my stupidity, and I have plenty of that. 

Ok, back to work/play.  What movie we running now?  Ohhh... binge-watch of full Brisco County Jr... and then 3 seasons of Jack of All Trades?  It's a Bruce Campbell festival!!!   Yay!

 

[ozRocker]

My rings didn't show up because touch.pointerID is undefined.  Touch is no longer an object, its a value with the ID in it.  So once I changed that the rings showed up

I changed this:

    private _drawVirtualJoystick() {
        if (this.pressed) {
            this._touches.forEach((touch: any) => {
                if (touch.pointerID === this._joystickPointerID) {

to this:

    private _drawVirtualJoystick() {
        if (this.pressed) {
            this._touches.forEach((touch: any) => {
                if (Number(touch) === this._joystickPointerID) {

 

[davrous]

Hi guys,

 We've planned to refactor the Virtual Joystick using a new approach with @Nockawa in the upcoming weeks.

If you could share in a nutshell, using SHORT sentences  what you'd like to improve, it would help

Stay tuned!

Bye,

David 

[ozRocker]

It would be neat to allow .PNG images to be used for the rings.  If I hand this to a designer I guarantee they'd want to redesign the rings using Photoshop/Illustrator

[Wingnut]

@davrous  thanks for interest.  I'll try yet again.

1.  Option to make vjcanvas automatically fit perfectly atop rendercanvas, via deriving vjcanvas size and position from current renderCanvas size and position.

2.  Option to use vj as a digital joystick instead of analog joystick.  Digital joystick would have small inner-ring movements, thus small screen space needed.

3.  An ID attribute on vjcanvas.

4.  Option to allow users to append vjcanvas to any html element they wish.

5.  Option to allow user to size and position vjcanvas in any way they wish.  If possible, expose top, left, width, height properties.

6.  Related to OzRocker's request - separate the graphics from the functionality, so users can easily change rings to other widgets.

7.  Please expose the z-index and/or any other layer-depth type of setting.

I hope this works for ya, Dav.  I'm sorry for talking in a difficult-to-understand way.  *sigh*

[Wingnut]

@ozRocker thanks for info.  How did you find that (if I may ask)?   Did you see a console error that reported touch.pointerID is undefined?

It was @Nockawa that fixed http://www.babylonjs-playground.com/#OORFZ#12 for me, but I just bet he didn't PR that fix into the framework source.  I bet he ONLY fixed #12 which has the entire VJ source in the playground.  I think he fixed the vj source in #12, but he didn't fix the framework-level source.  So, when I removed all the VJ source from #12 (made #13 and beyond), the repair was no longer active.  The BJS source for VJ wasn't fixed.

Refs:
https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.virtualJoystick.js#L240
http://www.html5gamedevs.com/topic/22269-cannonjs-dynamicfloatarray-compile-failure/?do=findComment&comment=127014
 

[ozRocker]

9 minutes ago, Wingnut said:

@ozRocker thanks for info.  How did you find that (if I may ask)?   Did you see a console error that reported touch.pointerID is undefined?

Console didn't spit out any errors because it was an IF statement which would just return FALSE if number == undefined.  I just had to put random log statements throughout the code to see which bodies of code weren't being executed.

[Wingnut]

Oh man, sorry to hear that, Oz.  Nice find!  https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.virtualJoystick.js#L241

Ok, this is all starting to make some sense, now.   Unreal.  The VJ-failed Spacedek demo was telling the whole story... VJ's are currently still broken.  @Nockawa  MUST have only fixed my #12 demo and not fixed BJS source, too.  Yuh yuh yuh.  I think we've finally got it.  phew.  Lots of hours put into this one.  erf.  Thanks again, Ozzie!   

http://www.html5gamedevs.com/topic/22474-how-does-babylonjs-get-pointer-events-working/

You told us about it there, eh?  Duh, Wingnut.  (I'm behind on my reading, sorry).

One thing @Nockawa adjusted in #12 (line 339)... was line 240 too.

Source line 240:  this._touches.forEach(function (touch) {

Demo #12 line 339:  this._touches.forEach(function (key, touch) {

Yep, me thinks source needs to be updated like #12.  But who knows, anymore?  I think I'm bailing until others get done fiddling.  Nockawa and Davrous need to talk with one-another, and get the spacedek demo working.  After that, we might be able to work on the PILE of other issues with the VJ. 

Step #1, get VJ's working again.  If that goes well, THEN we'll get back to fighting with the vjcanvas position/size, and advance Ian's VjArcRotCam challenge.

[davrous]

Thanks @Wingnut! It's definitely clear now

[Wingnut]

Good.  http://www.babylonjs-playground.com/#OORFZ#28

More work.  Still have bad erasing and bad rings, but...

I have a vjcanvas size and position... that is DERIVED from renderCanvas size and position!  YAY!  Took awhile, I'm not very smart.

No more timers in the PG!  We can now work on VJ's in the playground without having the vjcanvas disable the editor and the PG gui.  YAY!

I did SOME work in the updatePosition func (resize).  Not much success yet.

Restore the window (make it medium sized), and then try drag-resizing the window.  The red-border vjcanvas should keep perfectly matched to the renderCanvas.  It doesn't, not yet.  But at least we can work on it now, without the vjcanvas disabling the PG editor and its GUI (including the RUN button). 

Only 20+ playgrounds to accomplish!  Efficient, aren't I? 

Me thinks that the VJ was originally written for fullscreen only, and for nav only.  This is why it uses window.innerHeight/innerWidth, which makes it cover all GUI, including playground buttons and PG editor (and kills mesh picking). 

VJ's became a big hit (very popular).  But it is not mature enough to handle super-stardom and Paperazzi.    Users are trying things with it... that are beyond its intended purpose.  Soon we'll have VJ2 - Lord of the Rings!  Wow!

[ian]

On 5/13/2016 at 2:09 AM, Wingnut said:

oh goodie.    Thanks Dal.  yuh yuh yuh.  I bet @Temechon is experimenting with the bGUI layer godrays issue... and messing-with the playground's framework version.  Temechon can do that, ya know?  He's like a BJS ninja!  He knows how to sleaze-around in the cracks and crevices. 

It's all good. 

http://www.babylonjs-playground.com/#OORFZ#19

I watch the firefox html inspector... and can see activity.  Scroll the inspector down to bottom of doc (bottom of <body> element)... and you can watch the vjcanvas element get added to the document (when you press RUN). 

Firefox even flashes the element for me... saying... "Hey Wingy... this node just got added". 

VJ drag... works fine (for 15 ses).  Rings... where are they?  hehe.  Gone!

15 secs later... canvas element disappears from inspector view... correctly per my PG safety timer.

What?  Are the rings being drawn on a canvas in Yemen?  On a remote island?  Under a rock at the bottom of the Red Sea?   

I'll fig it, even if it kills me.

Lines 2-37... look at that fancy web-stolen getPosition(anyElement) func.   It finds the top left corner of renderCanvas.  IF I ever get past this "where's the rings?" issue, I'll start with the vjcanvasPerfectlyAtopRenderCanvas() feature... maybe.    If that is ever successful, no more need for safety timer in the PG... when testing VJ's.  Yay.  Vjcanvas will cover ONLY the renderCanvas and no more.  YAY!

If THAT works, we'll peddle it to @davrous and see if we can get him to include it on VJ's.  I don't want to cram new things onto his VJ's without his approval and wisdom.  He can spot/stop my stupidity, and I have plenty of that. 

Ok, back to work/play.  What movie we running now?  Ohhh... binge-watch of full Brisco County Jr... and then 3 seasons of Jack of All Trades?  It's a Bruce Campbell festival!!!   Yay!

 

so where is virtual joystick ? playground #19 desn't have it?

[ian]

Wingnut
How it is now virtual joistick for arc camera?
Where can I get it? Do I need include extra babylon.virtualJoystick.js? Or it is inside babylon.js implementation?

Which version of babylon.js have vitural joystick for arc camera?

How to use it? 

The only example which work is #12 (and yes it have a lot of code...). So is there virutal joystick for arc camera implemented in babylon.js librarry, which version and how to enable virtual joistick?
Can you show any example code?

greetings

[Wingnut]

Hi again, Ian.  To be blunt, nobody is keeping me in the loop, regarding this challenge.  Did you change to BJS 2.4?  If so, may the force be with you.   

Nope, I don't have those answers, Ian, sorry.  I jumped ship on the issue... after I assemble pg http://www.babylonjs-playground.com/#OORFZ#28

That barely works.  It's less code than #12, though .  I quit working the issue... because of talk about (Davrous?) converting VJ to the new Canvas 2D system (by Nockawa).   Perhaps Nockawa  got delayed on some click/touch issues, so Davrous got delayed on a VJ refactor (actually, he never promised to do a VJ v2, but I was hoping to convince him to do it). 

Where does that leave us?  Broken VJ.  Still using HTML canvas.

I am going to wait for the 2.4 "What's New?" list.  MAYBE I'll be able to hear a status report on VJ2 there... but my hopes aren't high.  2-3 weeks after 2.4 What's New is published, if nothing has moved, I will begin coding a new VJ using Canvas2D and as much of the new camera inputs system as possible.  That's going to require about a month of study, for me.    I'm sure how the VJ's will play-into the new custom inputs system for cameras.  I think it is still undecided.

Meantime, you have nothing.  Sorry.  Solution dependent on many other issues and projects, and I'm not involved in those. I just wired a VJ to a arcCam to try to keep your project moving.

I guess... stay with BJS 2.3 for now, is my advice.

Maybe I will do some experiments later today... to see if Canvas2d is ready for mouse/touch dragging.  Help if you can.  VJ is the area of concern, not the arc cam.  Once the VJ is working, wiring its deltaValues to ANYTHING, including an arcCam... is easy (if you don't look at camera.inputs.virtualJoysticks.blah.foo.someOtherCompletelyUnrememberableUntypeableName.ohmygod.)  hehe

Three months.  I'll have something for you in three months, Ian.    (Let's hope others comment)

[Wingnut]

heya @ian... I was able to beat something into shape, I think.

http://www.babylonjs-playground.com/#OORFZ#35

What's that look like?  Still lots of code... most of it unnecessary, but good rings and good arc rotate movement, I think.  Sorry for so much code.

Lines 63/64 - hooks VJ to camera

Line 80 - shuts off vj canvas after 15 secs so playground editor can be used again.

Hope this helps.  If not, let me know, I'll try something else.

 

[ian]

it's ok, look qul. (do I need any special .js file. or just babylon.js and this code.

I can seperate this code. Still lots of code (this is not a problem).

[Wingnut]

Should work without extra files, I think.  (paste-in)  Not sure.  I hope so. 

 

[webGLmmk]

On 6/3/2016 at 1:45 PM, Wingnut said:

heya @ian... I was able to beat something into shape, I think.

http://www.babylonjs-playground.com/#OORFZ#35

What's that look like?  Still lots of code... most of it unnecessary, but good rings and good arc rotate movement, I think.  Sorry for so much code.

Lines 63/64 - hooks VJ to camera

Line 80 - shuts off vj canvas after 15 secs so playground editor can be used again.

Hope this helps.  If not, let me know, I'll try something else.

 

I"ve been thinking of something more like Playstation controls. two "touchpads"..one on each side. One to rotate camera, the other to move backward and forward. I have a space scene that the touch camera is just not sufficient for. There doesn't seem to be any way to go DOWN. Swiping up and down moves forward backward, camera pans left/right. Just not right for a zero G environment ... :-) 

I was thinking build a canvas2d group that linked to freeCamera. A set of 4 buttons/arrows on the right to move the camera, and a set on the left to move forward/backward. I have to study up on free camera too, but probably the biggest danger is over complicating it. Camera position responds to Canvas2d listeners, that should be it, I would think. No need for quick response, this is more of a "move around google maps except in 3D" kind of control scheme.