scene.pick() pickedPoint is off/inaccurate somehow

[JLHolmes]

Got an odd issue here... I've apparently done something bone headed. I'm new to the forum but been working with BJS for a while. This is the first time I've run into something that has really stumped me.

When using scene.pick() I am getting inaccurate pick locations. Calling it like so:

let pickInfo = sceneState.scene.pick(sceneState.scene.pointerX, sceneState.scene.pointerY);

Whenever I pick a point near the window origin, it's accurate, but the further along the windows X axis I pick... more off the results are... but they are consistently off. Even if I flip the scene around by reorienting the camera, the picked x position is off... and the farther towards the "right" I click the more it is off.

Really confused here. Could this be something to do with device scaling and window size? Tried explicitly setting engine width and height... still no luck.

Any thoughts on where to begin?

[Wingnut]

Hi @JLHolmes, welcome.  Here's a little testing playground I made... to help us with this issue.

http://www.babylonjs-playground.com/#10RHFF#9

Every click on the canvas, no matter if it hits a mesh or not, reports SCREEN coordinates of the click... to the console.  Are you seeing any problems with that?  In my console, I'm showing value X = 6 when clicking canvas far left, and about X = 878 when clicking canvas far-right.

Even though the onClick is a classic DOM eventHandler, the console report I am using... comes from scene.pointerX and Y (see line 29).

In line 31, I grab a "pickResult" (It is type pickingInfo, so I should have named it so, I guess). 

If hit, I report the coordinates of the mesh's picked-point.

Can you make the problem... be shown in this playground (series)?  Perhaps do some edits, and make more saves, send us the new URLs?  Thanks! 

[JLHolmes]

Thanks for the response... let me work with that a bit. I'm not convinced I'm not causing the problem, so let me work with you go there. Will post results shortly.

[JLHolmes]

Ok... so here's an update... and it may speak more towards a fundamental lack of understanding of what BJS is doing with the canvas.

I realized there's one fundamental difference all the examples, including @Wingnut's, and our application. Our implementation of BJS is not fullscreen, rather it's constrained to a box which floats around and resizes itself according to flex-box rules. There are certain cases, depending on screen size/orientation, when the offset occurs. I can take an ordinary BJS scene that's rendering correctly, change my screen size (like with Chrome device toggle), and then we're in bad territory... even when I refresh rather than allow things to just "move around."

It could possibly be that I need to detect when "things move" and then do an engine resize()? I keep noticing that the width and height attributes that BJS forced onto the canvas are always slightly different than the actual canvas size. Not sure what that signifies... but I did observe that the more those numbers stray for the actual canvas size... the more pronounced the error is.

[JLHolmes]

Ok... forgive me for talking to myself in this forum... but maybe I'm on the road to the source of the problem. I notice that in all the playground examples the width and height attributes exactly match the actual size of the canvas. So... I am doing something to short circuit BJS's ability to get that correct, perhaps?

[GameMonetize]

Do you mind providing a playground so we can help you identify the issue?

[Wingnut]

Hi DK, others.  JLH might need a canvas in a flex-box (for testing), @Deltakosh.  My #9 playground might not be apropos.  Not sure.

Good stuff here, JLH.  http://stackoverflow.com/questions/30229536/how-to-make-a-html5-canvas-fit-dynamic-parent-flex-box-container

Let's say you have GUI surrounding the canvas... that is changing... reflowing... nodes appended and removed.  The flex-box container is changing the canvas container size with every browser re-flow.  SO, yes, perhaps listen for resize events on the flexbox, and possibly do BJS engine resize every time.

After that, you cannot work in "hard coded pixels" in your project at all.  All "where is pointer atop canvas"... must be converted to percentages.  X must be 0-100%, and Y as well.  You cannot do any if (scene.pointerX > 400), right?  (the canvas COULD BE < 400 px wide, who knows?) 

You would need if (convertToPercentage(scene.pointerX , canvasCurrentWidth) > 65.45 (percent)... etc.    With me?

I could be full of crap, too.     I don't know much about flex-box stuff.  Haven't tested.

canvas.parentNode.addEventListener("resize", function(evt) { engine.resize() });    *shrug*

[JLHolmes]

Hmmm... well... @Deltakosh, the playground doesn't seem to give you control over the HTML layout into which the BJS scene would be rendered... unless there's some trick I don't know about.

This situation arises when the canvas is being tucked into a small area of the web page. I'm pretty well convinced that the problem arises when the canvas width and height properties are different than the actual canvas size... which seems to occur in flex box models because the element "flexes" in response to the content, which seems to be after BJS has made some decisions about the canvas dimensions.

I've tried explicitly setting the width and height properties, using engine.setSize(), but it was still off. I'm going to try @Wingnut's lead on establishing width and height properties before I even setup BJS on the canvas and see where I get.

On an ironic note, we got feedback results today from testers over the week... and no one seems to like having the canvas area restricted, so there's a push now to just let BJS have the entire window and lay the interface over it (which make our web designer twitchy  ). That would make the issue moot for one of our projects, but still outstanding in another where we show product details in 3D in a small outset. 

[adam]

You can use jsfiddle to recreate the issue:

https://jsfiddle.net/nzexh6a6/4/

[JLHolmes]

Thanks @adam, I'm working with it... my quick attempts haven't been unable to reproduce the issue so far. In the real case, the problem is occurring several layers into a React.js application, so there may be some shenanigans in react that are causing me to make assumptions.  I'll keep you guys updated.

[JLHolmes]

Hey there guys... I wanted to update this thread with my testing results.

So... I was completely unable to nail down this behavior... it was working differently based on various touch devices. Then I decided to reconstruct the problem in the Playground... and I found I could not reproduce it. So after days of narrowing down my code vs. Playground differences, still with no luck, I did find one difference. I noticed that the playground was using BJS 3.0 alpha.

So, I switched my code to 3.0 alpha, and bam... the problems went away. Not only that, I was having a POINTERTAP vs POINTERDOUBLETAP issue... and that went away, too.

@Deltakosh, @Wingnut... did I just stumble across a fix?

 

[GameMonetize]

We fixed a lot for v3.0 so this is highly possible

[Wingnut]

I'm thinkin'... hmm... umm... rendercanvas-within-a-flex-box... has more troubles ahead.

JLH, after "troubles went away", were you able to RE-test all those other devices (that each acted differently), again? 

If you can, that would be cool.  I bet you find troubles, if you do (even with 3.0).  Another question...

When you could NOT repro the issue in a PG, did you actually "hack DOM" and change the CSS of the PG canvasZone container (to flex-box-ish)?  Perhaps it already IS so, or the x-splitter element is.  I haven't checked.

If so, did you then packed other elements "around" the render canvas (siblings)?  (all ALSO within the flex-box-styled canvasZone container)

If so, did you then use sin and cosine crap inside the renderLoop... to cycle the borderWidths of those canvas-siblings... fatter and thinner?  (make them breathe) 

You know what I'm getting-at, yes?  We should make a playground, which has flexy styles on the canvasZone (canvas container) and then do ANYTHING to make canvasZone need to continuously resize (such as swell and shrink borders of adjacent html nodes).  (This SHOULD simulate a user continuously drag-resizing a container or window).

THEN... once we have canvasZone flexified... and something causing it to re-flow and re-size constantly... ONLY THEN can we really see how well our scene.pointerX and pointerY are maintaining accuracy.  Not an easy playground.  No React.js... just a playground-based flexi-CSS torture chamber.  Probably easier in JsFiddle.  Adam gave us a nice little starter-fiddle... https://jsfiddle.net/nzexh6a6/4/

I suppose nobody wants to search for trouble... if nothing is obviously wrong.    But... hmm, I also don't trust "it fixed itself".  heh.  If we DO make a test scene... we should probably aim-for small canvas in upper-right, the situation where JLH reported seeing problems most-often.  *shrug*

[JLHolmes]

@Wingnut Yep... when bugs just "go away" I get nervous, too.

I was not nearly as exhaustive as your suggestions. It's really easy to reproduce... revert to 2.5... voila... problem is back on any touch device (but not desktop), problem goes away with 3.0.0-alpha.

I have not dug into why, however... and have not tested how stable it is. It a couple weeks, our project goes out for a round of concept testing across a wide range of devices... so I may end up diving back into it.

[Wayne Jenkins]

I was experiencing the same effect with later versions too.  For me this was caused because I had css padding on the canvas.  I changed it to css margin and everything was fine. 

 

[JLHolmes]

Thanks Wayne.!