Mapping 2D Mouse XY to 3D XZ movement relative to camera

[diamondage]

Hi 

 

I am just starting out with Babylon JS for a research project and it seems I am stuck finding a solution for mapping 2D screen mouse coords to 3D XZ movements relative to the camera.  The project involves simple moving and stacking cubes.  The problem is when my camera is looking into the z-axis I am able to figure out how to make the x and y of the mouse related to XZ in the app.  However this goes to heck when I move the camera out of the z axis say by 90 degrees.  The x and y will still translate the object as x and z but in from the camera perspective I've turned 90 degrees and the x and y of the mouse should change to z and x.  I've tried unproject but it seems I don't understand the problem well enough and am missing the Z for the screen coordinates.  I cannot use the pick ground strategy because it causes blocks stacked on top to 'jump' to the ray that hits the ground behind it.

 

My sample is here:

http://www.babylonjs-playground.com/#BTHKN

 

Thanks in advance for any help!

Wil

[GameMonetize]

Hey, why this technique does not work for you ? http://www.babylonjs-playground.com/?18

[diamondage]

Hi Deltakosh,

 

Great questions.  This technique only works when you are able to select an object with a ray that hits the ground.  If I start to stack cubes 3 to 4 high, and you move the camera almost parallel to the ground, the pick ray will not hit the ground.  As a result we will be unable to move the cube until the pick ray (the mouse) is moved to somewhere on the ground.

 

thanks,

Will

[GameMonetize]

Ok gotcha!

 

Did you see this topic then?  http://www.html5gamedevs.com/topic/16687-axis-dragging/

[diamondage]

Hi Delta,

 

Yes I have, and the issue with creating a hover plane is that if you are right at the center y point of the camera you can't select and move the object.  Again it only works if you are looking slightly down at the plane.  Where as if I can figure out how to unproject the value for the XY screen coordinates to the object when the camera is not just looking at the Z axis that would be optimal.

 

W

[diamondage]

Hi Delta,

 

Also I've attempted to do some research on this and the closest thing I've found are:

http://forum.unity3d.com/threads/unity-4-3-get-2d-mouse-coordinates-relative-to-the-scene.214091/

and related to unproject.

 

W

[GameMonetize]

If you are only controlling your objects on XZ, then why not just using scene.pointerX and scene.pointerY. on pointerdown, you save the current value and then on pointer move, you translate your object accordingly?

[diamondage]

Hi Delta,

 

That is correct if your camera is as I positioned it in the example looking at the Z axis.  If you use the arrow keys and click mouse to rotate the camera 90 deg or even 180 the x and y no longer maps to X Z.  If you rotate to 90 deg, mouse X is not Z and mouse Y should be X.  So I am missing a transform that takes into account of the camera look at vector.

 

W

[GameMonetize]

Ok gotcha: so you were right you can convert 2d to projected 3d with Vector3.Unroject

 

Example here:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Math/babylon.math.ts#L3195

[diamondage]

Hi Delta,

 

I think you are onto something.  I will have to give that a try.  When in doubt RTFC.

[GameMonetize]

Lol!

[KramSurfer]

Attempting to do the same thing.  But am unclear on the implementation.

I looking to place a mesh under the mouse pointer at a z-depth of 40 relative to the camera.  

I have other selectable items beyond that depth, so putting a large plane or sprite and using a ray is not a good option. 

Thanks