Confused by all unproject implementations

[KenRubin]

Hi Again,

Let me explain what I'm trying to do first:

I want a panel of 3D objects which line up along the left-hand side of the canvas/viewport.  I'm trying to create an object factory of sorts.  The user will click and drag objects from the object factory into the scene proper, the code will clone the selected object when this occurs.  I'm using two scenes for this, one is the main scene, with a ground and control-enabled camera receiving the clones and accumulating state; and another scene, for the factory, that is essentially static but has a camera and lights and meshes (the factory prototypes).

As I've said, I want this factory to be arrayed down the left-hand side of the screen.  If I choose just any position for the factory objects, they will go into and out of scope when the canvas aspect ratio changes.  So I want to be able to calculate a ray which vertically cuts the camera's left-hand viewing-frustum plane, say somewhere in the middle, parallel to the near and far edges.  I think this should be a vertical slice, abutting the left edge of the viewport.

Here's how I tried to do it:

First, I tried using getFrustumPlanes. But the planes it gave me were oddly skewed.  When I generated geometric planes for each of the six, for instance, they did not form a convex solid.  And even when I mentally tried to figure out which plane was which part of the frustum, I just couldn't make sense of the numbers.  I would have expected 2 along each camera-centric axis (e.g. + & - along X, + & - along Y, + and - along Z--I setup a straight-on camera to help in understanding the numbers to no avail).

Next, I tried a few of the unproject methods.  I think these methods should have converted a 2D point on the screen to a 3D point (or ray) in the world.  I read through many posts on this site to use what I thought were the best values for the camera's projection matrix, world matrix (identity), etc....  Never got anything even close to good numbers.  Mostly they were NaN until I saw one post which accessed maxima from the engine.  But when I positioned my mesh (using a sphere) at the returned 3D vector, it always seemed to go to the same, incorrect place (irrespective of my choice for 2D starting point).

Next, I successfully wrote an iterative algorithm which used many of the same transformation matrices along with project.  This way, I would choose a 3D point, test where it was in 2D, and knowing the fixed orientation of the camera, choose another 3D point which projected closer to the 2D point for which I was aiming, etc....  Sort of like a newton's method for iteratively approaching the correct value that unproject should give me in fixed time.  OK, I can do it, but I want to do it correctly.

So my last attempt, for which I have a playground, uses: createPickingRayInCameraSpace.  I think this should give me a ray which looks like a point, but really rides the intersection of the left and top viewing-frustum planes (because I choose 0,0).  However it does not.

Playground demo:

http://www.babylonjs-playground.com/#1CHH2O#1

Please help.  I don't care if I use any of these methods.  Is there a better way?  Should I be using the aspect ratio directly?  Should I just do my own matrix math? 

Thanks,

Ken

Edited March 19, 2017 by KenRubin
fixed title

[GameMonetize]

Hello

did you read this code?

https://github.com/BabylonJS/Babylon.js/blob/master/src/Math/babylon.math.ts#L4332

This could help you better understand how planes are built.

 

The unproject could also work. Can you provide a PG of what you tried?

[KenRubin]

Hi Deltakosh,

I did not read that code.  I'll take a look.

Also, I'll build a PG for the unproject attempts when I get home tonight (in a few hours).

Thanks,

- Ken

 

[KenRubin]

OK, I made a PG with my flawed unproject implementation.

http://www.babylonjs-playground.com/#UWSIL#51

 

Some notes:

1) If I just position the cube at "worldVector", it is not visible at all.  Perhaps because it is too close to the camera?  So I make sure the camera looks down the z-axis and move the object to z = 0.  I don't want to do this but I think it should work as long as I just size the browser rather than update the camera position.

2) It doesn't matter what coordinates I pick for source.  The cube is always positioned in the UL corner.  I would think that using: (1, 1, 0) (or perhaps: (engine.getRenderWidth(), engine.getRenderHeight(), 0)) would choose the LR.  I would like to choose the viewport-relative position for the mesh.

 

When I change the aspect ratio by sizing the browser, the object doesn't stay fixed against the side of the viewport.  Perhaps this is because I'm fixing z?

Thanks for taking an interest, I appreciate any help you can give me.  

- Ken

[GameMonetize]

1. You are right, you cannot see the box because it is clipped by the near plane. The trick could be to just slide the initial vector a bit: http://www.babylonjs-playground.com/#UWSIL#52

2. It works for me: http://www.babylonjs-playground.com/#UWSIL#54 

[KenRubin]

Awesome!  Thanks so much, Deltakosh.