Perspective correct texture mapping

[devhang]

I have opened the same question on stackoverflow: https://stackoverflow.com/questions/47342853/perspective-correct-texture-mapping-in-pixi-js

My purpose is making a simple 3d engine with 2D sprite (I choose Pixi.js this time as I can control the vertices of sprite arbitrarily), but I cannot understand how to map the texture to a perspective projected quad correctly. 
texture: https://imgur.com/Z5xDL8V 
I am using the PIXI.mesh.Plane for the quad, if the subdivision is low

var verticesX = 2, verticesY = 2;
this.field = new PIXI.mesh.Plane(resources['palette'].texture, verticesX , verticesY );

This will give out the result of screenshot01: https://imgur.com/a/m261h 
You can see the texture mapping is incorrect. 

Goes on I increase the division to 4 after the projection to screen vertices, and calculate the division point linearly.

this.field = new PIXI.mesh.Plane(resources['palette'].texture, 5, 5);
var lt = [this.field.model.screenVertices[0], this.field.model.screenVertices[1]];
var rt = [this.field.model.screenVertices[2], this.field.model.screenVertices[3]];
var rb = [this.field.model.screenVertices[4], this.field.model.screenVertices[5]];
var lb = [this.field.model.screenVertices[6], this.field.model.screenVertices[7]];
var segX = this.field.verticesX - 1;
var segY = this.field.verticesY - 1;
var vertices = [];
for (var i = 0; i < (segY + 1); i++) {
  var l = (lb[0] - lt[0]) / segY * i + lt[0];
  var r = (rb[0] - rt[0]) / segY * i + rt[0];
  for (var j = 0; j < (segX + 1); j++) {
    var t = (rt[1] - lt[1]) / segX * j + lt[1];
    var b = (rb[1] - lb[1]) / segX * j + lb[1];
    vertices[(i * (segY + 1) + j) * 2] = (r - l) / segX * j + l;
    vertices[(i * (segY + 1) + j) * 2 + 1] = (b - t) / segY * i + t;
  }
}
this.field.vertices.set(vertices);

that is 4x4 division as 5 points for each edges, the result is shown as screenshot02: https://imgur.com/a/6HREk, but the height of each division is the same, the illusion of perspective is broken 

How can I achieve the result as the screenshot: https://imgur.com/a/ZEUHu 
The height of division proportion to the depth respectively 
Or am I heading to a wrong direction to solve the problem, please suggest me some advise, thanks.

PS. The debug message in the screen is describing the vertices coordinate of the field lay on ground in clipping space, not the screen space.

[ivan.popelyshev]

If you want just to map a quad, https://github.com/pixijs/pixi-projection is enough. It emulates GL.QUAD which is absent in WebGL  It also uses 3x3 matrices instead of 3x2.

However your formulaes are for bilinear projection which is different, and I implemented that in same plugin too. Its not that easy, because its actually a surface of hyperboloid and requires extra pain in shader and on renderer-side. After I implemented it, I compared my formulas with others and found error in http://iquilezles.org/www/articles/ibilinear/ibilinear.htm

As you see, there are many ways to map texture to quad. I have multiple formulas for different projections that I didnt implement yet.

As for real 3d 4x4 matrices, I yet to implement that, I'll tale them from my old (two years!) pixi 4.0-0-alpha fork https://github.com/gameofbombs/pixi.js with cards example: https://gameofbombs.github.io/pixi-bin/index.html?s=flip&f=cards.js&title=Cards , there were 3 projects that used it, they are waiting for me to improve pixi-projection enough. 

In pixi-v5, GoodBoyDigital will add their own 3d plugin, that they use for games like https://www.bbc.co.uk/cbbc/games/the-worst-witch-magic-adventure-game (f u bbc for regional policies)

One more related thread: 

 

[devhang]

Nice to meet you ivan, I started my 3D journey in Pixi.js with your pixi-projection plugin at first, Spirte2d (cannot easily tuned the vanishing point to make the illusion of 3D, when the camera is moving), then I build up my perspective projection engine and using Sprite2s for render the result (but some how there is some issue unwanted, I think this is also related to the texture mapping, please see https://youtu.be/nOapS5ThAaI, the border in white enlarge non-realistic as the quad stretch too large when near the camera).

So that I want to start s ome approach to fix it, but as you said my approach is similar to your bilinear method (I wasn't know the term of this kind method  ). 

I have heard some u/z, v/z, 1/z interpolation method http://www.reedbeta.com/blog/quadrilateral-interpolation-part-1/ , but I have no clue to apply in this situation (I just a n00b in 3D  ).

Is their any document or example about the mesh related classes? I wish to know more about the indices & uvs of the mesh class, thank for your information.

 

[ivan.popelyshev]

No, there are no docs, I advice you just read the sources, its simple enough.

There's better mesh in heaven (https://github.com/gameofbombs/pixi-heaven) no demos, but its basically this branch but in "PIXI.heaven.mesh" namespace: https://github.com/pixijs/pixi.js/pull/3902 
But I dont believe you need that thing right now.

Bilinear projection is not perspective, it has strange effects like you've shown in video. If you want to move forwards/backwards, you need perspective, but with Sprite2d and Mesh2d you wont be able to get up/down. That's the best you'll have: http://pixijs.io/examples/#/projection/plane.js . Left-right movement ok, up-down forward-backwards isn't.

You can either wait for me to implement 3d for pixi-projection, either use old pixi-4.0.0 fork.

If you are ready to do stuff with 4x4 matrices and not afraid of required math, you can try fork pixi-projection and implement 3d yourself. You wont get that "dont starve" effect with simple 2d.

Such demo existed for pixi-v3 and previous version of 3d fork, even with performant GPU patch for spine, but I cant find the sources  

[ivan.popelyshev]

Btw, there's a big project that waits for me to implement those features in projection. They used 4.0.0-fork before. Imagine that this thing will be in 2.5d: