Canvas2D, Origin and Positioning System

[Nockawa]

This post might interest @royibernthal @MasterK @TMTH and @adam I think.

Most of you have reported issues with using scale/origin and the positioning engine, I've looked to these issues and then to my code and I finally ended up wondering if what I'm currently doing is the right way and also if there's a right way to do things.

Ok, let's be a bit clearer, for now, here's how things work:

You create a Prim, set its margin, marginAlignment, origin, scale, rotation

1. During the pre rendering phase I compute the position/size of this prim based on the margin and marginAlignment properties
2. Then, I compute the globalTransformation matrix using the position computed by the positioning engine, the rotation, the scale (both considering the origin).

The thing is the rotation/scale/origin are processed after the positioning engine. So let's take some example to illustrate the consequences of that

1. If you set a scale of 0.5 and align center the prim the result will be as expected if the prim's origin is 0.5,0.5, because the scale shrink the prim at its center so the align center is still good because there's the same amount of pixel shrunk to the left and right (let's just consider the X axis, the same apply to the Y). If the origin is 0,0, then the Prim shrink related to its bottom/left corner, so all the pixels that are shrunk will be on the right of the prim, hence resulting to a bad align center.
2. If you set the scale to 0.5 and align left, then the result will be good if you set an origin of 0,0 and it will be bad for other values (0.5,0.5, 1,0, etc.), if you think about it a bit, you will understand why and realize it makes sense considering the current implementation of things.

Now I could solve this issue by changing my code with the following: applying the scale to the prim before doing the positioning, the positioning engine would center the prim with its actualSize: after the scale was applied.

I first thought it would be the right solution: apply scale and rotation (in respect of origin as always) before the positioning engine, but now consider this:

1. You create a prim with origin 0,0, align it at left and then make it spin by increment its rotation each frame.
2. The result is a prim that rotates around its left/bottom corner (imagine the motion, the prim doesn't rotate around its center, but rotation forming a large circle with its right/top corner).
3. How will the align left behave in this situation? I mean I can code it, it's easy, but the result would be weird, don't you think? At least I do, I don't think it as a solution...

So what is the solution? That's my question for you guys and I need your input/feedback/thoughts, anything.

• Would the solution be applying the scale before the positioning and the rotation after, would it make more sense?
• I already have two scales internally: one you know about and a "post" scale I had to introduce to fix a bug reported by @TMTH where a Sprite with a SpriteSize of 64 and a Size (display size) of 100 was not positioned correctly, because I was changing the scale to 1.5 (roughly) for the sprite to be 100 on screen and it was bad because scale was considering origin and I needed a scale that wasn't, so I've introduced a postScale, computed at the very end. 
  Anyway, I could introduce a third scale, (a second for the user), which would be a preScale and that would be applied before everything, I think it would solve @royibernthal issues. But I don't know if it's clever to expose to the user more than one scale property.

So basically, I need to know what you guys want when positioning is used with primitive rotation/scale, because now that's what is missing: a behavior, a model that fits any needs.
So please take the time to think about it as much as needed and...talk, give me Use Cases, propose solution if you can, anything!

Thanks

[adam]

Scale, Rotate, Translate

It's the only way to accurately position an object with an origin that isn't touching the side it is being aligned to.

[royibernthal]

How does alignment behave at the moment?

Here's a PG where origin doesn't change anything, can you please help me understand it?

http://babylonjs-playground.com/#1EJUJH#2

 

In the case you described above, if I got it right, I'd imagine the following order:

1) internal align of object according to its origin

2) scale

3) external align (in this case left align)

4) rotation

[Nockawa]

@royibernthal

Look at this PG: http://babylonjs-playground.com/#272WI1#81

Now imagine that we set a marginAlign of "h: left, v:center", I rotation is made before the positioning, I think the result will be the rect spining weirdly, but still sticking the left border and being centered vertically.

My feeling about this is it's not really what the user would want. but I don't really know, I need the feedback of as many people as possible

[royibernthal]

@Nockawa Why is it weird?

http://babylonjs-playground.com/#272WI1#82

Aligned to left and rotates around its origin, seems fine to me.

[Nockawa]

If I perform the rotation/scale before the positioning you won't get this result: the position of the rect will be changed in order for it to match the "h: left, v:center" alignment. You will always see the rect in the canvas and its position will always be left/center. It will probably spin as if its origin would be 0.5,0.5 but I'm not even sure of that, I think it'll be spinning differently, because I'll change the position of the rect.

Currently I'm using actualSize to perform the positioning, but if I compute the rotation/scale before, I will have to use the resulted boundingRectangle to do the positioning, I don't think how I could do this differently.

[TMTH]

Just a few thoughts, unstructured, stream of consciousness style.

1. Working with primitives we have two very different areas – primitive transformations (all that matrix things) and primitive alignment (something to make developer’s life easier). On can leave without the latter, doing everything by hands, without any help from the framework, but not without the former. So, that transformation layer has to be stable and has to have very predictable behavior. Now, the framework has some problems with that predictable behavior.

1.1. Origin is not origin.

Origin is the point (0, 0) in the coordinate system where our primitive is defined. It is natural to define primitive in its parent local coordinate system – we can define scale, rotation and translation inside the parent, based on parent’s origin point, and using translate * rotate * scale transformation we can place the primitive anywhere we like.  

That is not the case of Prim2DBase.

1.2 In search of underlying model

Now I’ll try to reproduce the model as I managed to understand it during my evaluation of canvas2d:  

* Size of the primitive is specified with respect to local coordinate system of the parent, so scaling parent will scale child also, and setting child.size = parent.size will make child to fill the parent. No problems here.

* Position of the primitive is specified in some kind of local coordinate system of the parent, with origin point moved to its bottom left corner. You can set parent’s origin to any value you like, it will not change child position. So, it is not the local coordinates of the parent – child positions ignore parent’s origin. And one more thing – position of the primitive is a vector, that corresponds to some point inside the primitive. And that point is not the origin of the primitive. And we can’t change it. And the only way to find that point is to read the docs or to try use it.

* Rotation is specified in local coordinate system of a child, with respect to child’s origin. As I understand it, applying several rotations to primitive is not directly supported. If you want to create some rotation animation about arbitrary point (rotation2) of already rotated primitive (rotation1), you have to use two primitives. One with scale, position and rotation2 and another (child of first) with rotation1.

* Scale. I give up here. I can’t understand the logic of scale (see PG for example http://babylonjs-playground.com/#1P2NDU#1). From general considerations I see only two ways to apply scale – based on parent origin or based on child origin. Neither seems to be implemented (or it is, but with the current implementation of primitive position it’s hard to understand how it is working).

1.3 And …

My current opinion is the following – the whole module is over-engineered, it lost the simplicity and logic of OpenGL transformations, and gained nothing in return. It’s easier for me to position thing using matrix transforms then to understand the logic of canvas2d.

What I suggest is to refactor the system using the following principle – provide user with a simple way to do simple things, and let him do complex things any way he wants.

One way to do it:

State that Canvas2d – is 2D vector graphics library. Vector graphics is all about vectors and transformations. The framework does not have to hide that fact from user. Parent-child hierarchy has to be interpreted as coordinate systems hierarchy. Framework have to simplify some standard thing, providing some tools to deal with complex things.

For example, in the pair of two primitives – parent and child, parent will have origin property, that will affect child primitives only. Every child will have scale, rotation and position properties that will be used to build most frequent transformation (translate * rotate * scale). For every other possible transformation, child provides method to add user defined transformation matrix, maybe providing some helper method. (child.addTransfrom(new RotationTransform(point, angle)).  

I warned that it would be a stream of consciousness

2. About primitive alignment

First let say, that there are two very different kind of alignments – positioning (left, right, top, bottom, center) and fill-stretch.

2.1 Let’s talk about first group. May be I don’t understand something, but I see the problem in the following way – we have user defined scale and rotation transform, and we have to calculate translation transform by ourselves. And…where is the rocket science?

1. Take bounding rectangle of scaled and rotated primitive (AABB, in coordinates of parent),

2. Apply margin vector (+ to tr, - to bl) to get primitive effective rectangle,

3. Take parent primitive rectangle (AABB, in self coordinates)

4. Apply padding vector (+ to bl, - to tr) to get layout area

5. Calculate required translation (align one rectangle inside another)

But if you take into account current implementation of origin/position things, you will get the rocket science.

2.2 Scale also have to be computed … oh my

 

That’s all is only my opinion after more than a month of canvas2d evaluation. May be I just don’t understand something; may I don’t get the idea of the framework. But what I see now  – canvas2d tries to hide the complexity of WebGL, but the abstraction used to do it is itself more complex than WebGL (in the part of transformations)

[Nockawa]

Wow, thank you for taking the time to answer with such detailed info, I don't mind the stream of consciousness at all, on the contrary!

1.1 The origin property has nothing to do with translation and parent/children relationship. It's used to define the point that will act as the center for rotation and scale. To take the rotation for instance, with 0.5,0.5 the prim will rotate around its center, with 0,0 it will rotate around the left/bottom corner of its BoundingBox. It is very useful because based on your need this point may have to be positioned at different places. Changing the origin won't move the primitive (if you change either or both of the parent's or the prim's origin).
If these rules are not true as of today, then there's a bug.

1.2

* Size: OK

* Position. The transformation origin is the bottom/left of the prim, as it's state clearly and many time in the doc, as I say above, it has nothing to do with the origin property.

* Rotation. It will consider the primitive's origin to determine the center of rotation, it currently works

* Scale. You gave up and you were right to do so, because it was bugged. I've tested your PG locally with the latest version and it works as expected now. The rule is the same as for rotation, the primitive will scale (with the amount of the accumulated scale through the parent/child hierarchy which is actualScale) relative to the center defined by the origin property. I think it's also something that the user could expect, well, I hope.

1.3 Well, with what you test, I can understand this opinion, because all the features didn't work as expected, but it's getting solved. Believe me, I would have loved to keep it simple and never consider a custom origin, it drove me mad, but it's a needed feature. By default the origin is 0.5,0.5, maybe that was a mistake because when everything will work as expected this indeed make things still a little complicated. 0,0 would have set the origin to bottom/left and all the transformations would have applied as you expect, simply... I think the current model (again, once debugged) will be easy enough for people to use things, again, your experiments was unfortunately made at a time when I changed the scale behavior and origin to make a feature for @royibernthal and I've crashed everything, then I went to refactor the positioning because it's was bugged too...

So I won't change the current properties/state, the only thing that could change and it's something for us to discuss (because this would be a breaking change) is the default origin value. 0,0 would lead to a "simpler" library as the concept of origin would be inexistant (it's useful when the value are different of 0).

Ok...:) 2.

2.1 YES! I completely agree, and even if it's not rocket science, I was dumb enough to make the positioning computation before the rotation/scale was applied...Well to my (poor) defense, I never intended to add positioning in C2D (and I never should have, one of the biggest regret so far, it should have been a GUI Lib stuff) and when I did, well, I didn't think that much and put it in a place where it was computed before...
But no, it's not rocket science (anymore, since I've refactored the positioning engine) to invert the process and make everything in the order you stated.

Taking the origin into consideration won't change anything, the origin will only change the position of the primitive, as you stated, we compute the position during alignment, so origin should be harmless, I'm not sure an animated prim that is rotation will act the same if its origin is 0,0 or 0.5,0.5, it's something that should be tested. But the more I think about it the more I'm convinced that origin should be ignored when positioning is on. I don't think this property brings added value in case of positioning...
The good thing now I've refactored the positioning engine is that internally I know/flag if a prim is using it or not. There're two separate states and something two separate code paths.

Concerning your conclusion, well, I think it's a bit harsh but I understand it because you tested thoroughly at a time when there was regressions and also with things that never worked as expected (scale and scale propagation, designMode, positioning...).

Thing is: @MasterK came with the need of the designSize, I thought it was a great idea and I understood the need, but the way I've implemented it broke lots of stuffs and I had hard time to make cohabite designSize and regular mode.
I'm solving this issues right now, I'm also supporting Device Pixel Ratio (DPR) now a changes of hardwareScalingLevel.

For now, I think I'll release first a version that works better with scale, scale propagation, origin, DPR/hardwareScalingLevel and designMode. Hopefully bug free. Then I don't think it will take a lot of time to invert the transformation/positioning computing order.

[Nockawa]

ok guys, @TMTH @MasterK and @royibernthal there's a new preview and updated PG with many fixes... the designMode, scaling through hierarchy, origin and now the DevicePixelRatio (DPR) is supported (so please @MasterK if you can test your game on a mobile with the latest preview, that would be nice).

[Nockawa]

Hello people,

I just wanted to give you some info about what I did lately. I have a nasty little test where I combine DesignMode, Scale in a Group2D and Primitives aligned inside this Group2D with also scale and margins...
And right now this is working!

I've followed one rule which makes sense: ignore the origin when a margin different than bottom/left is used. Then I had to consider scaling before the positioning and now things work as expected.

My last challenge and clearly not the least will be supporting also rotation with margin alignment. @TMTH it's not that easy one might think!
My first test of a rotation Rect2D with no origin (so left/bottom corner) clearly demonstrate that to find the right offset won't be an easy task... But I hope I'll succeed.

[Nockawa]

Hello Guys,

You'll find attached a video of the progress, here's the code:

let primScale = 2;
let primSize = new BABYLON.Size(10, 5);

let ssc = new BABYLON.ScreenSpaceCanvas2D(scene,
    {
        id: "ScreenCanvas",
        designSize: new BABYLON.Size(500, 300),
        designUseHorizAxis: false,
        backgroundBorder: "#80C080FF",
    });

let group = new BABYLON.Group2D({
    parent: ssc, width: 150, height: 100, scale: 2, x: 100, y: 100, id: "group", children:
    [
        new BABYLON.Rectangle2D({ id: "groupBackground", fill: "#8080C0FF" })
    ]
});

let rect2 = new BABYLON.Rectangle2D({
    parent: group,
    marginHAlignment: BABYLON.PrimitiveAlignment.AlignRight,
    marginVAlignment: BABYLON.PrimitiveAlignment.AlignTop,
    marginRight: 10,
    marginTop: 20,
    size: primSize,
    fill: "#00FF00FF",
    scale: primScale,
    rotation: Math.PI / 2,
    padding: 1,
    children: [
        new BABYLON.Rectangle2D({ fill: "#C0C0FFFF" })
    ]
});

let inc = 0.01;
setInterval(() => {
    rect2.rotation += 0.005;
    if (rect2.scale >= 4) {
        inc = - inc;
    }
    else if (rect2.scale <= 1) {
        inc = - inc;
    }

    rect2.scale += inc;
}, 10);

I have still things to deal with but I'm near the end of this dev.

posEngine.mp4

[Nockawa]

Ok folks, I've just commit on the master branch the new release with the updated positioning engine and few fixes, I'll write a post about it this afternoon, there's two little things left to be done, but not regarding positioning

[Raitch]

I'm quite confused currently. I was using origin (0.5, 0.5) to scale from the center. How could I achieve similar results currently?

I tried creating a giant Group2D and have my Sprite2D have:

    marginHAlignment: BABYLON.PrimitiveAlignment.AlignCenter,
    marginVAlignment: BABYLON.PrimitiveAlignment.AlignCenter,

But I'm not really sure what happens.

Changing the rotation value seems to alter the position of the Sprite2D for some reason. 

Changing the scaling seems to move the position of the up for some reason. 

I can set different 'scale' values upon creating the Sprite2D, but when I try to change .scale afterwards it just "escapes" out of the screen after a couple of frames. 

[Nockawa]

it is still the case, there was a but with sprite2d but I have fixed it yesterday but I didn't create a release yet.

I'm away right now I will do it asap

[Nockawa]

@Raitch I've made a new release, the bug was that a Sprite2D was always considered as using the positioning engine, and then the origin was forced to 0,0. It's now fixed.

[AlbertTJames]

Great job Nock !

Let us know when you have the documentation post done, or if we can help for something.  

[Nockawa]

I've already updated some doc, but now we don't update it on the fly, it's a weekly build so you'll have to wait a bit, well, I didn't update that much but you can see a new pg here. The latest posts I've made in the Canvas Main post thread are pretty much summarizing things well. 

[AlbertTJames]

Hey @Nockawa,

 

Could you explain this behavior : 

http://babylonjs-playground.com/#BN6OY#1 

- What makes the rectangle float away like that ?

- Should the design aera crop overflow from children view ? just saw one of your post mentioning that as a future feature

 

Thanks !

[Nockawa]

6 hours ago, AlbertTJames said:

Hey @Nockawa,

 

Could you explain this behavior : 

http://babylonjs-playground.com/#BN6OY#1 

- What makes the rectangle float away like that ?

- Should the design aera crop overflow from children view ? just saw one of your post mentioning that as a future feature

 

Thanks !

Well, apparently if you don't use alignment and animate scale of the rect things get bugged....

I've just created a new issue I'll take a look after I've coded the Clip feature that I started few minutes ago (the area crop you talk about).

[AlbertTJames]

You're the best

[adam]

This bug has been fixed.

Note: Even though this has been fixed, newcomers should choose GUI over Canvas2D.