Canvas2D Main Post

[Nockawa]

Ok people, it took times, sweat, blood and tears (yeah, that much), but I refactored the positioning engine of Canvas2D !

Why? Because it was bugged, the margin/padding wasn't considered when the primitive was inside another one that were autoSized (without explicit width/height). AutoSize wasn't working as expected and there was a bunch of little annoying bugs here and there.... And the code was messy (well, I'm not sure I improved on this, but well...).

A simple example would be the Canvas Profile Info:

let canvas = new ScreenSpaceCanvas2D(this.scene, {
    id: "ProfileInfoCanvas", cachingStrategy: Canvas2D.CACHESTRATEGY_DONTCACHE, children:
    [
        new Rectangle2D({
            id: "ProfileBorder", border: "#FFFFFFFF", borderThickness: 2, roundRadius: 5, fill: "#C04040C0", marginAlignment: "h: left, v: top", margin: "10", padding: "10", children:
            [
                new Text2D("Stats", { id: "ProfileInfoText", marginAlignment: "h: left, v: top", fontName: "12pt Lucida Console", fontSignedDistanceField: true })
            ]
        })

    ]
});

You can now do such thing with a good reliability, I mean the Rectangle is aligned on the top/left of the window away from 10 pixels (the margin), the content of the rect (which is the text) will have a padding of 10, so there will be 10 pixels around the text. Thing is, there's no size set for the Rect! Because we want to fit the Rect from the Text and the marging, padding and also the space took by the rounded Radius of the rect. So now I correctly compute all of this, when the Text is changing, I compute its size, then I compute its layoutBoundingInfo object which is the area it takes for the layout and I'll do the same for the Rect, where the padding, rounded border will be considered. It may not sound like it is, but it's was really hard to achieve because there's code re entrance everywhere, a child may needs the size of where it can be positioned to compute its own size, but this size is sometime also determine by the child (and its siblings) itself, so well...not easy.

I'm also now supporting scale with positioning, request by @royibernthal it may not fully work because I didn't test is thouroughly but the code base is here and it's only a matter of tweaks now.

@royibernthal I think you can try the new preview for your project, the alignment should work better, but you will report me issues concerning the propagation of the Scale, so please let me know what's not working so I can take a look and fix it asap.

@MasterK You should try this with you current project, the designSize should work again, if it's not the case I'll take a look. I have other reports from @TMTH about bugs on this matter, I'll take a look at all of this starting tomorrow.

There's a new preview build available, ready for you to shot all the regressions, I honestly hope there will be as few as possible, because I've spent a really big amount of time to check everything I could check again, but I couldn't check your code, so please, do it for me.

What's next?

I've many little bug fixes and enhancements to do, there are all reported here: https://trello.com/b/eKMKhCFc/canvas2d and I'll start on it tomorrow.

Any regressions will be treated as absolute priority.

I'll also take a look at @adam GridLayoutPanel tomorrow, there will maybe need some changes...

Once all of this is done I sill have the Geometry clipping feature to code, and I think I'll also code the parent/child change feature.

Then a little break will be done on C2D.... with a minimum of maintenance of course... So I can resume the work on the GUI Lib and I think I'm about to finally have the foundations to make a good one!

[MasterK]

@Nockawa It looks you did a hard fight with bugs. great ! I'll update and try it when things not too much.

And my current version of c2d has a problem for long: set primitave levelVisible = false. and do some change to it, when canvas use Cache Strategy, it will throw gii is null or other error on next display frame. I dont make case. but lots of cases of our project show this problem. So lots of our UI panel need to delete canvas and recreate again.  Just let you known about this.

[Nockawa]

1 hour ago, MasterK said:

@Nockawa It looks you did a hard fight with bugs. great ! I'll update and try it when things not too much.

And my current version of c2d has a problem for long: set primitave levelVisible = false. and do some change to it, when canvas use Cache Strategy, it will throw gii is null or other error on next display frame. I dont make case. but lots of cases of our project show this problem. So lots of our UI panel need to delete canvas and recreate again.  Just let you known about this.

Can you elaborate a bit more? Is it something like:

1) use the CANVASLEVEL cached strategy

2) hide a primitive by setting its levelVisible = false

3) change the primitive position/size or any other attribute

Then an error is thrown next frame.

Is that it?

[MasterK]

That's it. But i tried to make a simple case and failed. don't know if there is other issue effect.

But all error caused by primitive levelVisble=false.

and dispose a hidden primitive may cause it too. 

[Nockawa]

Hello people,

There's a new preview of Canvas2D and the PG is updated. What's new lately...well, bug fixing and improvements!

1. Text Rendering

• I've improved the way characters are generated and stored in the FontTexture. It should be more reliable now, I've also finally introduced the concept of Text Baseline, and that will be useful in the near future for Text2D, to make sure the text is always correctly aligned vertically
• I've improved the support of the BMFont files at the request of @TMTH I can know display font generated with the BMFont software, well, at least the ones I tried.
• I'm still experimenting improvements of Text2D rendering at the request of @jschwuch and also previous people, I hope to solve it soon.

2. Device Pixel Ratio
Well, I had it on my todo list for a long time and now it's done, if you run a Canvas2D on a mobile device with a DPR > 1, the size will be identical to a DPR 1 equivalent. You can use Chome debugging "device toolbar" to play with it and you'll see it works. I've also fixed the bug of the SceneOptimizer that was changing the HardwareScalingLevel which caused the canvas to double in size...

3. Scale and DesignSize
It has been a long fight and I hope it's over...I've changed the way DesignSize is managed, it's no longer stored in the Canvas2D Scale property but in an internal one, so it should be more stable/reliable for now.
Scaling through hierarchy wasn't working so well lately, I made some regression trying to improve things. I can say now that I've stabilized things and the scaling work as it should be. I guess I have to check with @royibernthal if his project still require some work for me, but I'm confident I can make it work if there's still some little bugs

What's Next!?
I will follow the many remark/advises of this thread to make positioning working with scale/rotation, I think I know what to do, but I can't guaranty it will be the unique solution, I hope.
I'm finishing text improvement then I switch to this final task before resume what I was supposed to work on

Canvas2D is getting more and more reliable, the many input/bug reports from everybody and especially @TMTH lately pushed me in this direction, I know I still have some improvement to make on the caching system, but well...it will wait a bit!

Give me some feedback, remember, if you have issue, post a new question mentioning me!

Thanks!

[Nockawa]

It's Christmas time! Well, almost!

I have few presents, I worked hard to deliver them, before speaking about everything that was made in the latest preview (available here) I would like to speak about text rendering! So this post is only about that! @jschwuch and other should be happy about it.

What was wrong until now

• To construct the FontTexture texture, I use a HTMLCanvas to render each character into it, then copy the result back in the texture. The JS API would tell you about the width of a character (which is not always accurate by the way, the number it returned it sometimes smaller than what is really rendered, wtf...), but it's not giving you neither the height or the baseline. And these are two values very important when you do text rendering. I finally have the piece of code that find them.
• I discovered that the center of a pixel for a HTMLCanvas 2D Context was 0.5,0.5 and not 0,0, so far I was rendering everything aligned to half a pixel....not the best way to have a great quality, specially for the small font!
• During rendering I was using bilinear filtering and the ALPHA_COMBINE blending mode (except for Signed Distance Field which doesn't require blending). Turned out that things were a bit complicate than this, that's why @jschwuch had the bug about white font over white text

Here's a screenshot of "before and after"

Clearly, things improved!

So what is the deal now?

• During the writing of a char into the FontTexture, I do a pass to compute the pixels in Premultiplied Alpha, the RGB pixels are computed with their alpha and I store everything. Then I use the (debugged, thanks @Sebavan) ALPHA_PREMULTIPLIED blend mode of Babylon.js to render the text, the quality is improving because the antialised pixels (you see, the grey pixels on the left pictures) are now correctly blended with the background color.
• Another thing, during the development of Text Rendering I switched many time between bilinear and nearest texture filtering, now I know why: there's not a right choice of filtering, it depends on the situation
  • If your text is rendered in a Screen Space Canvas2D, then nearest filtering has to be chose if your text is rendered with a scale of 1. Using bilinear filtering would lead to a blurry text for small fonts, like the picture below. To compensate this issue you can choose to render the text using the fontSupersample setting, the size being double internally, the bilinear filtering produce a more accurate result. Note that bilinear filtering should be also chose if the text is rendered with a bigger scale

• If your text is rendered in a World Space Canvas2D, then the bilinear filtering should be always used, if you're dealing with a small sized font, use the superSample settings. Below are two WSC with bilinear filtering on top and nearest on bottom for the "World Space Canvas" text

 

What should you do?

Well, nothing special in pretty much all the cases, because even if there's a useBilinearFiltering setting for Text2D if you don't set it I will take nearest filtering for SSC and bilinear for WSC. You can choose to override this behavior by setting the value to true or false. Remember than this conversation is about non SDF mode, if you're using Signed Distance Field you won't have to worry about texture filtering (because it has to be bi/trilinear).

Are we there, finally?

Well, almost! There's one more thing left to be done to have a pretty decent text rendering and that would be considering the font's baseline while centering/positioning the Text2D primitive. Today it's not the case and you can see on the green buttons above there's more space below than above, it's because I center the Text2D with the font's height and it's not what should be done, a small offset should be applied to consider the baseline.

Don't remember, Signed Distance Field is your friend! No blending, no transparency, very fast rendering and good rendering quality in WSC.

[Nockawa]

Ok, it's time to talk about the latest preview. You can find the git issue here. Some topics about the issues it solved here and there.

I'll try to be as short as possible. Some people wanted the scale and rotation to be considered when the positioning engine was involved (i.e. when you use margin/marginAlignment) and also with the DesignSize on, I have thought about this quite a lot and came up with the answer that it could be done if the origin property was ignored. So I programmed a lot to make this happens, it's not trivial and I've made more tests and fixes than coding the feature to make sure it would be the most stable/compatible with the previous code as possible. I've tested everything I could including a Canvas with DesignSize, containing a Group with a Scale of 2 which contain a zooming/rotation primitive (also scaled by 2) aligned to the right/top corner of the group with a margin. I could resize the whole window, everything working fine. I'll do some doc soon and share some PG too.

@TMTH and @royibernthal that is for you, if you still have issues, contact me directly, I'll try to solve them. @MasterK whenever you can find some time to test it, tell me what you think about it.

I've tested so much that I found a lot of little bugs, I took the time to fix them all, I wanted this release to be the most stable as possible. I don't intend to touch this part of the code unless there's bug fixing to do, from now on I'll think very carefully about adding a feature involved in this part of the code.

This development came with additional ones:

• Text rendering was improved, as explained in the post above.
• I'm very proud of the little logging system I made to log the many methods called during the positioning of a prim, that was useful for debugging. This system has the major benefit to waste no cpu time when it's turned off (which is the case by default). To turn it on you have to set this variable to true and reload the web page. FYI I use a feature of (the infamous) TypeScript called Decorator to overload the method/properties that should log info to output text in the console, before/after their call. If you are curious about what is output, take a look at the file attached.
• There's a new setting you can use during the creation of primitive: alignToPixel. When true the primitive will be aligned to the render target's pixel, which mean if you position a Rect2D to 10.5,10.5 it will be rendered at the 10,10 position avoiding an antialiasing. I had to code this because the positioning engine no longer round the computed positions of the primitives, because it's dangerous when scale is involved. The only primitive that force this setting to true is the Text2D one, otherwise the text rendering quality is not very good.
• There's a small breaking change in the constructor of the BitmapFontTexture class. The class was introduced in this beta of babylon.js, I figured I could change it before the official release, you'll see that it's no big deal, there's one more argument before the onLoad one, called premultipliedAlpha, because now the FontTextureBase class support the usePremultipliedAlpha property (which is used by the Text2D to select the adapted blending mode).
• I've added in the Canvas2D profiling windows the time taken by the rendering of the Canvas2D...Go figure why I didn't do this before...
• I've made several optimizations, caching more stuffs and also...avoiding some methods to be called more than once...no comment...

Well, that's it! Try it, it should be better, if it's not (because I'm always too cautious), I'll help and fix asap. But I do hope it's a stable release.

I'll update documentation soon and include new PG to demonstrate the new features.

log.txt

[Nockawa]

Me again, after a long silence, I have a lot of things to say!

With the help of @RaananW I just did a refactoring I planned for quite some time now: dropping the Matrix class for a new Matrix2D one!

Performances are critical to me in Canvas2D and the use of the Matrix class were not the best fit for a 2D Engine. This class stores a 4x4 matrix, 16 floats, and use them all for the various operations.
A 2D Engine has no care of Rotation around X or Y and doesn't know about Z values and don't use projection matrices. And this is a big deal because we can switch from 4x4 to 2x3. In theory it's a 3x3 matrix, but the last column is always equal to [0,0,1] so there's no point to store these values and the computations are greatly reduces when we assume these values.

The result is quite impressive, I wouldn't think it would be that much, I guess it's a testament to the high performance of the rest of the code of Canvas2D (heavily relying on caches), one of my local test which is like this PG which is rotation 50 prim inside a rotating group was about 9ms of rendering time in Internet Explorer (yeah, the crappiest JS browser, ideal for perf optmizations) to 5.5. So Matrix operations were a big deal of the whole rendering process (when you move/rotate the prim of course). On Chrome I switched from 1.5ms to 1.2... Well, I'll take it!

The consequences for you my friends: there's a price to pay!

Breaaaaaking changes, I no longer expose Matrix related properties, but Matrix2D. I don't know if you all use the few properties that are concerned, I think it's unlikely, but well, you'll tell me. But I think it's a good move for the 3.0 of Babylon.js.

That's all!

[davrous]

Wow. Awesome job @Nockawa. Perf are really impressive!!!

[Nockawa]

I've made a little update to reflect various feedback, from @MasterK and @royibernthal mainly, you can see the PR here

Basically here how things are working now for setting the position/x/y properties:

• if you don't use the positioning engine (no margin involved) you can use position/x/y to set an absolute position (well, relative to the parent's reference of frame)
• Before this change if you tried to set position/x/y while positioning was involved I would issue a warning message in the console telling you that you shouldn't do that because it wouldn't make sense, so I've changed that because some people don't look at the console message... @royibernthal is you read this  
  Now things will be more like @MasterK wanted them to be: it will set the margin value (in pixel) considering the alignment:
  • if there's a left/center/stretch horizontal alignment: it will set the marginLeftPixel
  • if there's a right horizontal alignment: it will set the marginRightPixel
  • if there's a bottom/center/stretch vertical alignment: it will set the marginBottomPixel
  • if there's a top vertical alignment: is will set the marginTopPixel

I wait you feedback on this: is it a good thing or not, I want to here from you, please do in this thread, I won't get mad: promise!  Tell if it's a good thing, if you rather liked the way it were before or if you'd like me to throw an exception when using position/x/y.

I've included new method to set the margin/marginAlignment and padding from a string value, you have to use the setMargin, setMarginAlignment and setPadding method, all taking a string which accept a format identical to the construction of the primitive.

[royibernthal]

I'm pro throwing an exception as you already know  I think what you did could end up being even more confusing and harder to debug than a soft warning message, especially in complex cases.

The new shortcuts sound good though.

[MasterK]

Yeah. Follow me is right...