Canvas2D Main Post

[Nockawa]

2 minutes ago, Samuel Girardin said:

@nockawa

Maybe this is a bug. PG here : http://babylonjs-playground.com/#UVDG0#47 (tested in edge,ff and chr)

The 2nd button has an alpha < 1.

After a second clic on the 1st button, even if levelVisible is set to true, I can see the text but not the 2nd button's background.

Is it specific on my computer or not ?  

Buuuuuuug, let's the hunt begins! I'll keep you posted! Thanks reporting this! (I have to admit, I needed a little break concerning transparency after struggling so much, so I didn't test it thoroughly, my bad).

[Nockawa]

22 minutes ago, Samuel Girardin said:

@nockawa

Maybe this is a bug. PG here : http://babylonjs-playground.com/#UVDG0#47 (tested in edge,ff and chr)

The 2nd button has an alpha < 1.

After a second clic on the 1st button, even if levelVisible is set to true, I can see the text but not the 2nd button's background.

Is it specific on my computer or not ?  

Bug fixed! You will have to wait for @Deltakosh to merge/update the Playground, I will let you know when it's done!

Thank you very much, keep testing!

I'm working on Layout/Alignment engine, it's gonna roxx!

[Samuel Girardin]

Thx ! I ve just seen your pr on github. Will grab the 2 files. I always working with full ts source when I'm working on a project.

edit : the fix is ok with my project.

[jerome]

same here

 

aaarff, you fix bugs before we can confirm them !

[Nockawa]

Hello everybody, a little morning update.

Alignment, Margin, Padding is progressing nicely, I've something that works, but as usual I will need to push the use cases to make sure it's working on a broader spectrum.
I've introduced the notion of a LayoutEngine that you can attach to any Primitive, if you don't the CanvasLayoutEngine will be used and its behavior is similar to what you currently experience.

Now, let's move to less pleasant news...

There will be breaking changes in the upcoming version...Three reasons for that:

1. I can still allow to make breaking change because it's not release yet and I won't have the same luxury in one week...
2. The Alignment feature introduced a small breaking change that you shouldn't be impacted, I've introduced the notion of actualSize which is computed from the size property + alignment engine. So now when you create stuff you still use size, but if you want to get the "real, on viewport size" of your primitive: actualSize is the property to use.
3. A big breaking change is necessary to make the Canvas2D feature really great, let's explain it a little more.

Creation of a Canvas and its content

Due to my poor skills in TS/JS, I mislead myself and used a pattern I should have avoided for creating the primitives. I'm talking about the "Rectangle2D.Create()" stuff, a static method in the class to create your instance.
While I initially thought it would bring a possible diversity of different CreateXXX methods for a given type, I later realized that it's not the way to go and it prevents us to use a more declarative approach, rather than fully procedural as it is of today.

Ok, let's give a concrete example, it will be simpler to understand. Here's what I want to achieve:

var c = new ScreenSpaceCanvas
(
    {
        width: 100,
        height: 10,
        children:
        [
            new Rectangle
            (
                {
                    width: 20,
                    children:
                    [
                        new Text
                        (
                            {
                                text: "Hello World" 
                            }
                        )
                    ]
                }
            )
        ]
    }
);

Note: this example is just an example, it does not reflect the final API, because I still have to write it!

This is kinda following the JSON principal, but it's still TypeScript! Why? because a plain JSON object would have leave you with no documentation, no intellisense, no strict type checking at compilation: well, JavaScript for you..

The purpose of this change is for you to declare a whole Canvas and its content using a cascading approach, like it is when you design it in your mind, I think it's more natural, easier to visualize.

The pros:

• It's fully typed, so you have documentation while you type your stuff, with intellisense, the compilation ensure you've done something that has a great chance to come nicely on screen at run-time.
• You can take advantage of the maaaaaany optional properties that all have a default value, so it's supposed to be very low on verbosity 80% of the time (when you use the default behavior) but you will still be able to address the remaining 20% by specifying any possible property you want.
• You can still use the procedural approach and create your Canvas, then add children to it: it's the way you want, that suits you the best.
• You can still go for plain JS code, if you like.

The cons:

• Honestly I don't see one so far, but you will certainly find some, give me your feedback: that's what I need.

It's a considerable amount of work for me to refactor all of this. But I think it's the right time to do so. Developing the alignment/margin/padding/layout made me realized there's ton of parameters and this declaration would be easier to address such diversity.

This dev leads me to a soon to be new feature, because it's the next step anyone could have in mind: serialization. For this I will get in touch with @Deltakosh to get a clearer understanding of the current system in babylon.js and it's going to be interested to be able to serialized/deserialize a full Canvas.

Feedback is welcomed! And I'm already sorry for those would already took time to play/create stuff, because you'll have to refactor your code too...

[jerome]

greatissimo ! 

[FlashyGoblin]

I love the JSON approach! It will allow devs to easily update UI outside of compiling their site. Just tweak a JSON file, and BOOM, a new new UI.

And I think its good that you are still allowing procedural scripting as well. 

[FlashyGoblin]

@Nockawa Is there currently a way to load in a single image into Canvas2D? I'm only seeing it possible through using Sprite2D, which seems overkill for a single framed image. 

[Nockawa]

1 minute ago, FlashyGoblin said:

@Nockawa Is there currently a way to load in a single image into Canvas2D? I'm only seeing it possible through using Sprite2D, which seems overkill for a single framed image. 

Sprite2D is the way to go, there would be no difference with an Image based type everything is computed in the Shader, there's absolutely no perf hit of using this type.

[jerome]

What would be the best choice to display one single billboarded image in the world space between the Canvas2D sprite and the legagy BJS sprite ?

What is to be taken in consideration to prefer one or the other according to you ?

[FlashyGoblin]

2 hours ago, jerome said:

What would be the best choice to display one single billboarded image in the world space between the Canvas2D sprite and the legagy BJS sprite ?

What is to be taken in consideration to prefer one or the other according to you ?

Great question! I'm also trying to choose between which one to use. The canvas2D would allow for greater flexibility down the road, for example, when the UI designs change last minute and you need to add an additional element. You'd already have a canvas2D there for you to easily draw into. I would hope that the performance is the same, but only @Nockawa can say.

[FlashyGoblin]

Question about raycasting...

Is there any way to raycast into a Canvas2D element and determine what elements it is over? 

For example, this code casts a ray out from the center of the camera:

var ray = new BABYLON.Ray(camera.position, camera.getTarget().subtract(camera.position));

var pickInfo = scene.pickWithRay(ray, function (mesh) {
    console.debug(mesh);
});

The mesh parameter would return the Canvas2D's worldSpaceCanvasNode property, and the pickInfo variable would return all the information on where the ray hit it. So with these two items, is there a way to return the item within the canvas that was hit?

[Nockawa]

1 hour ago, FlashyGoblin said:

Great question! I'm also trying to choose between which one to use. The canvas2D would allow for greater flexibility down the road, for example, when the UI designs change last minute and you need to add an additional element. You'd already have a canvas2D there for you to easily draw into. I would hope that the performance is the same, but only @Nockawa can say.

Well, I think these are two separate purposes, the Sprite Engine of BJS is really made to display many instances of the same sprite in an efficient manner (I don't think it support Instanced Array but this could be an easy evol to make and the perf would be unmatched).

On the other hand I don't see the Canvas to be used to display many times the same instance at different places. You can do it, the perf could be better than the Sprite if you use InstancedMesh, but I don't see a use case, so it's up to you guys if you have one.

 

[Nockawa]

58 minutes ago, FlashyGoblin said:

Question about raycasting...

Is there any way to raycast into a Canvas2D element and determine what elements it is over? 

For example, this code casts a ray out from the center of the camera:

var ray = new BABYLON.Ray(camera.position, camera.getTarget().subtract(camera.position));

var pickInfo = scene.pickWithRay(ray, function (mesh) {
    console.debug(mesh);
});

The mesh parameter would return the Canvas2D's worldSpaceCanvasNode property, and the pickInfo variable would return all the information on where the ray hit it. So with these two items, is there a way to return the item within the canvas that was hit?

Yes of course, the interaction mode is not supported in WorldSpaceCanvas, so not only you have what you describe but it's on a Primitive Level too !

You can see an example here: http://babylonjs-playground.com/#1BKDEO#9

Just click on the button (or around) and you'll see it's supported the same way than a ScreenSpace one...The way to use it is exactly the same. And I think it's nice.

[FlashyGoblin]

31 minutes ago, Nockawa said:

Yes of course, the interaction mode is not supported in WorldSpaceCanvas, so not only you have what you describe but it's on a Primitive Level too !

You can see an example here: http://babylonjs-playground.com/#1BKDEO#9

Just click on the button (or around) and you'll see it's supported the same way than a ScreenSpace one...The way to use it is exactly the same. And I think it's nice.

Yes that's awesome, but that is using a click. I meant using a ray cast to determine the click event. For instance, I'm developing a VR game where there is no click in mobile VR (Google Cardboard). So I have implemented a reticle system using ray casting, where you align the onscreen reticle at the item you want to interact with. 

[Nockawa]

36 minutes ago, FlashyGoblin said:

Yes that's awesome, but that is using a click. I meant using a ray cast to determine the click event. For instance, I'm developing a VR game where there is no click in mobile VR (Google Cardboard). So I have implemented a reticle system using ray casting, where you align the onscreen reticle at the item you want to interact with. 

Ok I understand better, indeed you're right the actual method that process pointerEvent is private right now. But I was thinking there would be no other need. If you want me to make this method public I can, it's not the hardest thing to do!

The method is called `_handlePointerEventForInteraction`

 

[Samuel Girardin]

@Nockawa, 

Just notice if you forget to set  a backgroundFill for  a 'CreateScreenSpace', z ordering (note sure that exactly that)  of your  'normal' 3d objects is weird.

http://babylonjs-playground.com/#UVDG0#65

[Nockawa]

47 minutes ago, Samuel Girardin said:

@Nockawa, 

Just notice if you forget to set  a backgroundFill for  a 'CreateScreenSpace', z ordering (note sure that exactly that)  of your  'normal' 3d objects is weird.

http://babylonjs-playground.com/#UVDG0#65

Ok, looks like I don't restore a proper Z Compare/Write...I fix that right away, thanks!

Edit: yep, I don't restore depthWrite...that's why, it's fixed and will be in the next PR, today I hope...The big bang is coming...

[Nockawa]

Hello people,

It's been a while since I didn't give you an update. I couldn't spend time on coding as much as I wanted this week and I got myself into one last major refactoring which was really necessary for a V1 of the feature.

It was exhausting but I'm almost there, the refactoring is done, now I have to close the alignment/margin/layout features. You will see some breaking changes in the declaration (very light actually, you can adapt your code very quickly) but also in the behavior of the Primitives' positioning. I tried very hard to get something that would please most use cases while staying simple to learn/use...Integrating a feature that changes the size/position of the Primitive really did impact a lot of things in my code...

I should have foreseen this from the start... I hope to make a release by the end of the week end.

[Nockawa]

Hello everyone !

The latest version of Canvas2D was integrated few minutes ago, I've updated the playgrounds, once the documentation will be merge/rebuilt you will have access to their new versions.

This version is supposed to be the "first official release", there were breaking changes with the previous ones, most notably:

• You don't use the public static CreateXXX method to create a primitive, you can now use the constructor, the parent is an optional parameter in the settings object (see sample below).
• At construction you can use a settings object that contains many many properties that you can set as you wish, some are somewhat overlapping (e.g. there's size, width and height, you set either the first or the two latest). This was designed to bring you some flexibility.
• You won't certainly notice, but the location of a primitive will always be relative to its bottom/left corner, regardless of the origin you set. This change was necessary to bring something coherent and understandable when alignment/layout comes into play.
• You have to use the actualPosition and actualSize properties to get the "real" values. position/size properties are the value before layout/positioning takes place.

New features:

• This new creation mode allows you to cascade the creation of objects (see below).
• You'll see that Brushes can now be initialized with a string to make the code more simple. The same is true for margin/padding/alignment. You will find example in some playgrounds, doc will come asap.
• Lines2D now fully support intersection/picking, as well as roundedRectangle.
• The size of a primitive is now optional, if omitted it will be sized to its content bounding box. (the same behavior as Group2D)
• Margin, Padding, Alignment is introduced, you can play here. Margin/Padding support units in pixels or percentage, the value can be inherited from its parent or simply be automatic (0)
• A LayoutEngine feature now allows to position/size the primitive, by default it's the CanvasLayoutEngine that is assigned to all primitives, unchanging the behavior you already know.
  But I've made in few minutes a StackPanel Layout that you can test here (don't forget to CTRL-F5 your web browser to fetch the latest .js file)
• By the way, there's a new playground for Transparency testing.

A bit of code:

var rect = new Rectangle2D
(
    {
        parent: canvas, x: 200, y: 200, width: 100, height: 100,
        fill: "#808080FF", border: "#FFFFFFFF",
        borderThickness: 4, roundRadius: 5, children:
        [
            new Ellipse2D({id:"RedDot", x:45, y:45, width: 10, height: 10, fill: "#C04040FF" }),
            new Text2D("Right", { x: 150, y: 0, fontName: "Bold 12pt Arial"}),
            new Text2D("Left", { x: -150, y: 0, fontName: "Bold 12pt Arial" }),
            new Text2D("Top", { x: 0, y: 150, fontName: "Bold 12pt Arial" }),
            new Text2D("Bottom", { x: 0, y: -150, fontName: "Bold 12pt Arial" }),
        ]
    }
);

You can see how simple/easy it is to create a graph of primitive. In this case you have a Rectangle that owns an Ellipse and 4 Text. The brushes are initialized with string directly, you could have use the "old" way also, both types are supported.

What's next?

I would like to go and try for a GridLayout control before the release of the 2.4. It won't be long to code a simple version I guess and it would be useful I guess... more complex layout could come later.

As usual, thank for all the people who test, give feedback and help!

[jerome]

Fantastic and really smart 

I suppose that, in your snippet, we have to prefix all the class names with "BABYLON.' when using it in vanilla JS or the  JSON parser does care of this ?

[adam]

This is f***ing awesome!!

[jerome]

new regex : * matches [uck]

[Nockawa]

6 hours ago, jerome said:

Fantastic and really smart 

I suppose that, in your snippet, we have to prefix all the class names with "BABYLON.' when using it in vanilla JS or the  JSON parser does care of this ?

Yes, you have to prefix with 'BABYLON.' I'm not doing it because I declared aliases at the start of the code, like 'var Group2D = BABYLON.Group2D'.

[jerome]

ok,

my question, more precisely, was : will this way (to declare for instance var Text2D = BABYLON.Text2D) also work for the objects declared in ths JSON-like array passed as a parameter ?

I suppose yes according to your former answer.