Canvas2D Main Post

[Nockawa]

Hello good people of the babylon.js community!

Update #14 from January 29th, 2017

Primitive Collision Manager (PCM, @ller request) 

The PCM is finally done! Well, first release, I've already spotted some not supported cases... Anyway, you can check a PG demonstrating the feature here: http://babylonjs-playground.com/#1PXWLR#2 (I've commented the PG for you to see how it works)
Turned out that I didn't do what I wrote in the previous message, I didn't check for OOB and I've implemented a generic triangle set versus triangle set intersection algorithm right away.
Basically all primitive types now create/update a Tri2DArray which contains a list of triangles describing the primitives surface. It needed to be that way because an Ellipse with a intersection of 8 is graphically not shaped like a perfect ellipse: there's not enough polygons, so I couldn't use a intersection algorithm involving a true/perfect ellipse, same goes for the Rectangle2D with round corners. So I decided to go for the most generic and accurate way.
All primitive types are supported except the new WireFrame2D one.

Through the PCM object you have a list of all primitives intersecting with the Canvas' border (if you activate the feature), it's useful. You also have a list of all the pair of two colliding primitives.
From a given prim you have the intersectWithObservable and intersectWith properties to get notified of new/removed intersections and to access the full list of them.

New Text Alignment (vertical and horizontal) and word wrapping features for Text2D (coded by @adam, I'm very thankful to him! It's good to feel less alone on this whole coding...)

Basically now you can set a left/center/right text horizontal alignment and a top/center/bottom for the vertical one.
If you specify a size for your Text2D primitive you can now have auto word-wrapping if the line exceed the horizontal limit.

Maybe @adam could share a PG demonstrating the feature! 

New Primitive Type: WireFrame2D

Well, I needed it to draw the debug World AABB and Cluster of the PCM, so I wrote it. The Primitive accept many WireFrameGroup2D, each one having a default color and contain a list of lines. Each stored vertex has its own color.
Basically you create a WireFrameGroup2D instance then use its method to fill it with vertices. Note: it's Line List, not line strip: each line is defined by two own vertices. You can use an API to start, fill, end a LineStrip but internally it will double the vertices for everything between the first and last vertex.

Right now this Primitive doesn't support any kind of intersection (point or prim-prim), I'll add an intersectionThreshold and implement the algo any time soon...It's not a priority right now...

Misc things

• The BoundingInfo2D now support a WorldAABB with an efficient dirty/update, I obviously needed it for PCM
• Each Prim Instance has now a uid property which is a unique id (a GUID), I use it as a key to store the prim or prim related data into a StringDictionary, it's an efficient way to retrieve a given prim without going O(n).
• There's a Triangle versus Triangle intersection method in the Math2D.ts file, its implem is slow. If someone has the code for a Tri-Tri in 2D implemented using the Separating Axis Theorem (SAT) algo: I'm in !

Update #13 from January 24th, 2017

WorldSpaceCanvas TrackNode feature

So I've developed the feature requested by @royibernthal to create WorldSpaceCanvas that track a 3D Scene Node with the optional feature for the WSC to always face the camera (billboard mode). You can find a demo in this playground.
It was truly missing to the whole "WorlSpace Canvas interacting with 3D Scene" scenario. Other users requested more about this specific scenario and it perfectly makes sense: C2D was develop to provide 2D support for the 3D Scene, I've been focus on the "2D stuff" by itself a lot, because, well, it's already a big task, but yes there should be more things to come on this area, I'll try to dev feature requests on this matter with a higher priority.

Update #12 from January 11th, 2017

Few announcements:

• We can now use BMFont to render text with Text2D PG here the documentation of the Text2D was update to explain the different techniques you can use to render text (normal, Super Sample, Signed Distance Field, BMFont) and which one you should use. (Thanks @MasterK for the idea and help)
• There's a new page about how the Rendering is working, it will help people to understand how to get transparent sprites, for instance. (this page is currently only accessible by the link I gave before, because I've messed up the link in the Canvas2D Homepage, yeah, that's my special ability, DK loves it)
• Some new Playgrounds are available at the bottom of the Home Page
• I was ill during the release of the 2.5, so I couldn't update the doc and the what's new, so I've updated the what's new of the documentation web site, go take a look if you want. As a separate module, Canvas2D has its own section in the What's New.

Update #11 from January 5th, 2017

Long time that I didn't develop new features but here we go:

• AtlasPicture is now supported, see the documentation about it.
• Scale9Sprite is a new feature of Sprite2D class.The Sprite2D documentation was also updated.

You can play with the following PG:

• Scale9Sprite
• AtlasPicture

Of course you can use Scale9Sprite for a Sprite generated from an AtlasPicture.

Update #10 from October 21th, 2016

allow3DEventsBelowCanvas

Until now, pointerEvent occurring above a ScreenSpaceCanvas were also sent to the 3D Scene, many people (rightfully) complained about this so I decided to add the feature, which is called "allow3DEventsBelowCanvas" (which is false by default).

The PR is here for more info https://github.com/BabylonJS/Babylon.js/pull/1448 (preview files updated and PG not updated so far).

You have to realize that this is a change in behavior, now by default events won't be sent to the 3D Scene if they occurred above the Canvas, but I figured it made more sens this way. If you want them to be sent anyway, just set the setting to true!

Update #9 from October 19th, 2016

Stabilization

The latest weeks were spent on keeping stabilizing the Canvas2D feature, fixing bugs and adding feature only when necessary. Overall bug fixes were concerning Positioning Engine (alignment), StackPanel Layout Enigne, pointerEvent observable (leave/out now works better), some dispose issues solved, you're supposed to have no more opacity/transparency issues (well, let's say, "less").

One new feature

One new feature was developed recently, during the creation of a ScreenSpaceCanvas you can now specify the renderingPhase setting, allowing you specify for which camera and which renderingGroup the Canvas is supposed to be rendered to.
Simpler version: you can now interleaved 3D layers (which are renderingGroup) with 2D layers (which are ScreenSpaceCanvas). Many people requested it ( @MasterK, @HenryPeng, @stormwarestudios) to render 3D content above UI for FX (particles) or simply some 3D preview (like a 3D character preview in a UI). This playground  is a very raw demonstration of the feature.

Modularization

Oh, by the way, I nearly forgot to tell something pretty important: we have recently extracted the Canvas2D feature (and the forthcoming GUI Lib first code lines) from the main babylon.js file. This is the first step of what is going to be a modularization of babylon.js, Canvas2D/GUI being obvious (and easy) candidates.

What is changes for you?

Well, not that much, if you use C2D/GUI you now have to use the "babylon.canvas2d.d.ts", "babylon.canvas2d.max.js", "babylon.canvas2d.js" that are sitting next to the "babylon.js/d.ts/max.js" files in the "dist" directory of your choice (right now only the "preview release" directory contains the Canvas2D related files, for obvious reason. The module's name is still "BABYLON", which will ensure you'll compile without changing anything.

On a repository point of view, the source code was moved from /src/Canvas2D to /canvas2d/src/. Our lovely boss made the move and the Gulp config to compile everything the same way babylon.js does so you don't be lost.

La surprise du chef!

And the "chef" is not me, but another core member @Temechon spent some very valuable time (and I can't thank him enough for that!) to work on a feature I desperately need for so long but that I'm unable to code (well, in the time frame it will took me) which is called: c2dinspector.js.

Basically the c2dInspector is a HTML Pane that is added on the right side of your HTML pane and that displays all the Canvas2D instances in your Scene (well, Engine to be more accurate) and their content as a tree of Primitives.
You can select a given Primitive to see its details on the Detail Plane below, you can even change some values (still work in progress).
In the tree, for each item there're two icons at the very left, an "eye" to show/hide the prim, and a "double frame" which is used to show/hide the debugDisplayArea (more on this below).

This feature will help you (and me!) debug and understand better what's going on and why! It's not released yet, but a beta will be very soon (tonight or tomorrow), it's still Work in Progress, but hey, it's better than nothing and it won't harm you.

The debugDisplayArea (the only thing I coded, bugs are to expect!): as you may know a primitive has four area: layout, margin, padding and content, this feature will display them with a given color code (layout being light red, margin is yellow, padding is magenta, content is cyan). This screenshot below is the same UI as above but with the debugDisplayArea on the "GUI Visual Placeholder of MainWindow", which has a margin of 50, 50, 50, 50 (the yellow area shows it) and padding of 100, 100, 100, 100 (the magenta area show it). The content in blue is the Windows' background, in light grey, which is here covered by the blue area.

GUI Library

I've started the GUI Library, it's going on very well, but I just can't work as much as I wanted on it the last couple of weeks, so there's delay: that's life. There's a separated thread about the GUI here so don't expect me to say more in this post, got watch the other thread if you care about it.

That's all folk, enjoy, as usual: feedback are welcomed, you can do it in this thread of course! I just don't want this thread to be a "bug tracker" one, if you have issue, keep creating a new post and don't forget to mention me as I don't look at the forum everyday to see if someone needs help (but DK does, happy him...)

Update #8 from August 31, 2016

I pushed a new commit today that address many things around using Scale on the Canvas object directly. I'm lazy so I copy/paste the PR description:

• You can now set a Scale (or ScaleX/Y) to a Canvas2D object, it will render as expected, for every caching strategies.
• At creation time of a ScreenSpaceCanvas you can set a designSize which will be used to compute the Canvas' scale automatically in order for you to specify absolute coordinate in the frame of the designSize (MasterK request)
• Any Cached Renderable Group can have a new GroupCache Behavior: GROUPCACHEBEHAVIOR_NORESIZEONSCALE. When set the cached bitmap for the Group won't be resized when the scale of the group will change, improving performances over rendering quality
• Ellipse, Rectangle are now drawn with a prescale computation to ensure the best result when using high/low scaling
• Prim2DBase has not the actualScale and actualScaleX/Y property that return the accumulated scale of the primitive.
• At their creation, primitives can specify if they should not inherit from the scale of their parent through the dontInheritParentScale setting.

I don't know if many of you will get the GROUPCACHEBEHAVIOR_NORESIZEONSCALE and find a usage, but I thought it was the time or never to do it, so I did. And it works pretty well. I think for mobile gaming it may help to improve the perf by setting/keeping a fixed resolution of what's being rendered into a Cached Renderable Group2D.

The designSize mode was made upon a request from @MasterK and it's really a good idea, I can see the use case from a game UI point of view and I hope it will work the way it should be, otherwise I'll do bug fixing!

If there's no more urgent thing I'll go back on my working list and the next thing (that was already started but paused) is supporting the Device Pixel Ratio, which is another big thing for mobile gaming I think. I will be a breaking change for people who are right now "compensating the lack of this feature", but well, I'm sorry, there's nothing I can do about it except I wished I knew more about HTML few months ago!

I know most of you are still trying to understand how things work with the Canvas2D and don't have time to try and use different Caching Strategies (and then rely on the DONTCACHE mode). Truth is the other modes were pretty bugged, they are less bugged now, but I'm not sure it's working the same.

To solve this matter I'm revamping my whole "Unit Test" project, which was a local thing and not as powerful as it should be. There are so many different combination to test a given Use Case against: the different Caching Strategies, the support of the Instanced Array and with different Device Pixel Ratio values. I have to find time to put a good Unit Test framework and publish the whole stuff once it's done, because I think it will help all of you to discover the many things you can do and how.

When this will be done then I think I'll start the true "why I did this": a GUI Library based on Canvas2D, with the UI Controls you all are waiting for. The foundation of Canvas2D is getting more and more stable so I think it will be soon the right time to start it and it will be the perfect chance to fix dozen of bugs, I'm sure of it!

Update #7 from August 13th, 2016 @ Legoland, Billund, Denmark

The latest fix is a long ongoing one noticed by @FlashyGoblin concerning transparency of Sprite2d. To make a long story short: I mixed up the implementation of AlphaTest and AlphaBlend (aka Transparency), the result were...well ugly...and you couldn't do things like the one FlashyGloblin ran against. So here are the new ground rules about this:

• I try to behave the same way the 3D Engine StandartMaterial does, so people familiar with it will catch things up pretty quickly. One noticeable difference is that useAlphaFromTexture is true by default.
• There are three render modes: Opaque, Alpha Test and Transparency (Alpha Blend).
  • If you don't use Alpha in a Brush or a Texture and the Opacity is 1, the primitive is rendered in Opaque mode
  • If you use Alpha in a Brush, if you set BaseTexture.hasAlpha to true in a Texture your primitive uses or if the Opacity is not 1, then it will be either Alpha Test if useAlphaFromTexture is false or Transparency if it's true.

Beware: until now you didn't have to set hasAlpha for a Sprite2d to deal automatically with transparency, this is no longer the case! With the latest version your Sprite2d will be rendered as Opaque if BaseTexture.hasAlpha is set to false (which is the default value no matter what your bitmap stores).

To sum up

Text2d is set as a Transparent primitive to ensure a proper blending. Shape2d based primitives will be either opaque or transparent based on Opacity and Alpha in their Brushes. Sprite2d can use the three modes, the default behavior being Opaque if the texture has hasAlpha to false or Transparent if it's set to true and you can use useAlphaFromTexture to switch to Alpha Test.

Update #6 from 15 June 2016: babylon.js 2.4 is out!

 

I took the time to finish the overview/tutorial documentation.
I also updated the playground and use the BABYLON. prefix everywhere it was needed (which mean I removed the alias like "var Scene = BABYLON.Scene") in order for you the user to copy/paste more easily.

The documentation entry point is: http://doc.babylonjs.com/overviews/Canvas2D_Home (with all the playgrounds at the bottom of the page).
The architecture/design doc is: http://doc.babylonjs.com/overviews/Canvas2D_Overview_Architecture

Please, I've also created a PRO for the Canvas2D feature in Slant, for those who think it's a great addition to babylon.js, please follow this link and upvote, this work is on my spare time so fame is my only reward!   thanks!
http://www.slant.co/topics/3777/viewpoints/4/~open-source-javascript-3d-engines~babylon-js#15

Update #5 from 13 June 2016: new features added and also a lot of little bug fixes

This version is supposed to be the "first official release", there were breaking changes with the previous ones, most notably:

• You don't use the public static CreateXXX method to create a primitive, you can now use the constructor, the parent is an optional parameter in the settings object (see sample below).
• At construction you can use a settings object that contains many many properties that you can set as you wish, some are somewhat overlapping (e.g. there's size, width and height, you set either the first or the two latest). This was designed to bring you some flexibility.
• You won't certainly notice, but the location of a primitive will always be relative to its bottom/left corner, regardless of the origin you set. This change was necessary to bring something coherent and understandable when alignment/layout comes into play.
• You have to use the actualPosition and actualSize properties to get the "real" values. position/size properties are the value before layout/positioning takes place.

New features:

• This new creation mode allows you to cascade the creation of objects (see below).
• You'll see that Brushes can now be initialized with a string to make the code more simple. The same is true for margin/padding/alignment. You will find example in some playgrounds, doc will come asap.
• Lines2D now fully support intersection/picking, as well as roundedRectangle.
• The size of a primitive is now optional, if omitted it will be sized to its content bounding box. (the same behavior as Group2D)
• Margin, Padding, Alignment is introduced, you can play here. Margin/Padding support units in pixels or percentage, the value can be inherited from its parent or simply be automatic (0)
• A LayoutEngine feature now allows to position/size the primitive, by default it's the CanvasLayoutEngine that is assigned to all primitives, unchanging the behavior you already know.
  But I've made in few minutes a StackPanel Layout that you can test here (don't forget to CTRL-F5 your web browser to fetch the latest .js file)
• By the way, there's a new playground for Transparency testing.

Update #4 from 03 June 2016: new features added and also a lot of little bug fixes

• intersection works better on Lines2D, I now support every Cap types, it's also accurate on rounded Rectangle (if you click on the empty zone create by the rounding corner it won't generated a hit).
• interaction mode can now be enable in WorldSpace Canvas, you can take a look at this PG for an example: http://babylonjs-playground.com/#1BKDEO#9
• I've also added a alignToPixel property in Sprite2d to make sure the rendering of the sprite will be aligned to the render target device pixel, is ensure the best rendering quality, but in rare case of slow animation you would probably want to turn it off to ensure smooth animation. by default it's on.
• I won't detail it too much here because it need proper doc to be fully understood, but now you can give your own WorldSpaceNode to render a WorldSpace Canvas on something else that a simple Plane mesh. This mode is easy if you don't use Interaction, but if you do, you have to give a method that compute the world space intersection with your mesh and then compute the local position in the Canvas of the resulted intersection. This is not the simplest method to code but you have an example with the current implem for Plane World intersection to Canvas.

Update #3 from 25 May 2016: new features added

• Breaking changes in the API: concerning the way to create Canvas and Primitives, now it's using the same signature than other object in babylons.js, with an options{} json object for optional parameters. There are default value each time it's possible.
• Including many bug fixes and little features improvements.
• We have two new primitives: Ellipse2D and Lines2D. I worked hard to dev a nice Lines2D primitive and I hope you will like it!
• A full Overview Documentation is now online, the main entry page is here: http://doc.babylonjs.com/overviews/Canvas2D_Home, from this page you'll be able to access to every others pages of the overview doc and also all the playgrounds I've made so far.
• The reference documentation is now online for the 2.4 you will find documentation for about 80% of the Classes/public methods.

Update #2 from 19 May 2016: new features added

• Lots of little bug fixes, I hope everything will be ok with the origin property, nesting primitives, the text render is way much better now than before.
• Interaction is supported! There's an pointerEventObservable property in Prim2DBase for you to be notified about many things concerning interaction with a pointer (mouse, touch, pen). I've also added the support of the ActionManager, it's the same as Mesh and Sprite. I support all triggers exception Intersection and KeyUp/Down.
• Animation is supported too! You have the animations property in the base class of a Primitive for you to add animation the exact same way you would do for the scene objects.

Update #1 from 16 May 2016: added link in the Documentation section to a new tutorial that explains how to write your own primitive types

Original post

I am very pleased to announce you a new feature I've been working on for weeks now!

Well, to put things simply, Canvas2D is...a 100% WebGL 2D Engine!
It may sound strange at first but I really thought it was missing to this lib, all started from the time I realized I couldn't efficiently display text in babylon.js. So I started to work on a new feature to do so and one thing led to another and I realized we could use of a real 2D Engine for many things!

It's not related to the HTML Canvas Element at all, Canvas being a generic term (like Rectangle, for instance). I've called it Canvas2D because the main class controlling the feature is called Canvas2D and acts as its name suggest: you can draw 2D content inside.
For performance reason the rendering is 100% WebGL your graphic card loves to do these kinds of things if you spend the right amount of time to create a good rendering engine.

Developing such 2D Engine wasn't an easy task because I wanted to lay strong foundations to allow it grows decently through time. Future will tell if I did well or not!

At last, for a better understanding this features serves the same purpose than Pixi.js, but the intent were slightly different: Pixi.js relies on two rendering engines (HTML Canvas or WebGL Canvas), but the Canvas2D relies only on WebGL, because it's made to be used along with a 3D Scene, which requires WebGL anyway.

Primary focuses while designing the feature: fast, flexible and hopefully easy to contribute.

Concerning the "fast" if your device support the WebGL Instanced Array extension, you should be really pleased with the result, the engine was designed to work with this extension (even though if it also works without). I think I overall did well although if there's still room for improvement.

Basically, how does it work?

You can create many Canvas2D instance on a given Scene, you have two kinds of Canvas:

• ScreenSpace: the canvas is displayed above the 3D Scene, in 2D space, like you would expect See this example. But there's another interesting mode:
• WorldSpace:the canvas is displayed in a 2D Rectangle as a part of the 3D Scene, you setup its position, rotation and scale, like a Mesh (well, it's in fact a Mesh of a Plane). See this example.

When you want both Performance and Flexibility

Things get a little technical! When you create a Canvas you have to choose a Caching Strategy. Why? Because depending on what you will do with it you may use different rendering strategies.

1. CACHESTRATEGY_DONTCACHE: easy, nothing will be cached, all Group2D are called "logical" and the whole content is redrawn every render. If you have few things or if it always change, you may use this mode. It's also the most stable right now.
2. CACHESTRATEGY_TOPLEVELGROUPS: my favorite! Only Group2D that are direct childs of the Canvas will be cached and considered as RenderableGroup, their children Group will be logical only. If you have some On Screen GUI to display at the different corners of your Viewport, this mode may be the best for you.
3. CACHESTRATEGY_CANVAS: Well, the whole Canvas is put into a cache, you better avoid this if you create a Canvas with the size of the Viewport because a texture/depth buffer will be allocated with the size of the viewport. Note: this is actually the only mode that can be used with World Space Canvas. (the other at the exception of DONTCACHE will follow later).
4. CACHESTRATEGY_ALLGROUPS: this is the most technical one, sound brute force at first, but its the opposite: the most complex one. Potentially every groups are Cached, but in reality for each Group you create you can set a CacheBehavior which states if:
   1. The Group follows the cache strategy (then be cached),
   2. It won't be cached (then rendered every time) or
   3. If it's cached in the nearest Cached group parent.

This mode has been elaborate to allow the creation of GUI controls being part of a complex canvas with a lots of Depth and diversity. As for today, this is the least tested one, be indulgent.

A canvas is made of

• Primitive: this is the base type, has many property like position, rotation (along the Z axis), scale (uniform), visibility, an origin, zBias (if you're not happy with the default behavior), and a list of children primitives.
• Group2D: to create a new reference of frame, but also if you wish to cache its content, to avoid redraw at every render. If you have primitives that are most of the time static (or changing sporadically, in response of an event for instance) it'll be well advised to cache them to speed up the canvas rendering.
• Renderable Primitive: as its name suggest, it's a base class for primitive that render to the canvas, so far:
  • Text2D: to display text.
  • Sprite2D: to display a sprite (a part of a texture if you will).
  • Shape2D: another based class for a 2D Shape made of an optional Border and or Fill content.
    • The Border and Fill Content are drawn using a specific type of Brush, for now SolidColorBrush2D and GradientBrush2D are supported. TileBrush2D (so based on a bitmap pattern) should make its way in the 2.5.
    • So far only the Rectangle is implemented, but it does support roundRadius, which is nice. I think I can implement Ellipsis and Circle in no time, but well, I wanted to focus on the engine first. nPolygon, WireFrame should come later. Vectored based graphics (SVG support) should be a nice addition too.

Documentation

The tutorial can be found here.

An overview of the feature architecture and capabilities can be found here. That's where you'll understand everything you can do and what you can expect in term of performance.

A long tutorial to explain how to write your own primitive type can be found here.

Few last things

My mantra for this feature: by indulgent with the current state of the Canvas2D, but be fearless on the feedback side!

I will try my best to fix things as much as possible, I've made already some testing, but nothing will replace the impartiality of the user!

I also have some bugs already logged (like don't try to change the origin of a Text2D, the result will be rather funny...and unpleasant) and will fix them asap.

Critical features are missing: Group2D and Text2d horiz/vert alignment, more shapes. As well as interaction support through an event system and a better support for animation. I will work on them right away. I don't make promise of what will be there for the 2.4 RTM but I expect everything except animation improvement, really hoping that interaction will make it (edit: also done).

A big thanks to the core contributors of the lib for their help: @Deltakosh @RaananW @Temechon @jerome and @Sebavan!

I'm waiting for your feedback!

 

[GameMonetize]

It's live

This is a fantastic work!!! Congrats!

[jerome]

HUGISSSIMO !!!!  

 

CHAMPAAAAAAAAGNE ! 

 

and fantastic doc + tuto ... I'm currently reading them

 

 

mmmhh.. beat this, 3js ;-)

[adam]

Incredible

[Dal]

Woo! Something I needed for my ideas

[Samuel Girardin]

Hi , 

Is there  mouseEvent on ?

 

sam

 

[Nockawa]

@Samuel Girardin

I'm working on it right now! I've started yesterday and it's the next feature coming. I hope to commit it in the next couples of days max.

[jerome]

I think this guy leaves in a dimension where there are 37485 hours per day 

[Samuel Girardin]

@Nockawa thanks ! Time to say good bye  to my  export class CustomDynamicTexture extends DynamicTexture !  

[Nockawa]

@Samuel Girardin you're welcome, I'll work on the text horiz/vert alignment soon, should be done in few (<5) days.

Give me feedback concerning features you'd like for the text, I think it's a very important feature and new ideas of improvements will be carefully studied!

 

[Dal]

@Nockawa Stretch goal - can you somehow render HTML to your canvas and pass the mouse inputs back to the DOM?    

[Nockawa]

5 minutes ago, Dal said:

@Nockawa Stretch goal - can you somehow render HTML to your canvas and pass the mouse inputs back to the DOM?    

Hu...what !?

Can you explain a little bit more what you meant and the intent?

Thanks

[Dal]

Just now, Nockawa said:

Hu...what !?

Can you explain a little bit more what you meant and the intent?

Thanks

Well... I am thinking that in theory, it should be possible to render HTML elements like textboxes and buttons to the 2D canvas and then place it in worldspace... but of course it would need some tricky mouse input handling to allow the buttons to be clicked etc.

[Nockawa]

1 minute ago, Dal said:

Well... I am thinking that in theory, it should be possible to render HTML elements like textboxes and buttons to the 2D canvas and then place it in worldspace... but of course it would need some tricky mouse input handling to allow the buttons to be clicked etc.

The whole goal of this feature is about: not having to do what you say, but still somehow get something that reach the same intent!

Let me explain myself: I don't want/can't rely on HTML to render GUI elements, it would be great in theory because you'd have a wide varieties of GUI stuffs (the whole HTML!) but performances and compatibility issues would kill you instantaneously!
But still, it would be great to have TextEdit, ComboBoxes, Buttons as part of the Render Screen or directly inside the 3D Scene, right? With User interaction possible of course!

That's the final purpose of this feature. Let me rephrase: this feature lays down the foundation to achieve this goal: writing a great/efficient GUI Controls Library for your creations, without having to rely on HTML.
I'm (well, we, because I hope some people will soon contribute) not going to rewrite something as powerful than the HTML GUI, but I'm sure we will succeed to develop a GUI Library that will allow you to do many things. I'm pretty confident because I've already done it in the past and I've never realized a 2D Rendering Engine that optimized, so I'm sure we'll succeed!

What I'm working on finishing right now is the feature itself, that's the first step. When it's reached I'll start the GUI Library.

By the way, I've wrote a long tutorial to explain how to write your own primitive type: you'll find it here

[jerome]

This is just magic : each time someone asks something, @Nockawa answers with a full feature, or even a complete lib, or a highly detailed documentation  

[NasimiAsl]

Great Job   thanks for share @Nockawa

[Wingnut]

Nice job, Noxy!  yay!  I haven't read the docs, yet, but... assuming that each of these overlays is a canvas (perhaps not true), is it possible to:

1.  Set ANY size dimensions and position on these overlays?  Users sometimes want them ONLY covering bottom 1/4 of renderCanvas, and sometimes not a full-screen renderCanvas.

2.  Append these aux canvi (2D overlays) to ANY html element... including the renderZone element of the playground?  How about the lower 1/4 of the renderZone element?

3.  Easily set the transparency of all items on a 2D overlay... with a single property?

4.  Use multiple over-lays, and easily adjust the z-indices or layer number... of those overlays?

5.  Have clicks be ignored by front-layer overlay/items, yet still be detected on back-layer items?  isPickable?

6.  Use the actionManager when/if possible?

7.  Use CSS on all 2D overlays... for borders?

8.  Use CSS borders for "items" included on those overlays?  Are the items (widgets) each, themselves, a canvas element?  Probably not, eh?

9.  Use html IMAGE elements as members of an overlay?

Ok, that's plenty of dumb Wingnut questions for now.  Congrats again, Nockawa... nice work.

 

[Nockawa]

@Wingnut I suggest you read the doc first, I've spent some time trying to explain as much as possible.

The Canvas2D feature is not based on HTML at all. I understand the name may misleading people, but it's a 100% Custom WebGL based feature to render 2D Content.

However, it's made to serve a purpose close to what you have in mind, I think. It's just the way to do it is different.

1. is possible. 2., 7., 8. are not (because it's not HTML based). 3, 4 already made. 5, I'm working on interaction right now and yes this feature is already supported. 6. I don't know much about the actionManager and I'm ashamed to be, but I know its core purpose and I think after I've developed the core interaction feature, I'll take a look at the actionManager to see if I can wire it to the Canvas2D. 9. you have to use the Sprite2D primitive to render an image.

[NasimiAsl]

for a last question  

we can Have Canvas2D and WebGl in One Canvas?

 

[jerome]

yes... What @Nockawa calls Canvas2D is a BJS feature. It works inside the html canvas element in the WebGL context, like any other BJS feature.

The name seems confusing because people are used to call the HMTL canvas element also "canvas 2D" when used the "2d" context.

The Canvas2D feature has nothing to do with HTML, neither with the canvas HTML element : it's a full BJS feature that aim to provide some 2D drawing and writting primitives on a 2D zone (called canvas2D as a reminder... and this name is the only relation with the HTML canvas element) either in the WebGL screen space, either directly in the 3D scene.

Actually, you need to imagine the Canvas2D feature as a re-implementation (although different) of the HTML canvas in its 2d context WITHIN the BJS scene or screen. What can you do in a HTML canvas in 2d context ? you can basically draw and fill primitive shapes (rectangles, arcs, lines) and put some text from some CSS settings. You can translate or rotate some states. Well, the @Nockawa's feature provides slightly the same to a new BJS object... and we can add more and more primitives with our own code.

 So it's better because it's extensible and it's WebGL, so far more faster than the genuine HTML canvas element in 2d context.

I personaly find his work amazing because he added a real 2D engine within a 3D engine, both in genuine WebGL.

I guess he is experimented for decades, like DK, in programming 3D professional softwares and he already did this kind of 2D features in other 3D softs, but anyway, it's huge to have it (and him) in BJS so fast after he started to contribute some weeks ago only.  

[NasimiAsl]

 	var canvas2 = BABYLON.Canvas2D.CreateScreenSpace(scene, "ScreenCanvas", 
		new BABYLON.Vector2(0, 0), 
		new BABYLON.Size(300, 100), 
		BABYLON.Canvas2D.CACHESTRATEGY_DONTCACHE);
 

	canvas2.backgroundFill = BABYLON.Canvas2D.GetSolidColorBrushFromHex("#C0C0C040");
	canvas2.backgroundRoundRadius = 50;
	
	var text = BABYLON.Text2D.Create(canvas2, "text", 80, 40, "20pt Arial", "Hello World!", new BABYLON.Color4(0.6, 1.0, 0.6, 1.0));	
	

what is in CreateScreenSpace happend 

is that make DynamicTexture and Use POSTPROCESS?

 

[NasimiAsl]

http://babylonjs-playground.com/#2AVSFH#1

you know \n work on Text2D  but need revers

[NasimiAsl]

and Can i change text after first drawing

like 

var text = BABYLON.Text2D.Create(canvas2, "text", 10, 10, "20pt Arial", "World\hHello", new BABYLON.Color4(0., 0., 0., 1.0)); 

text._text = "33 55"   
    

[Nockawa]

2 hours ago, NasimiAsl said:

 	var canvas2 = BABYLON.Canvas2D.CreateScreenSpace(scene, "ScreenCanvas", 
		new BABYLON.Vector2(0, 0), 
		new BABYLON.Size(300, 100), 
		BABYLON.Canvas2D.CACHESTRATEGY_DONTCACHE);
 

	canvas2.backgroundFill = BABYLON.Canvas2D.GetSolidColorBrushFromHex("#C0C0C040");
	canvas2.backgroundRoundRadius = 50;
	
	var text = BABYLON.Text2D.Create(canvas2, "text", 80, 40, "20pt Arial", "Hello World!", new BABYLON.Color4(0.6, 1.0, 0.6, 1.0));	
	

what is in CreateScreenSpace happend 

is that make DynamicTexture and Use POSTPROCESS?

 

When you create a ScreenSpace canvas, you specify a Cache Strategy, in the example quoted it is: CACHESTRATEGY_DONTCACHE. Which means I will draw "over" the 3D Scene, using a fresh Depth Buffer to avoid rendering issues and make sure the 2D content is always above the 3D content. Basically a Text2D drawn with this strategy is about two triangles being rendered by letter in the Text2D, directly on screen (activate the debug layer and switch to wireframe, you will see).

If you use other cache strategies like CACHESTRATEGY_CANVAS, TOPLEVELGROUPS or ALLGROUPS, the rendering of the Canvas will be made in a dynamic texture. The simplest example is the CACHESTRATEGY_CANVAS: I create a texture of the size of the canvas, render the whole content of the canvas in this texture, then when it's time to render the canvas in the 3D Viewport, I display the texture content using a Sprite2D, its basically what you said: I render to a dynamic texture, and use some kind of post process to render it in the Viewport. But the actual implementation is somewhat different, but you nailed the intent.

[Nockawa]

2 hours ago, NasimiAsl said:

and Can i change text after first drawing

like 

var text = BABYLON.Text2D.Create(canvas2, "text", 10, 10, "20pt Arial", "World\hHello", new BABYLON.Color4(0., 0., 0., 1.0)); 

text._text = "33 55"   
    

You have to use the Text2D.text property to change the text's content. Text2D._text is private, you're not supposed to interact with it.

2 hours ago, NasimiAsl said:

http://babylonjs-playground.com/#2AVSFH#1

you know \n work on Text2D  but need revers

Sorry, I don't understand, can you be more explicit?