Performance Problems Under "2D Graphics Editing Engine"

[yiheyang]

I am using Pixi.js to make a 2d graphics editing engine similar to Figma. The demand scenario is that users need to create up to 10k+ graphics in the Container and have free editing, grouping, and sorting functions. Besides, the visual area needs to be moved and zoomed.

For the various performance issues that might be involved in this scenario, I searched a lot of existing posts and learned the following optimization measures:

Quote

1. For the movement and zooming of the visible area, we achieve it by modifying the position and scale of the root Container. The current effect is good.

2. I have read "V4 Performance Tips".

3. Reasonably using cacheAsBitmap, Texture, Mipmap, and Texture Compression to reduce drawing overhead.

4. I understand the advantages and limitations of ParticleContainer and Pixi-TileMap.

5. I understand that Pixi.js does not automatically exclude objects in the non-visible area, but the shader does not calculate pixels outside the visible area (which means we can manually exclude invisible objects to speed up rendering).

6. I understand the basic usage of Web Worker and FrameBuffer.

I am optimistic about the performance of Pixi.js in this scenario. I believe that algorithm-level optimizations can solve most performance problems, but there are still problems remaining:

Quote

Q1: If there are a large number of sub-children will it greatly affect the performance when doing Container position and scale operations? (If possible, could you please tell me about the specific calculations and renderings performed?)

Q2: After I cacheAsBitmap on a large number of Sprites / Containers, why does the rendering performance decrease? How to optimize in this case? At present, I consider that I could group them as much as possible, wrap them with many Containers, and use cacheAsBitmap for caching. Is there a more suitable optimization?

Q3: In order to edit any graphics (implemented by Sprite), the graphics will be drawn only once after editing (implemented by Graphics) and their textures will be saved to Sprite. When the number of such non-repetitive graphics reaches 10k+, each Sprite is bound to a unique texture, and some of the Sprites must even be real-time (having filter, mask, and transform), in this case:

Q3.1: Can a large number of textures be cached in the GPU Memory to avoid repeated uploads?

Q3.2: How to optimize real-time rendering performance?

Q4: In a scene with a large number of graphics, it is impossible to draw views fully in real-time. How to elegantly implement progressive block rendering like Google Maps, when zooming in the visible area (Sketch and Figma both implement this)? Specifically: the visible area is divided into several small blocks of N x N for rendering. After zooming in, the small blocks in the visible area will be updated in batches (might going through "unclear, slightly clear, and completely clear" stages).

Q4.1: What scheme should be adopted to realize dividing the visible area into small blocks of N x N? Can I use Pixi-TileMap library?

Q4.2: How to update a small block asynchronously without affecting the moving and zooming smoothness of the visible area? (Does it mean that small block rendering should be performed in another WebGL context to prevent blocking rendering updates in the main context?)

Q4.3: How to upload shader data only once, repeatedly render different cropped areas, and repeatedly render different definition? (I don't want repeated data uploads for every rendering, it will be extremely slow!)

Finally, even if I am very satisfied with the performance, various out-of-the-box features, and powerful extendibility of Pixi.js, I still want to know, in the scenario of the 2d graphics editing engine:

Quote

Q5: Is the solution of using WebAssembly (such as Canvaskit based on Skia, and I don't know other more suitable solutions for now) a more sensible choice? If you understand the principles and mechanisms of Skia, can you briefly explain why it is more appropriate or inappropriate?

Q6: I have followed a post that mentioned the question "Is there any plan to migrate Pixi.js to WebAssembly in the future?" The official reply is that in most scenarios, JavaScript will not become the main reason for the performance bottleneck. But I am still very worried: Is there any potential dynamic code in Pixi.js? (It means this part of the code will not be optimized by JIT, which could affect the rendering performance greatly.)

I searched a lot of posts and information, but still have a lot of confusion above, I am very sorry for asking so many questions?! You can cite relevant documents or links to answer my questions as much as possible, and I will learn more about it, which can save you precious time!

If possible, I hope that my product will eventually become an excellent example of Pixi.js, and I have confidence in this!

I would like to thank the Pixi.js team for their continuous contribution to the WebGL field!

Edited January 3, 2021 by yiheyang

[yiheyang]

Sorry for having so many problems again?!

[ivan.popelyshev]

This is big. The least estimation for all that is a year.

Q1 We are working on new camera. Several people have custom implementations in their game , and if you look at issues made by https://github.com/pixijs/pixi.js/issues/created_by/dev7355608 - yeah, its required for DnD stuff. "renderer.render(... transform <-- this param)". 

Q2 and related to filters: its possible to cache pre-rendered, i use it in TheHouseHold,  but I didnt move it to pixi plugin yet. Hope to have it in 6 months or so

Q3. Sounds like shape-cache im doing in https://github.com/gameofbombs/pixi-blit . Beware, there are many bugs and only two examples. However its being used in https://www.facebook.com/playthehousehold/ which is biggest pixijs game at the moment, but i havent move some fixes from it back to pixi plugin.

Q4. pixi-tilemap is just a generated mesh, it doesnt even have high-level algo - user has to write it Even then, your textures are different which makes pixi-tilemap obsolete. It should be high-level algo, you described it correctly.

Q5: You can use pixijs with skia if you want better shapes. "Texture.from(skiaCanvas)" will help with it, just make sure you bind it so it gets uploaded to pixi context before you make another shape with skia. Lots of details about it.

[yiheyang]

So many thanks for replying.

I still confuse about the following point:

Q1: I implemented this by scale and position and it looks good. My doubt is if it's a more proper way to implement this.

Q4: To realize this, do i need to write native WebGL myself? Or is there an alternative way of implementation in Pixi.js? (even it will be a little bit slow)

Edited January 3, 2021 by yiheyang

[ivan.popelyshev]

Q1 The problem is that there's no proper way regarding the camera  we are developing it at the moment, the way it doesnt trigger re-calculation of all matrices in tree. Some other people did that before for pixi apps but it was hard.

Q4 If you dont zoom out too much, Sprites on low-level will be enough. In project of this scale, you have to learn low-level webgl anyway just to understand what pixi does and where you can optimize and where not.

[ivan.popelyshev]

For your knowledge - those kind of projects should raise millions or tens of millions of dollars on first round. Ask those guys: https://phase.com/ , there are several projects like this.

[yiheyang]

Thanks a lot.

I'm working on those concepts and understanding those low-level implementations!

At the moment, the main target is still on its prototype, and I need to know the tech boundaries as much as possible!

[yiheyang]

For Q1 & Q4, I found these two problems seem to be the same: only need to upload draw data once and the only difference between renderings is the transforming matrix.

I have gone through the code from Renderer.render(...) to the batch renderer's drawBatches().

It seems that I could try to solve the two problems by gl.drawElementsInstanced().

Is that a way out?

[yiheyang]

It seems that gl.drawElementsInstanced() cannot help on dynamic transforming matrix, and I'll try to make a shader to solve it!

[ivan.popelyshev]

I'm not aware of problems that can be solved with drawElementsInstanced in vanilla pixijs. I only know one case: many animated objects consisting of several quads with your own shaders. Others , well, its just to make buffer smaller, like i did in my experiment with candles&plot: https://github.com/gameofbombs/pixi-candles/ 

and even then i had to make a fallback in case user doesnt have instanced draw (on some mobiles): https://github.com/gameofbombs/pixi-candles/blob/master/src/Bars.ts#L203 , maybe its not the case for you because you are targetting PC's

Edited January 4, 2021 by ivan.popelyshev