PIXI, Suggest optimizations to minimize CPU/GPU

[Karthik]

Hello Gurus,

I'm working on a project where I have 400,000 static rectangles (5px*5px)  and a 20 rectangles that get added dynamically every second. At any given point in the frame, I only need to keep 3000 rectangles in the viewpoint. I'm using single graphics object and clear() method to render 3000 rectangles every sec (draw + clear). I have pan/zoom functionality added as well, and I'm using downsampling to remove overlapping pixels and reduce the number of points that I draw at any given point (to around 5000 rectangles).

It works fine but the CPU/GPU usage is around 15%-23%. I'm looking for some suggestions to optimize CPU/GPU and  memory usage and reduce the load on CPU/GPU. I set antialias:false on the app. It seems to help a bit. Any further suggestions to improve performance would be greatly helpful. Thank you.

 

Here are couple of things that I tried, but no luck.

1)Instead of draw+clear, I created sprites using graphics and moved them around. That did not help either.

2)I cannot avoid clear and redraw because I have 400,000 points to draw. So I cannot draw all at once and have to downsample. Also, dependeing on the view window, I need to rescale the rectangles, which is extremely difficult to achieve without repaint.

 

[ivan.popelyshev]

400k  is too slow to draw with any geometry, whether its Graphics or Sprites. maybe pixi-tilemap plugin can handle that but im not sure.

You need RenderTexture's, organize them in chunks, redraw only those that were changed. In case of zoom you can make a queue - which renderTextures has to be re-drawn with better resolution.

[Karthik]

Thanks for the reply Ivan. I see  your response in all the other posts and you don't know how much you have helped  me without you ever knowing it. I want to first thank you for all the time you spent sharing your knowledge in this forum.

Let me elaborate a bit so that I explain my issue more clearly. I'm trying to plot time-series data that is captured every second upto a history of 1 day. That equates to 86400 points (roughly 100k) and 4 properties each second, so 400k points. Now, by default I show the last 1000 points, and the current information that comes every second. So the  frame is constantly moving to the right showing the current data. When the user zooms or pans I have to show historical data (that is static) by squeezing it and fitting it in the frame. So the overall data looks like a long horizontally elongated rectangle (image : https://ibb.co/7gw1mPb);

• So 99% of my data is static. 
• 1% is dynamic data that gets captured second.

So if there is a way to add all my points to a container that takes all the data, and later use some transformation to rescale and display, that would be the best.

Also, I wouldn't mind paying for hourly rate for consultation  to provide some technical guidance on how to approach this problem.

Thanks again.

[ivan.popelyshev]

Thank you!

You dont need to show 400k points on the same screen because screen is maximum 4k , right?

You certainly need chunks. Use transform (position) to scroll things. Store 200-500 points in a single graphics, update only last one, destroy or just make "renderable=false" those who dont appear in camera right now. For zoom use different set of graphics - every 2nd point, every 4th poiint, every 8th and so on, in other words spawn log2 strips , level of detail. LoD also helps the quality, because 100 vertex per pixel is really an overkill for geometry ?

The other issue is Graphics lineWidth - if you want it to be adjustable through zoom, you change the style internally "graphics.geometry.graphicsData.lineStyle" and call "geometry.invalidate()" on all visible graphics elements.

Dont use drawCircle() for points when you have many points - geometry will have colossal size. Use lines or rectangles.

 

[Karthik]

My response in red

You certainly need chunks. Use transform (position) to scroll things. Store 200-500 points in a single graphics, update only last one, destroy or just make "renderable=false" those who dont appear in camera right now. For zoom use different set of graphics - every 2nd point, every 4th poiint, every 8th and so on, in other words spawn log2 strips , level of detail. LoD also helps the quality, because 100 vertex per pixel is really an overkill for geometry ? . I am doing the same thing (pick every ith point depending on window size, instead of showing it all) to reduce points. That being said, I'm having to do this real time (as I zoom in and out) so have to clear and repaint every frame depending on window size & total points in frame. May be if there is a way todo this without clear and repaint, it might solve my issue. If you could point me to some examples that do what you explained, that can really help. Thanks again.

The other issue is Graphics lineWidth - if you want it to be adjustable through zoom, you change the style internally "graphics.geometry.graphicsData.lineStyle" and call "geometry.invalidate()" on all visible graphics elements. The line thickness is constant.

Dont use drawCircle() for points when you have many points - geometry will have colossal size. Use lines or rectangles. I am using rectangles, and lines. No circles.

[ivan.popelyshev]

Is line thckness constant relative to stage (zoom) or screen? You dont need to clear() and repaint every time if you just adjust position and scale. clear&refill is needed only for chunks that weren't visible in previous frame, or the one that is updated (new data).

UPD: I love that kind of threads! When you make the main algorithm, while pixijs or any other renderer or framework is just your building blocks

[Karthik]

The line thickness is always constant (3pixels) in relative to the screen. The line length however is varying as I have to resize it to the window size. i shared a picture below that shows exactly how it looks.

https://ibb.co/Gkhzw7G

I'm not clear on how to avoid repaint with just reposition and scale, without a way to hide/show dormant rectangles.

 

[ivan.popelyshev]

OK, if its relative to screen then we're fucked. Try change lineStyle internally and call "geometry.invalidate()". https://www.pixiplayground.com/#/edit/3AKL1vurX38ct_sY~43Ap

That eats a bit less CPU but its still a re-upload. I know that there's a plugin in development that allows to change lineWidth relative to screen entirely on GPU, but cant give you code yet.

[ivan.popelyshev]

Oh, I forgot to welcome you to the forums!

I'm glad another "hi im new here but i need to do this hard task"  went well. Usually its 50/50 whether the person is good enough to solve hard problems straight from first post.

 

[mobileben]

Why don't you send a few examples of what a graph looks like at different scale so we can see the spectrum of visuals you are trying to achieve and how they interrelate?

[Karthik]

I did share a screenshot, here it is incase you missed it.

https://ibb.co/Gkhzw7G

[mobileben]

Ahh, okay, I saw that. I misunderstood. You talked about zooming/scaling so wanted to see how it would differ. Essentially the rough views are the same (rectangles and lines)?

[Karthik]

That is correct. All I have is rectangles and lines. I figured out the zooming part as well, the big issue is that the CPU/GPU usage is very high when I clear and repaint every second.

Also, this is a bit unrelated to this topic but I don't know how you can add a line graphic into a sprite and reuse. A rectangle sprite was easy to scale and change x,y co-ordinates as a sprite, but I don't know how you can do this with a line.

Thanks again for your time.

[Karthik]

16 hours ago, ivan.popelyshev said:

Is line thckness constant relative to stage (zoom) or screen? You dont need to clear() and repaint every time if you just adjust position and scale. clear&refill is needed only for chunks that weren't visible in previous frame, or the one that is updated (new data).

UPD: I love that kind of threads! When you make the main algorithm, while pixijs or any other renderer or framework is just your building blocks

This is so true. I was initially intimidated to use a game rendering framework for building charts, but considering the volume of data I have to render and the handy apis PIXI offers to leverage GPU, I decided it's worth giving a shot, and I'm so happy with that decision. 

btw, I have a youtube channel where I post programming related videos. I'll try to make a series on PIXI to share the knowledge that i gained, mistakes I made, and why i recommend PIXI over other libraries.

[Karthik]

Hi All,

After some performance testing with some random data, I managed to resolve the issue. I want to share the test cases below in case  someone is interested.

As I have 400k points and at any given moment, I only need 4k elements, I tried to replicate this scenario using a container, sprites.

Test Cases:

Scenario	Graphic	Sprites	container	start with (sprites in container)	Action taken every sec to simulate my use case	performance
1	1	100k	1	100k	change x/y properties of container	very bad
2	1	100k	1	100k	randomly set visibility to true for 4k sprites	bad
3	1	100k	1	0	remove all children from container and add 4k children	good
4	1	4k sprites	1	4k	randomly move x/y and width and height properties of 4k sprites	best

 

So instead of clear and redraw, I'll load 4k sprites to my container to start with and set x/y/width/height properties as per my sample. Thanks again for all your time.

[mobileben]

I think for generating a texture from the line graphic you would do something like this, however I will be quick to add is in my test, it comes out a bit jaggy. The actual sine wave as a graphic is slightly jaggy as well, just not as bad. I also notice clipping.

You should be able to draw the two and compare to see what I mean.

const graphic = new PIXI.Graphics();
graphic.lineStyle(2, 0xff0000, 1);
const startX = 0, startY = 0;
const increment = 0.1;
graphic.moveTo(startX, startY);
for (let x=increment;x < 100;x += increment) {
    const y = Math.sin(x) * 20;
    graphic.lineTo(startX + x * 10, startY + y);
}
graphic.endFill();
let sineTex = app.renderer.generateTexture(graphic, PIXI.SCALE_MODES.LINEAR, window.devicePixelRatio);
let lineSprite = new PIXI.Sprite(sineTex);

 

[mobileben]

Also thinking about this some more. It seems to me your main problem really is the lines. For the rects (points), you really just need to create a rect and convert it to a texture and then create a pool of sprites. Alternatively you could create that sprite based on a texture image.  I'd recommend a white rect and then tint it to the color you need. You can also scale it as you need.

Just create a pool of sprites and use them as you need (and hide what you don't). This should give you good performance regrading the drawing of your points. The line is a bit more problematic. 

You probably need to define "real-time". Depending on your application real-time isn't always real-time. Meaning at times you can actually eat up more frames doing something. For example, is it still usable if it is 30fps or 20fps?

For the line, when you zoom and scale, rather than invalidate, why not build "offline" the newer line? Then when it's done, show that line and hide and either destroy or invalidate the other?

So perhaps a good solution is a pool of sprites for points and rather than clearing creating a new line and the hiding and invalidating/destroying the old.

This is one of those ideal cases where being multi-threaded is helpful, since you can offload both the new line render and destruction on another thread.

[jonforum]

I also use app.renderer.generateTexture

From my own test it look like the best performance ways for my context!
Just don't forget, when you will no more need context, to destroy(true) textures and context.

I do not think I'm wrong, but if yes, please tell me, so your that generating a texture that should be displayed in a lapse of time is more efficient than displaying a canvas element that is computed by the pixi rendering ticks engine.
I hope I have not wrong because in most cases, I generate alway a textures for my texts and graphics, I never let pixi rendering canvas like text and graphics.
I use pure pixi graphics and text only in debugging context, in game context i generate textures.

but this subject interest me very high for final optimisation, in a temporary context, is it better to generate a texture from a graphics or a texts canvas and display it as a sprites in the pixi render, or rather just renderer it as it canvas?

In case we know generate a texture for a sprite cost a lot in a frame, but after cost less for rendering in time. true ?

But if in context you need to create and destroy inside ~500ms<, maybe it not good ways!
But if you need rendering thing between 1000 to 5000 ms or >>, maybe it better to generate textures and sprites and then destroy when need?
tell me if am wrong please.

[Karthik]

9 hours ago, mobileben said:

Also thinking about this some more. It seems to me your main problem really is the lines. For the rects (points), you really just need to create a rect and convert it to a texture and then create a pool of sprites. Alternatively you could create that sprite based on a texture image.  I'd recommend a white rect and then tint it to the color you need. You can also scale it as you need.Thanks for these inputs. Yes, this appears to be the best way to handle rectangles.

Just create a pool of sprites and use them as you need (and hide what you don't). This should give you good performance regrading the drawing of your points. The line is a bit more problematic. True, The line in my case is also constantly changing.

You probably need to define "real-time". Depending on your application real-time isn't always real-time. Meaning at times you can actually eat up more frames doing something. For example, is it still usable if it is 30fps or 20fps? True again! I fixed the app.ticker.maxFPS to 30 and it aided in the performance.

For the line, when you zoom and scale, rather than invalidate, why not build "offline" the newer line? Then when it's done, show that line and hide and either destroy or invalidate the other? I talked about this above.

Thanks again for taking time to respond.
 

 

Quote

So perhaps a good solution is a pool of sprites for points and rather than clearing creating a new line and the hiding and invalidating/destroying the old.

This is one of those ideal cases where being multi-threaded is helpful, since you can offload both the new line render and destruction on another thread.

 

[Karthik]

8 hours ago, jonforum said:

I also use app.renderer.generateTexture

From my own test it look like the best performance ways for my context!
Just don't forget, when you will no more need context, to destroy(true) textures and context.

I do not think I'm wrong, but if yes, please tell me, so your that generating a texture that should be displayed in a lapse of time is more efficient than displaying a canvas element that is computed by the pixi rendering ticks engine.
I hope I have not wrong because in most cases, I generate alway a textures for my texts and graphics, I never let pixi rendering canvas like text and graphics.
I use pure pixi graphics and text only in debugging context, in game context i generate textures.

but this subject interest me very high for final optimisation, in a temporary context, is it better to generate a texture from a graphics or a texts canvas and display it as a sprites in the pixi render, or rather just renderer it as it canvas?

In case we know generate a texture for a sprite cost a lot, but after cost less for rendering in time. true ?

But if in context you need to create and destroy inside ~500ms<, maybe it not good ways! Hey, Thanks for your time. Yes 500ms is a strict "No", 
But if you need rendering thing between 1000 to 5000 ms or >>, maybe it better to generate textures and sprites and then destroy when need?
tell me if am wrong please.

 

[jonforum]

3 hours ago, Karthik said:

But if in context you need to create and destroy inside ~500ms<, maybe it not good ways! Hey, Thanks for your time. Yes 500ms is a strict "No", lol. I had to rewrite 200 lines o....

oups ya sorry , yes a frame is ~16 ms, am wrong here.  but in my head i was thinking about unsync promise listener, maybe not related to your issue, but your issue linked to my answer.:)
Sometime i get some spike lag in generating too fast new constructor graphics and text with pixi sprite, but sometime no if i use only pure pixitext for example..
But pixi text cost a lot in rendering and cpu go freak.

but if i use text as sprite texture with higher computing camera, it less than if i use it as pure canva text.
Less math computing  by cpu maybe...?

[mobileben]

12 hours ago, jonforum said:

I also use app.renderer.generateTexture

From my own test it look like the best performance ways for my context!
Just don't forget, when you will no more need context, to destroy(true) textures and context.

I do not think I'm wrong, but if yes, please tell me, so your that generating a texture that should be displayed in a lapse of time is more efficient than displaying a canvas element that is computed by the pixi rendering ticks engine.
I hope I have not wrong because in most cases, I generate alway a textures for my texts and graphics, I never let pixi rendering canvas like text and graphics.
I use pure pixi graphics and text only in debugging context, in game context i generate textures.

but this subject interest me very high for final optimisation, in a temporary context, is it better to generate a texture from a graphics or a texts canvas and display it as a sprites in the pixi render, or rather just renderer it as it canvas?

In case we know generate a texture for a sprite cost a lot in a frame, but after cost less for rendering in time. true ?

But if in context you need to create and destroy inside ~500ms<, maybe it not good ways!
But if you need rendering thing between 1000 to 5000 ms or >>, maybe it better to generate textures and sprites and then destroy when need?
tell me if am wrong please.

I assume when you ask "render it as canvas" you are referring to rendering the `PIXI.Graphics` as is? So in other words something like:

 

const gfx = new PIXI.Graphics();

// Do stuff to make a graphic

gfx.endFill();
app.stage.addChild(gfx);

Yes, you are better off converting to a texture and the creating a sprite. The actual `_render` for a graphic does a bunch of work to display the parts of the graphic. Simplified graphic shapes like rectangles should be faster to draw. Just how much work is related to complexity and whether or not the graphic is batchable.

Sprites on the other hand just update vertices (presumably only if needed, although looking at the code it doesn't look like it has any dirty bits) and then renders.

Hmm, wondering. I'm not really a JS guy. But I've done some reading that seems like you can get some async stuff running. Has anyone dabbled with that? It would be super helpful if it is real async. Since then things like texture creation could be done off the main thread. The only caveat though ... which I don't know if pixi can handle is the need for some locks.

I'm more used to multi-threaded game engines where one has those features to help better hide latency.