How to understand when graphics have been fully rendered?

[shlajin]

Hey!

I have a very complex shape (graphics) I need to render. From what I understand, when any calls to PIXI.graphics happen, they all queue and execute later. I made this little playground to illustrate (take a look at the console) – it prints "End drawing" before the graphics appear on screen.

https://www.pixiplayground.com/#/edit/ZOYSkl3-FaUuWmrjNlGSW

I need to copy pixels from the canvas, so I absolutely need to find that moment when "everything is ready and ready to roll".
My second problem is that ideally, I would love to make browser not hang during the painting. Usually what people do is `setTimeout(proceed, 0)`, and I think it will work for me – I can pause creating graphics to allow browser run other tasks, but I have no idea how I can delay the rendering until the next event loop tick.

During my experiments I tried to use `BasePrepare` to upload Graphics to GPU, but I found that I have no control over the "lag" – even if I don't add graphics on screen, the lag still happens on the next tick after I draw lines, so I cannot delay when it happens.

I found old forum threads that `render` is sync, so I tried to override render methods of the root container, yet without any luck.

I feel like there should be a better solution to this -- I would highly appreciate any help. Thanks!

[ivan.popelyshev]

>  when any calls to PIXI.graphics happen, they all queue and execute later. 
 

yep, it just stores instructions

use "renderer.render" to your own renderTexture to get the info. 

Alternatives: "renderer.generateTexture", "graphics.generateCanvasTexture" (in pixi.js-legacy), and many others, but all have disadvantages, aka BUGS. if you dont know what is renderTexture, how it works - you'll spend many hours to get it right

Also you have to know how to get info from canvas, otherwise any bug that can spawn in "extract" operation will stop you. Hope you know what "readPixels" and "getImageData" are.

[shlajin]

Well, I’m oversimplifying things for this example. Render texture might not work, as I need to scale graphics without making them look “bad” (like raster graphics).

My latest idea is to mark all graphic instances as dirty which recently received a draw command, then tap into their “render” function and wait until they finish. It works (haven’t ran excessive test yet, would love to hear your opinion whether it may work or not), but it still does not solve the issue of lag during draw. How are the “draw instructions” called - maybe I can inherit from graphics and delay the work somehow?

Thank you!

[ivan.popelyshev]

there are multiple problems with that, and its active problem. No one solved that for all the duration of WebGL existance. You are asking something that does not exist in 2d webgl renderers, and even if exists - only partial and in user codebases that cost money. I have one project that has thing like that, maybe that gives you ideas: https://github.com/gameofbombs/pixi-blit . Again, no such projects that have examples and tests available in public.

Also, AA doesnt work on renderTextures. No smoothing for graphics.

You have two options:

1. Describe your task in details, then i can tell you more

2. Learn all the things. Your current idea is possible to code but have many problems in implementation. You cant just take pixels from videocard, and even if you learn such operations and how to do it in general webgl and pixi - it can be only once per frame, otherwise it slows down everything

 

[ivan.popelyshev]

"Learn all the things" - i mean, read the source code  PixiJS is codebase-oriented project, we maintain readability of code, because we cant make all the docs  

[shlajin]

Sheesh, could not seen that coming – I thought it's rather trivial to get moment when scene is rendered...

Quote

Describe your task in details, then i can tell you more

Sure, thanks!

It's like a promo-app where you can customize an existing movie. Animation is done via Adobe Animate, then exported via pixi-animate extension plugin, and then played by a custom lib. When the customization is done, user can export the movie (via https://github.com/thenickdude/webm-writer-js). Due to distribution policy, the app will be used only in chrome environment (likely an electron app).

So naturally, there are 2 different states of the app:

- Edit state: we don't care that much about quality here, but we do care about the performance. If the app freezes etc nobody will like it, so I need to find a way how to allow the browser to run other tasks in between rendering, ideally making sure a webgl "chunk of work" won't last more than (1 / FPS) ms. My backup plan is to run no-more than N draw commands each frame – this way browser will have some leeway, but it's a hacky way which stinks – e.g. if PC is powerful enough, rendering will take more time than it should (due to waiting for new frames to receive new commands).

- Rendering state: we don't care that much about performance here, as we care about quality – I'm considering using mocking MSAA with 2x canvas size for a better quality. I mean, nobody cares about fast rendering if the output result is bad, right? Yet I need to understand when it's "safe" to take a snapshot of the canvas, meaning when all objects are fully rendered.

 

[shlajin]

Ok, I've been running tests for the past few hours / reading through pixi code, and that's what I found:

- `_render` method of DisplayObject's indeed might finish long before all children get fully rendered. I guess I could tap into (aka inherit) every pixi class and patch the `_render` method to fire an event when they finish render, or...

- I can ditch the pixi's convenience `Application` (or swap ticker / do not auto-start the app) and call `render` manually (the `this.renderer.render(this.stage);` part).

 

The only caveat I found so far is that resources have to be fully loaded in RAM (and it makes total sense). E.g. I can't just `PIXI.Texture.from(url)`, throw it on stage and hope that `render` will synchronously wait until the image is fetched (and, frankly, nothing in JS in browser runtime can). However, if the image is loaded the `render` synchronously uploads it to GPU, and I for sure can wait until the image load in plain JS (e.g. via `image.onload`).

So far it feels like manually triggering `render` when all resources have loaded solves my problem with waiting until frame is fully drawn to grab the image. Am I missing something?

The problem of dividing huge renders into chunks to allow browser run other tasks is still a problem...

[ivan.popelyshev]

> - I can ditch the pixi's convenience `Application`

that's for sure is first step. https://github.com/pixijs/pixi.js/wiki/v5-Custom-Application-GameLoop . I dont understand people who use default application in serious projects

> So far it feels like manually triggering `render` when all resources have loaded 

There's also fromUrl() that returns promise, might help

> The problem of dividing huge renders into chunks

oh, that's one of those tasks. I dont envy you

[shlajin]

> There's also fromUrl() that returns promise, might help

I'll check it out, thanks!

>  I dont envy you

It turns out to be not that complex. I created a small very framework which can enqueue jobs and process them until they are ready (and taking pauses in between jobs if applicable).

After measuring what takes the most time I found out that for graphics I should do the drawing commands, and then call geometry `updateBatches`. After that rendering happens pretty fast, so I just have to run these as a "background job" before showing the result to the end user (I use `alpha = 0` to hide the container from the user yet make pixi render it).

For containers I simply do children construction in one "job chunk". (like, the creation of PIXI.Graphics / adding them as children). Once all children is ready, I declare the container as ready and it's good to be shown to the user after the next `container.render` finishes.

So far I can construct a whole scene with just one-step-at-a-time (with 0ms threshold – every step happens in a new `requestAnimationFrame`), with rendering loading spinners on every object at 144 FPS, and taking a `toObjectURL` once the scene is fully rendered (I don't even need to preserve drawing buffer, it happens right after `render` so it correctly captures the canvas!).

Apparently my use-case is pretty niche that all goes good so far, yet I'm a little anxious about your words:

> there are multiple problems with that, and its active problem. No one solved that for all the duration of WebGL existance. 

Could you elaborate what exactly is an active problem? Better understand now what's coming later 

[ivan.popelyshev]

Very glad that you resolved it.

> Could you elaborate what exactly is an active problem? 

drawing and caching vector stuff with different scale

Edited April 5, 2021 by ivan.popelyshev