Integrating PixiJS and a custom WebGL engine that owns the render loop, the target framebuffer and the viewport

[damix911]

Hello, I need to integrate PixiJS as a "guest" in a custom WebGL engine, which we will call the "host".

The "host" engine owns the render loop and at every frame it almost resets the WebGL state and then hands control over to PixiJS. After PixiJS returns, the "host" resets the state again.

In code:

function onFrame() {
  resetWebGLStateToDefault(gl);
  gl.bindFramebuffer(...)
  gl.viewport(...)
  thenWeUsePixiJSToDoSomeAdvancedStuff();
  resetWebGLStateToDefault(gl);
}

What I have tried so far to implement thenWeUsePixi... function is along the lines of:

function thenWeUsePixiJSToDoSomeAdvancedStuff() {
  renderer.reset();
  ...
  renderer.render(sprite);
  ...
  renderer.reset();
}

Which also seems in line with what is suggested in bullet 2 here: link

However, doing so caused the following error, and depending of the precise order of operations one of the two engines will typically fail to draw.

[.WebGL-0x7fd105545e00]RENDER WARNING: there is no texture bound to the unit ...

I get that error for all texture units but unit 0.

Question 1. What am I doing wrong?

Question 2. Is there any guideline about integration of PixiJS with other engines?

I have more specific question doubts (see https://github.com/pixijs/pixi.js/discussions/7210) but for now I would be happy to learn more about the big picture.

Thanks!

See also This question on StackOverflow

 

[ivan.popelyshev]

>  What am I doing wrong?

You are not debugging it.

[damix911]

Thank you Ivan, I am planning to get a debug version and step into the code but I was hoping it was a common case with a known answer. I'll let you know if I make progress.

[damix911]

Still don't know exactly what the problem was, but it seems that it was originating by the bound framebuffer not being the default one when calling into PixiJS.

Somehow this was causing the multitexture shader to be invoked with 15 out of 16 textures unbound, and possibly some of the other behaviours that I described in my post.

Anyway, PixiJS was doing the right thing, it was my code that was not resetting the state properly.

 

I guess the question now becomes; can you tell PixiJS to render into an externally created framebuffer?

I have a https://codepen.io/dawken/pen/MWbWXXQ?editors=1010 where I use a PIXI.RenderTexture which I then apply to a full-screen quad.

Is there a more PixiJS way of doing the same thing? Thanks!

Edited January 29, 2021 by damix911

[ivan.popelyshev]

>  can you tell PixiJS to render into an externally created framebuffer

NO, and dont try to look into "renderTexture.framebuffer" and "renderTexture._glTextures"!

I did it with ThreeJS , but usually its the other way around: importing webgl objects from pixi to another engine

Edited January 30, 2021 by ivan.popelyshev

[ivan.popelyshev]

Im trying to find that example but for now it seems that it was lost to time  Anyway, I actually respect people who work with multiple renderers/engines  I snoop in other renderers code often

Edited January 30, 2021 by ivan.popelyshev

[damix911]

Thank you! I work for a company that focuses on GIS software and one component of our platform is a 2D/3D Javascript mapping API. We have an extension point for custom WebGL visualizations and, while most of our SDK samples focus on raw WebGL, we are working on some integrations with 3rd parties. We are still missing an integration with a 2D, general purpose, library. PixiJS is awesome, I'll take some more time to study it and I will try to clean up the sample and make it presentable.

Thanks!

[ivan.popelyshev]

I made pixi-v4 + threejs integration for one nda-ed project, and it had 1. framebuffer sharing 2. z-sorting of different elements in pixijs/threejs 3. convertions between pixi coords and three camera coords

Your case will probably have more hacks than what I did

[damix911]

Quote

usually its the other way around: importing webgl objects from pixi to another engine

Yes, in a way that ugly line in my sample: `const texture = this.renderTexture.baseTexture._glTextures[this.renderer.texture.CONTEXT_UID].texture;` was a way of doing that. How would I import a WebGL object from PixiJS to another engine? If I understand correctly a PixiJS object can be used in multiple contexts and, more importantly, in 0 contexts; it enables you to start creating objects and structuring your rendering algorithm independently of the WebGL context; it's really cool but doesn't this mean that I need something like that ugly line to navigate to the real underlying WebGL object for a given context? Thanks! ?

Quote

Your case will probably have more hacks than what I did

In general an integration with our API involves:

1) Rendering solely to a specific framebuffer owned by the map

2) Render only on the render() method of the custom layer

3) Coordinate conversion; easy for points, little more involved for polygons and other visual entities that have a geographical extent

4) Make sure that you use premultiplied alpha in the final write

If you are interested, here is a link to the base class of all our custom visualizations: https://developers.arcgis.com/javascript/latest/api-reference/esri-views-2d-layers-BaseLayerViewGL2D.html

[damix911]

Hello Ivan, sorry to bother you, but do you have any suggestion wrt the right way to go about this?

Quote

 

NO, and dont try to look into "renderTexture.framebuffer" and "renderTexture._glTextures"!

I did it with ThreeJS , but usually its the other way around: importing webgl objects from pixi to another engine 

 

To recap, every frame, my engine needs to ask PixiJS to render into a `framebuffer: WebGLFramebuffer` with a specified `viewport: [number, number, number, number]`.

Thank you!

[ivan.popelyshev]

Please make minimal demo, I'll fix it.

[damix911]

Thank you, you are awesome!

https://codepen.io/dawken/pen/NWbNqKz

The JS now it's 80 LOC. You only care about LOC 2-63; that's where the "custom layer view" is created. You don't need to look into the HTML and CSS files.

In the "attach()" method I create the PIXI stuff (including the render texture) and some auxiliary WebGL stuff that I will need later.

In the "render()" method I draw with PixiJS and then overlay the render texture. Line 47 is the ugly one.

Let me know what's the best way to cite/reference PixiJS. The plan is to include this sample as part of our SDK and/or an upcoming (virtual) event for our developers.