Mixing WebGL and Canvas

[pongstylin]

So I have an auto detect renderer that renders the main application stage and a canvas renderer that renders a lightweight stage that includes some "manual/hacked" gradient effects.

The thing is that the whole thing works great if the main application is using a canvas renderer. But if I render the canvas right before rendering the webgl, then the webgl won't correctly render. Visual objects fall to appear.

Does anybody have any insight into this? Providing a minimal example would be a pain.

[hubert]

are you talking about using two different contexts in one canvas element? It's not going to happen.

[pongstylin]

It is separate elements.

[hubert]

yeah, minimal example would be nice but since it is too much code then ok.

 

Have you tried forcing the first canvas to render in webGL context?

 

http://www.sevenative.com

[pongstylin]

Creating a minimal example is a pain, but not unreasonable.  I'll do that if I have to.

 

I just now disabled my gradients on the smaller canvas and used webgl mode and it works fine.  So if both canvas use '2d' context, it is fine.  If both canvases use 'webgl' context, it is fine.  If we mix the two, everything is fine except when the 2d canvas renders right before the webgl canvas.  At least, that is what I'm seeing so far.

 

I couldn't figure out what sevenative has to do with my issue.

[achexi]

 

I couldn't figure out what sevenative has to do with my issue.

That's just his signature. It's unrelated.

[hubert]

So maybe the issue is the gradient not the 2d context that you have. How did you make these gradients!? You mentioned that they are some kind of 'manual/hacked'. Maybe your error is coming from trying to generate a gradient in 2d context and it blocks any further graphics rendering in webGL? 

 

could you at least provide us with the code for the gradients? If they are the issue we might help you out without showing the whole code?

 

http://www.sevenative.com

[pongstylin]

So that the gradients are not a distraction, I have removed them from my code and the issue remains.  It was very low impact.  My method was simply to pass a manually constructed createLinearGradient() to pixi's lineStyle as an extra parameter (I also hacked pixi.js to accept and use the extra parameter).  When rendering in canvas context, I made sure the gradient style was applied to the line.

 

So now I have a solid line where a gradient line used to be and the issue remains.

 

Has anybody ever ran 2 renderers in parallel where one was a 2d context and the other webgl (again, separate canvas elements)?  Or am I the only one crazy enough to do this?  Cause based on what I've seen, it's not very hard to bump into this issue with such an environment.  If you render the 2d canvas before the webgl canvas in the same javascript thread, textures that were not previously rendered fail to render.  At least, this is what I think I am seeing.  It is also very repeatable.

[agamemnus]

I think you need to deconstruct the issue some more.

 

It is definitely possible to have both a 2d context and a webgl context for each canvas element.

 

For now, I am using both 2d context and webgl context. I create my jigsaw puzzle pieces with a 2d context and then use Pixi to convert the canvas element to a webgl texture. After this, modifying the 2d context has no effect on the webgl rendering (of course!) unless I do the same procedure of creating a webgl texture from the 2d context.

[pongstylin]

agamemnus, do you build a pixi stage for your jigsaw puzzle piece and render the 2d context using a pixi renderer before turning it into a texture that you can use in your webgl canvas?

[agamemnus]

Ok, I see what you're doing.

 

It's most likely some pixi.js bugs. I myself found many renderer deallocation issues, but what you are doing and what I am doing is quite uncommon, so not well tested. If you are using two separate renderers this is likely going to be an issue because pixi.js was designed with only one renderer at a time in mind and most people only use one... just not enough testing.

 

I recommend that you just convert your 2d element to a webgl texture when it is supposed to be modified.

 

I would not use 2d context at all if pixi had exactly the same capabilities of 2d canvas, but it doesn't quite yet... although I'm working towards reducing my 2d context use.

 

At the moment my code mostly relies on the concept of one sprite to one texture, quite simplifying workflow. Here's some of my code:

function set_sprite (init) {  if ((init instanceof HTMLCanvasElement) || (init instanceof HTMLImageElement)) {   var canvas         = init   var logical_parent = init   var parent         = draw_area.piece_sprite_container   var position       = parent.children.length   invalidate         = init.invalidate  } else {   var logical_parent = (typeof init.logical_parent != "undefined") ? init.logical_parent : init.canvas   var canvas         = init.canvas   var parent         = (typeof init.parent   != "undefined") ? init.parent   : draw_area.piece_sprite_container   var position       = (typeof init.position != "undefined") ? init.position : parent.children.length   var invalidate     = init.invalidate  }  if (typeof invalidate == "undefined") invalidate = false  var cached_sprite_data = logical_parent.cached_sprite_data  if ((logical_parent.sprite) && (typeof init.position == "undefined") && (parent.children.indexOf(logical_parent.sprite) != -1)) position -= 1  if (logical_parent == canvas) clear_sprite (canvas)  logical_parent.texture = new PIXI.Texture.fromCanvas (canvas)  logical_parent.sprite  = new PIXI.Sprite (logical_parent.texture)   // Potentially merge some cached sprite data with the new sprite.  if (typeof cached_sprite_data != "undefined") {   for (var property_name in cached_sprite_data) {    logical_parent.sprite[property_name] = cached_sprite_data[property_name]   }  }  parent.addChildAt (logical_parent.sprite, position)  screen_refresh ()   // Invalidate the parent?  if ((typeof parent != "undefined") && (parent == draw_area.piece_sprite_container) && (invalidate == true)) parent.is_cached = false   return logical_parent.sprite }  function clear_sprite (canvas, invalidate) {  if (typeof invalidate == "undefined") invalidate = false  if (!canvas.texture) return  var parent = canvas.sprite.parent  if (typeof parent != "undefined") canvas.sprite.parent.removeChild (canvas.sprite)  canvas.texture.destroy (true)  canvas.texture = null  canvas.sprite  = null   // Invalidate the parent?  if ((typeof parent != "undefined") && (parent == draw_area.piece_sprite_container) && (invalidate == true)) parent.is_cached = false  screen_refresh () }

[pongstylin]

Hrm, I don't think any of that helps.  You see, this is my use case:

 

1) The main canvas (webgl if supported) is a board game that has a set width and height that may be scaled smaller to fit the device.  Even when scaled down significantly, the game is still playable since you can then then zoom in after seeing the overview.  Game canvas scaling and zooming are all managed by Javascript/CSS.  I only use pixi scaling when I am trying to achieve various effects.

 

2) The other canvas is an information card that does not scale with the rest of the game and is layered on top of the game in the top left corner.  It offers information that may change as you interact with the game.  It is also interactive.  You may toggle between different pages of information or temporarily hide the card if it is in the way.

 

Both canvases are permanent - always in use, even if the card is temporarily hidden.  Even the card canvas is somewhat complex since there are many text objects and various display objects that can become visible or invisible to toggle between different pages of information.  So doing all of this manually without using pixi stages and renderers involves way too much code.

 

I am using pixi version 1.6.1, so it works fine.  They have come a long way and I am able to manage more than one webgl stage and renderer without issues.  Good job pixi!  But, I want to make use of 2d canvas features that may be difficult to pull off in webgl.  I also don't want to have to resort to generating yet more gradient images.  That's my problem to solve through various hackery.  But my question has to do with this possible bug where the canvas context seems to be messing up my webgl context and it may be difficult for me to track down why, so I'm asking for any sort of guidance or suggestions on where to look.  I've had to fix various (minor) bugs in the pixi code before.

[agamemnus]

Well, in theory, the 2d context and the webgl context don't interact. The only way they do is if you do that in your code, by, for example, creating a texture from the 2d context...

 

Why not just create a texture from the 2d context after you have created it? You can just create a new element if you are worried about holding both the webgl context and a duplicate 2d context in memory...

[agamemnus]

Hmm. I was wrong. You can't have a 2d context and a webgl context in the same canvas. If you create one, the other returns null forever...

[pongstylin]

Ok, so I finally spent the hour+ that it took to copy my code to a new location and strip away all the noise.  I've reduced it enough that it shouldn't be too difficult to read while retaining enough that the concept should still be evident.

 

Bug summary: if you use PIXI to create 2 renderers (and 2 canvas elements) where one is webgl and the other is 2d, rendering the 2d before rendering the webgl, then textures don't show up in the webgl canvas.

 

Bug in action:

http://www.taorankings.com/pixi-bug/mixed.html

 

Board is visible when webgl is rendered before 2d.

http://www.taorankings.com/pixi-bug/workaround.html

 

Board is visible when both canvas elements use 2d rendering.

http://www.taorankings.com/pixi-bug/2d.html

 

Board is visible when both canvas elements use webgl rendering.

http://www.taorankings.com/pixi-bug/webgl.html

 

It is using the latest copy of pixi.js 1.6.1 (not modified).

 

Any assistance on what to fix in the pixi code to address this?  Please don't write the "if it hurts when you lift your arm, don't lift your arm" prescription.

[agamemnus]

Can you try with my fork? It has quick and dirty fixes for a number of issues I found.. I don't have time for proper pull requests yet.

 

https://github.com/agamemnus/pixi.js/blob/master/bin/pixi.js

[pongstylin]

Still broken.

http://www.taorankings.com/pixi-bug/agamemnus.html

[agamemnus]

All right. I'll take a look at the code later... If you want, we can try to figure it out together later tonight, maybe 11:00pm EST or so? I'm in the Freenode network under the "Agamemnus" nick.

[pongstylin]

Alright, don't worry about it guys.  Thanks Agamemnus for offering to help me track it down on IRC.  But I was mainly just hoping somebody had an idea of what was going on without too much effort.  If effort is required, I'll just spend it myself.  So I threw a lot of console logging into the pixi.dev.js to figure out what was different between running the canvas render before vs after the webgl render and figured out what was going on.  I've submitted a pixi bug ticket:

 

https://github.com/GoodBoyDigital/pixi.js/issues/1035