Is it possible to use a Sprite to load an iframe element?

[AnDG]

Hi there,
I am developing my project and currently I am using part from iframe to load another game to show it in my game. My question is does the Sprite element support loading games from an iframe element? Because I tried to download that game from PIXI.Spirte.from () but currently it does not download. Maybe I don't know how to use it properly. So can I use PIXI.Sprite.from to load an iframe element? Looking forward to receiving your help. I can still use the word iframe individually but I want to manage it more closely in display in the game. Because the used iframe is loaded on another canvas, it will not be able to edit that it will display in order, currently it is always on top of the game.

[ivan.popelyshev]

from() accepts anything. PDF, SWF, html, anything! The only problem is that you need to make a Resource for that element and code logic how to handle it.

This one is for HTML VideoElement: https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/VideoResource.ts , and here it is being added: https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/index.ts#L29 . You just have to make your own magic implementation of test() and other things! You can even write your own webgl code to upload stuff!

However, I actually doubt that you can do it with iframe at all. I would choose other method - position iframe element with specific affine transform that can be taken from sprite element after its updateTransform()-ed. AccessabilityPlugin does something like that to position edits on top. 

In case you dont understand what i'm saying , I recommend to learn PixiJS examples, read wiki https://github.com/pixijs/pixi.js/wiki/v5-Hacks , clone PixiJS repo and open it in searate IDE window, experiment with different things and forget about IFrames before you actually learn what PixiJS can.

> it will not be able to edit that it will display in order, currently it is always on top of the game.

Oh.. and now I have a feeling you dont understand what canvas is and how it actually works. Its array of pixels. It cant show DOM elements "inside". It cant process DOM at all. Maybe you should look in other renderers which use DOM tree and not canvas.

Edited March 25, 2020 by ivan.popelyshev

[eXponeta]

Hah.

U can't direct load iframe as texture, it is unpossible by design. iframe is a DOM container. How me should upload it to GPU mem?
U can use split renderer: that means, that there are canvas before iframe and after that and u sort ALL graphics object between. But there are problem - pixi can't support multicanvas renderer, u should run 2 separated Renderers and manyally select that rendered in first canvas before iframe and that rendered on secondary canvas.

Easy =)   

[AnDG]

2 hours ago, ivan.popelyshev said:

from() accepts anything. PDF, SWF, html, anything! The only problem is that you need to make a Resource for that element and code logic how to handle it.

This one is for HTML VideoElement: https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/VideoResource.ts , and here it is being added: https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/index.ts#L29 . You just have to make your own magic implementation of test() and other things! You can even write your own webgl code to upload stuff!

However, I actually doubt that you can do it with iframe at all. I would choose other method - position iframe element with specific affine transform that can be taken from sprite element after its updateTransform()-ed. AccessabilityPlugin does something like that to position edits on top. 

In case you dont understand what i'm saying , I recommend to learn PixiJS examples, read wiki https://github.com/pixijs/pixi.js/wiki/v5-Hacks , clone PixiJS repo and open it in searate IDE window, experiment with different things and forget about IFrames before you actually learn what PixiJS can.

> it will not be able to edit that it will display in order, currently it is always on top of the game.

Oh.. and now I have a feeling you dont understand what canvas is and how it actually works. Its array of pixels. It cant show DOM elements "inside". It cant process DOM at all. Maybe you should look in other renderers which use DOM tree and not canvas.

Thanks for your prompt and helpful response Ivan, I've been following you for a long time and still look forward to your assistance. I actually used a canvas element to create my game but due to the fact that I had to download another game on the section from iframe to display on my game, it interacted back and forth. between the 2 canvas elements of the game and the iframe closely. But the iframe is just a DOM element so I don't want to use it. I just want to put this iframe element in the same canvas as my current game so I can manage it best. I used from () to display a video element and it was successful but it was really difficult to load it as an iframe.

[AnDG]

2 hours ago, eXponeta said:

Hah.

U can't direct load iframe as texture, it is unpossible by design. iframe is a DOM container. How me should upload it to GPU mem?
U can use split renderer: that means, that there are canvas before iframe and after that and u sort ALL graphics object between. But there are problem - pixi can't support multicanvas renderer, u should run 2 separated Renderers and manyally select that rendered in first canvas before iframe and that rendered on secondary canvas.

Easy ?

I actually have a game displayed in the canvas element and an iframe with another game, which is really hard to manage. That's why I want to use from () to include it in the canvas element of the current game.

[ivan.popelyshev]

Maybe I and @eXponetacan help if you provide minimal demo in a zip file. We'll host it on server ("http-server -c-1" as default node-js server) and look if its actually something possible. Without minimal demo there are just too many things that can go wrong and we'll get stuck in it.

Edited March 25, 2020 by ivan.popelyshev

[AnDG]

Thanks @ivan.popelyshev, I will try to follow your instructions, if it really can't be loaded I will create a demo so you can review it. 

[ivan.popelyshev]

OK, just dont try "load as texture" aka `Sprite.from()` approach , I dont think it actually work. Try whatever minimal stuff you can possibly do: two canvases. You can even use pixi-legacy.js and 2 CanvasRenderer to simplify stuff for now, separate your stage into "before external canvas" and "after external canvas". 

Actually if you have that game sources you can really inject it into your stage if its webgl or canvas2d, need to look closer.

Edited March 25, 2020 by ivan.popelyshev

[ivan.popelyshev]

Suppose you have a invisible canvas with that game somewhere on page.

create a texture out of it : "Texture.from(outerCanvas)" , use it in a sprite (same as "Sprite.from" actually).

every frame you should 

1. call render of that game

2. call "texture.update()" of that texture you made from canvas

3. call pixi render.

If you are lucky, simple "app.ticker.add(() => { texture.update(); } )" should work , if not - study how to make your own application or add a handler before pixi renders: https://github.com/pixijs/pixi.js/wiki/v5-Custom-Application-GameLoop, . Maybe "app.ticker.add(() => { outerGame.render(); texture.update();} )" works, maybe you'll have to put it all in "render" method of your application, i dont know, you should debug it.

This "convert other canvas to texture for our renderer" trick should work and have good performance everywhere except Edge (old edge, not chromium edge). You cant pass iframe there, but if you somehow get access to canvas - yes, it should work because PixiJS has Canvas->Texture convertor in "CanvasResource" thingy, I mentioned resources in first post.

Edited March 25, 2020 by ivan.popelyshev

[AnDG]

1 hour ago, ivan.popelyshev said:

Suppose you have a invisible canvas with that game somewhere on page.

create a texture out of it : "Texture.from(outerCanvas)" , use it in a sprite (same as "Sprite.from" actually).

every frame you should 

1. call render of that game

2. call "texture.update()" of that texture you made from canvas

3. call pixi render.

If you are lucky, simple "app.ticker.add(() => { texture.update(); } )" should work , if not - study how to make your own application or add a handler before pixi renders: https://github.com/pixijs/pixi.js/wiki/v5-Custom-Application-GameLoop, . Maybe "app.ticker.add(() => { outerGame.render(); texture.update();} )" works, maybe you'll have to put it all in "render" method of your application, i dont know, you should debug it.

This "convert other canvas to texture for our renderer" trick should work and have good performance everywhere except Edge (old edge, not chromium edge). You cant pass iframe there, but if you somehow get access to canvas - yes, it should work because PixiJS has Canvas->Texture convertor in "CanvasResource" thingy, I mentioned resources in first post.

Thanks for your very detailed and complete instructions. I can still use the in-game iframe element as a standalone element that has nothing to do with the game, it's just like showing a pre-programmed game and just showing up for The user sees how it is happening. I could not interfere with the game loading from the iframe element, that was simply displaying it. I really want to optimize by including the word iframe into my canvas. In your own words, I understand that means we will have to draw another canvas and add it to the ticker to constantly update it. Is that really necessary? Because the game in the iframe element was drawn by another canvas element? I think it will take a lot of resources to display that way.

[ivan.popelyshev]

No, it will be basically texture copy, its not the most heavy operation, there are worse things. The problem is how do you get access to the canvas inside iframe.