create screenshot with post processing

[Kesshi]

Hi,

 

i need to create a high qualitiy (high resolution) screenshot of my scene.

I'm using BABYLON.Tools.CreateScreenshot() to do this. The Problem is that the attached post processing does not

resize accordingly. The result is a big pixelated image.

 

I created a playground for this: http://www.babylonjs-playground.com/#QNSHJ#4

To see the problem more clear, make the playground window very small and then press the "create screenshot" button in the upper right corner.

 

The Problem seems to happen only if you have more then one active post process. In the playground i have two. If you disable one the created snapshot will be not pixelated.

How can i solve that issue? I need more than one post processing effect.

[JCPalmer]

Never noticed that you could specify a size for CreateScreenshot().  Is this always been there?  Anyway, it looks like CreateScreenshot() is implemented as its own sort of "post process", meaning it is based on the last frame just produced.

 

If so, you might set the size of the engine yourself, using engine.setSize(), and allow at least one frame to be produced at the resolution you desire, for marketing quality results.

 

There is a hook in the animation system I am working on, Queued Interpolation, to make high quality mp4 videos.  The part that hooks in is on hold, and will never get published since it is marketing / business related. I implemented an early version of the time line control using a scene afterRender, where it determined what time would be represented in the coming next frame, and optionally process the last frame as part of the .mp4. 

 

In your button, you might call engine.setSize() & register a scene.afterRender, which actually snap the picture, and unregistered itself.  In my example, the scene afterRender is already running, so the after render itself calls engine.setsize()

[Vousk-prod.]

@JCP Yep, always been there the size param. And size.precision is used to downscale or upscale the render.

 

@Kesshi If I remember well the screenshot fonction does not take the post process in account, it creates a render target with the whole current scene, then dumps the frame buffer into a new canvas 2d context, but the post processes are never applied to the final image.

[GameMonetize]

Woot! this is a bug;(

 

I will commit it in a minute

 

(You can already force a refresh of your playground to see it live)

[Kesshi]

perfect thank you

[Vousk-prod.]

Oh, if we specify a screenshot size with a ratio different from the current one, the screenshot is stretched in one side, and the engine is then stretched forever the other side...

http://www.babylonjs-playground.com/#QNSHJ#5

[Vousk-prod.]

Oh in fact it's more complex :

 

¤ When specifying only width for the screenshot (or precision) :

. If we don't add postprocess to the scene, the screenshot is ok and the engine is ok

. If we add postprocess to the scene, the screenshot is slightly stretched, but the engine is ok (if you're not able to see the slight stretch by eyes, simply create a screenshot with postprocess disabled, keep the tab opened, and create a new screenshot with postprocess enabled, then swtich from one tab to other, and tadaaa you'll see)

 

¤ When specifying a new ratio for the screenshot (by setting width and height) :

. If we don't add postprocess to the scene, the screenshot is ok but the engine is then stretched

. If we add postprocess to the scene, both are stretched, in opposite directions

 

And also, I noticed that the default AA is not in the screenshots produced (by "default AA" I mean the one applied if we flag true the antialias param of the engine (new BABYLON.Engine(canvas, true)) and don't add explicit AA filter postprocess afterward).

[Kesshi]

Oh in fact it's more complex :

 

¤ When specifying only width for the screenshot (or precision) :

. If we don't add postprocess to the scene, the screenshot is ok and the engine is ok

. If we add postprocess to the scene, the screenshot is slightly stretched, but the engine is ok (if you're not able to see the slight stretch by eyes, simply create a screenshot with postprocess disabled, keep the tab opened, and create a new screenshot with postprocess enabled, then swtich from one tab to other, and tadaaa you'll see)

 

¤ When specifying a new ratio for the screenshot (by setting width and height) :

. If we don't add postprocess to the scene, the screenshot is ok but the engine is then stretched

. If we add postprocess to the scene, both are stretched, in opposite directions

 

I can confirm this results. This will be a problem in my project.

@Deltakosh: Is this also a bug and can it be fixed?

 

 

And also, I noticed that the default AA is not in the screenshots produced (by "default AA" I mean the one applied if we flag true the antialias param of the engine (new BABYLON.Engine(canvas, true)) and don't add explicit AA filter postprocess afterward).

 

This is a WebGL limitiation. No AA if you render into a texture.

[RaananW]

Regarding the resize bug - 

I believe the resizing of the engine is due to this line:

            this._gl.viewport(0, 0, texture._width, texture._height);

located in the "bindFramebuffer" of the engine. I don't want to mess with the engine at the moment as one small change can lead to a major catastrophe (and, we are in beta ), i'll wait for DK with that.

[GameMonetize]

Fixed!

[Vousk-prod.]

Many thanks.

[Vousk-prod.]

Hi!

I've just tested the new CreateScreenshot function, and we have with this new one the same problem we had with the previous method (now called CreateScreenshotUsingRenderTarget)

The final image is stretched a lot with CreateScreenshot if we define widht AND height values far from current canvas ratio.

And strangely, it also streched (but not so much) with the CreateScreenshotUsingRenderTarget method whereas this was fixed last year...

Check this PG : http://www.babylonjs-playground.com/#QNSHJ#7

And the below picture :

[Pryme8]

engine.setSize(), can I use that to make BJS calculate at half res, but still be fullpage?

[GameMonetize]

The RT version should work as expected. I'll check what is wrong

The new screenshot works as intended: you cannot define a ratio different than original one

[GameMonetize]

Here is the fix:

http://www.babylonjs-playground.com/#QNSHJ#8

(you cannot use BILINEAR when using non power of 2 resolutions

[GameMonetize]

I've just published an update for the CreateScreenshot function as well

[Nabroski]

@Pryme8Dump the Framebuffer and then im sure you know 99 ways to resize a base64 image

http://babylonjs-playground.com/#10OY8W#4

[Nabroski]

Be like assembler my friend, be like assembler

 

[Vousk-prod.]

Thanks DK! You rocks, as usual.

[Vousk-prod.]

Hmmm, sorry to bother again @Deltakosh but when scene's clearColor has alpha, the PNG generated by the new Screenshot function does not take alpha in account in Chrome (it works in FF).

And also (but in FF only this time ) there is a strange colored artifact happening on some objects edge and lineMesh. [EDIT: not a bug, to avoid outlines artifacts, use premultiplied alpha for clearColor, see Kesshi's post page 2]

PG : http://www.babylonjs-playground.com/#QNSHJ#9

 

[Vousk-prod.]

Do you have any idea, DK, concerning this bug ?

 

[GameMonetize]

Do you still repro it?

[Vousk-prod.]

Yes, I still repro it. With the PG above, however scene clearColor alpha is 0, with Chrome the screenshot has the background color instead of transparent.

And with Firefox (classic and developper versions), some objects edges are blueish.[EDIT: not a bug, to avoid outlines artifacts, use premultiplied alpha for clearColor, see Kesshi's post page 2]

I also noticed that when choosing a render width a lot greater than canvas original size, the image is pixelated (it looks like the render happened at canvas original size and the produced image is upscaled afterwards). You can check that with the same PG, using width:4096 instead of 1024.

[Nabroski]

@Vousk-prod. From my knowing Chrome handles PNG with Alpha different that Firefox since the early days of creation of universe.

[Vousk-prod.]

@Nabroski Yup, but the old render target method works. And a "manual" canvas dump ( canvas.toDataURL("image/png") ) also works with no problem in Chrome. Also the fact that the final screenshot is upscaled tends to show that there is something strange in the process involved in new screenshot function.