[Three.js] - WebGLRenderTarget.texture as uniform

[mchlr]

Hey there,

I've recently started to dig my way more into three.js in order to build my own image-viewer-app as my first three.js project.

I'm using three.js r83 and both the EffectComposer aswell as the Shader/RenderPass from the three.js examples. (View on github)

Since I'm familiar with other programming languages I was able to figure out a lot of stuff on my own, but currently I'm struggling with this specific problem:

My App should be able to add post-processing effects to the currently viewed image. The post-processing part already works like a charm, but I would like to add more effects as I want to test/experiment around with some new sorts of possibilities for an image-viewer.

Since I'm obsessed with performance, I came up with some ideas on how to scale the post-processing into different EffectComposers in order to keep weight (Number of Shaders to render) on each Composer low and therefore it's performance high.

 

What I did: After debugging both the EffectComposer and Shader/RenderPass from the three.js examples, I came up with the idea to render a texture, that I'm able to re-use as a uniform in another Composer later on. This would enable me to encapsulate and pre compute whole post-processing chains and re-use them in another Composer.

While I was debugging through the ShaderPass, I found what I think is the key element to get this to work. I won't post the Code here as it's accessible via github, but if you have a look into the ShaderPass.js on Line 61 you can see the classes' render function. The parameter writeBuffer is a WebGLRenderTarget and, afaik, it is used to store what the composer/renderer would usually put out to the screen.

I've created 2 identical Composers using the following code:

 var txt = testTexture;

                    var scndRenderer = new THREE.WebGLRenderer({
                        canvas: document.getElementById("CanvasTwo"),
                        preserveDrawingBuffer: true
                    });
                    scndRenderer.setPixelRatio(window.devicePixelRatio);

                    var containerTwo = $("#ContainerTwo")[0];

                    scndRenderer.setSize(containerTwo.offsetWidth, containerTwo.offsetHeight);

                    console.log("Creating Second Composer.");
                    console.log("Texture used:");
                    console.log(txt);

                    var aspect = txt.image.width / txt.image.height;
                    var fov = 60;
                    var dist = 450;

                    // Convert camera fov degrees to radians
                    fov = 2 * Math.atan(( txt.image.width / aspect ) / ( 2 * dist )) * ( 180 / Math.PI );

                    var scndCam = new THREE.PerspectiveCamera(fov, aspect, 1, 10000);
                    scndCam.position.z = dist;

                    var scndScene = new THREE.Scene();
                    var scndObj = new THREE.Object3D();

                    scndScene.add(scndObj);

                    var scndGeo = new THREE.PlaneGeometry(txt.image.width, txt.image.height);

                    var scndMat = new THREE.MeshBasicMaterial({
                        color: 0xFFFFFF,
                        map: txt
                    });

                    var scndMesh = new THREE.Mesh(scndGeo, scndMat);
                    scndMesh.position.set(0, 0, 0);

                    scndObj.add(scndMesh);
                    scndScene.add(new THREE.AmbientLight(0xFFFFFF));

                    //PostProcessing

                    scndComposer = new THREE.EffectComposer(scndRenderer);
                    scndComposer.addPass(new THREE.RenderPass(scndScene, scndCam));

                    var effect = new THREE.ShaderPass(MyShader);
                    effect.renderToScreen = false; //Set to false in order to use the writeBuffer;

                    scndComposer.addPass(effect);

                    scndComposer.render();

I then modified three's ShaderPass to access the writeBuffer directly.

I added a needsExport property to the ShaderPass and some logic to actually export the writeBuffers texture:

            renderer.render(this.scene, this.camera, writeBuffer, this.clear);

            //New Code
            if (this.needsExport) {

                return writeBuffer.texture;

            }

I then simply set the needsExport for the last pass to true. After rendering this pass, the texture stored in the writeBuffer is returned to the EffectComposer. I then created another function inside of the EffectComposer to just return the writeBuffer.texture, nothing too fancy.

 

The Issue: I'm trying to use the writeBuffers texture (which should hold the image that would get rendered to screen if I would have put renderToScreen to true) as a uniform in another EffectComposer.

As you can see in code block 1, the texture itself isn't resized or anything. The used texture got the right dimensions to fit into a uniform for my second composer, however  I'm constantly receiving a black image from the second composer no matter what I do. This is the code I'm using:

function Transition(composerOne, composerTwo) {
    if (typeof composerOne && composerTwo != "undefined") {

        var tmp = composerOne.export();

        //Clone the shaders' uniforms;
        shader = THREE.ColorLookupShader;
        shader.uniforms = THREE.UniformsUtils.clone(shader.uniforms);
        var effect = new THREE.ShaderPass(shader);

        //Add the shader-specific uniforms;
        effect.uniforms['tColorCube1'].value = tmp; //Set the readBuffer.texture as a uniform;

        composerTwo.passes[composerTwo.passes.length - 1] = effect; //Overwrite the last pass;

        var displayEffect = new THREE.ShaderPass(THREE.CopyShader);
        displayEffect.renderToScreen = true;

        //Add the copyShader as the last effect in Order to be able to display the image with all shaders active;
        composerTwo.insertPass(displayEffect, composerTwo.passes.length);

        composerTwo.render();
    }
}

 

 

Conclusion: To be completely honest, I don't have a clue about what I'm doing wrong.

From what I've read, learned while debugging and from what I've figured out so far, I would argue that this is a bug. I would be really glad if someone could prove me wrong or submit a new idea on how to achieve something like what I'm already trying to do.

 

If there are any more informations needed to solve this question, please let me know!

 

Regards,

Michael 

[mchlr]

Update / Push:

I tried something new:

I'm now trying to create a new Texture based on the Canvas filled by the FirstComposer (The FirstComposer should provide a texture that is useable in a uniform of another Shader). Now, I'm getting an Image which is great, but the Image is heavily distorted.

I think its because of the size of the Composer. Both of the Composere are using an unaltered version of the texture but their seize (What they display) is adjusted to fit the container in my app.

Here is the Sourcecode I'm using to get the Texture from the canvas:

function TestComposerTransition(composerOne, composerTwo) {
        composerOne.setSize(4096, 64); //Tried to "hack" the composers size, sadly without an effect.
        var firstOutput = document.getElementById("CanvasOne");


        var tmpTxt = new THREE.Texture(firstOutput);
        tmpTxt.needsUpdate = true;

        //Clone the shaders' uniforms;
        shader = THREE.ColorLookupShader;
        shader.uniforms = THREE.UniformsUtils.clone(shader.uniforms);
        var effect = new THREE.ShaderPass(shader);

        //Add the shader-specific uniforms;
        effect.uniforms['tColorCube1'].value = tmpTxt;

        composerTwo.passes[composerTwo.passes.length - 1] = effect;

        var displayEffect = new THREE.ShaderPass(THREE.CopyShader);
        displayEffect.renderToScreen = true;
        composerTwo.insertPass(displayEffect, composerTwo.passes.length); //Add the copyShader as the last effect in Order to be able to display the image with all shaders active;

        composerTwo.render();
}

I uploaded an Image of the expected and the distorted texture.

 

Does anyone have a clue why this is happening or what I could do to fix this?

//EDIT: I changed the FirstComposers output into the correct dimensions of the Texture, however I'm still getting the distorted image posted above.

 

Best regards,

Michael

[mchlr]

Aaaaaand we're done!

 

After publishing another issue on github and after annoying WestLangley once again, I finally figured out a solution for this problem.

Apparently there was a problem with the mipmapping for the texture I generated from the Canvas. I changed the .generateMipmaps property of the effected texture to false and set both the .min- and .magFilter to THREE.LinearFilter. Check https://github.com/mrdoob/three.js/issues/10486

 

//EDIT: @Mods: This topic can be closed.

 

Regards,

Michael