custom interpolation when scaling (glsl)

[d3bgger]

Hello all, 

i'm making something like an image editor, and i would like to have a bi-cubic interpolation when scaling stage down, and nearest when stage scale is > 2, 

So instead of using PIXI.settings.SCALE_MODE, which doesn't support bi-cubic and i think it can't be changed after initialization, i decided to create my own glsl interpolation filters for the sprites i use to display the images.

I disabled mipmaps, and created and applied a fragment shader like this:

PIXI.settings.MIPMAP_TEXTURES = PIXI.MIPMAP_MODES.OFF



const shader = new PIXI.Filter("", fragmentGLSL, uniforms);
sprite.filters = [shader];

and i made a glsl nearest neighbor implementation (which i'm not sure if its correct)

precision mediump float;

varying vec2 vTextureCoord;
uniform sampler2D uSampler;

uniform vec2 textureSize;

//todo
uniform float scale;

vec4 nearest(sampler2D tex, vec2 texcoord)
{
	float x = floor(texcoord.x) / textureSize.x;
	float y = floor(texcoord.y) / textureSize.y;
	return texture2D(tex, vec2(x, y)); 
}

void main(){
  gl_FragColor = nearest(uSampler, vTextureCoord * textureSize);
}

 

When i run this, i see some differences compared to default interpolation, but especially when zoomed in (stage scale), the interpolation is not nearest but almost exactly as the default (linear), but with a slow misplacement of pixels. 

It seems to me that whatever i do in glsl, the interpolation is still affected by PIXI.settings.SCALE_MODE. I can't understand it. 

Am i missing something here?

thank you

[ivan.popelyshev]

Welcome to the forums!

> So instead of using PIXI.settings.SCALE_MODE, which doesn't support bi-cubic

what do you mean?

> i decided to create my own glsl interpolation filters for the sprites i use to display the images

Filters should be applied for collection of elements. Your approach with Filter wont work because Filter deals with FrameBuffer where sprite was already rendered, it deals with scaled version of sprite.

Please read about other way, for Spites. We discuss mesh-shader first and then I show that guy a batched shader: https://www.pixiplayground.com/#/edit/CMKvgOt-bvlCG4QHdswIP

 

You can also try set mipmapping for texture filtering: "texture.baseTexture.mipmap = PIXI.MIPMAP_MODES.POW2" (same as 1) , but it requires texture of POW2 size. 

If you want to work with non-pow2 texture, custom shader can work in case your scale is below 4, just use multiple "texture2D"s .

You can also hope that all your users have WebGL2 enabled and then use "PIXI.MIPMAP_MODES.ON". In that case it will use mipmaps even for non-pow2 textures in case of webgl2.

Edited April 4, 2020 by ivan.popelyshev

[d3bgger]

> So instead of using PIXI.settings.SCALE_MODE, which doesn't support bi-cubic

> what do you mean?

I see it only supports LINEAR and NEAREST, not bi-cubic (at least without mipmaps).

As for pow2, yes the textures and sprites i use can be any image, so that's not an option, but i think i'll always have webGL2, as i'm building for electron. I tried MIPMAP_MODES.ON and when scaling down the result is indeed like a bi-cubic interpolation! smoooth!

But still the main issue is that i need to switch between interpolations! When stage scale is > 2 i want nearest. 

I'll read the other post, and i hope i'll understand it

thanks for your help and the fast response!

 

[ivan.popelyshev]

When stage scale is > 2 i want nearest. 

scale>2 as in enlarge? That's MAGFILTER right?

If you know exact webgl constants you want to use , you can make your own texture resource and override "style" method in it, it'll be called instead of TextureSystem one. Here's basic custom texture resource: https://pixijs.io/examples/#/textures/gradient-resource.js . You should base your resource on ImageResource, i guess, and pass there image from original texture (texture.baseTexture.resource.source) . Or you can override "style" ,method  in particular resource. Unfortunately I dont ghave time to explain that but I see that you already have enough knowledge and can do that if you only read TextureSystem + resources code. https://github.com/pixijs/pixi.js/tree/dev/packages/core/src/textures

The idea is that we allow people to override texture management because PixiJS team is small and no way we can predict all stuff you guys want. But we do not have enough examples for it yet, and hope that community makes them.

[d3bgger]

thank you Ivan, i'll try to understand and research the process you describe.

If i make it work i'll be glad to submit it as an example if you think it's something useful for others.

[ivan.popelyshev]

You can post non-working demo here if you make one, I'll help

[d3bgger]

Ok so if i understand correctly, creating my own Texture or style override as you describe, will allow me to set TEXTURE_MAG_FILTER so to achieve nearest only for scaling up. 

But this again will also use nearest for scales between 1.0 and 2.0. I want linear between this magnification level. I only want nearest from 2.0 and up.

So i think that with this special requirement about scale, creating a fragment shader (and maybe vertex?) is the only way. But based on your first answer, i should not provide it as a filter to a sprite  but some other way? 

Maybe i should use a PIXI.SimplePlane and provide a shader there? 

thanks

[ivan.popelyshev]

Have you looked at https://www.pixiplayground.com/#/edit/CMKvgOt-bvlCG4QHdswIP ? If your problem is scale, you need webgl derivatives (dx dy), @eXponeta had examples for them.

[d3bgger]

Thats what i did and it works: https://www.pixiplayground.com/#/edit/316biDeqjtiiAVbUO6o7g

I hope i'm not doing anything wrong

thanks for the help!

[ivan.popelyshev]

Yes, that should work for WebGL2. Congrats!

[d3bgger]

I try to find out why alpha doesn't work in my use case. I saw that uColor should be used, as MeshMaterial seems to update this, but this doesn't produce the correct alpha (it just brightens the texture).

For testing, i created a MeshMaterial providing it with a Program with the default shaders  (https://github.com/pixijs/pixi.js/tree/master/packages/mesh/src/shader) and still the alpha isn't working (same result as using my shaders).

If on the other hand i create a MeshMaterial without passing a Program, then alpha works!

So any idea why MeshMaterial seems to not handle alpha, if a Program is passed (even the default)? 

thanks