Get world coordinates in Filter?

[TobiasW]

I'm currently trying to find a way to render global-coordinate-based content with a shader using an AnimatedSprite as a mask.

Since I have no idea how I'd write my own AnimatedSprite shader I thought the easiest would be to write a Filter. After a bit of looking around, I was mostly able to use a Filter to have the AnimatedSprite work as a mask for another texture, but I haven't found a way to access the global coordinates in my Filter fragment shader or vertex shader.

Attached is an image that hopefully illustrates a little what I am looking for. The colorful background is repeating global-coordinate-based UV coordinates (not screen based - I can scroll and zoom around and the background moves with the grey blocks) rendered by another shader. The white parts are my AnimatedSprites, and I'm looking for a way to make the white parts of those AnimatedSprites look the same as that background.

(In the end I'm looking to make the white overlay half-transparent so it looks like those grey blocks are transparent at the bottom and the background is shining through. I can't *actually* make them transparent because I only want the background to shine through, not other grey blocks that overlap with them at the edges. And apart from that there's global-coordinate-based distortion I'm planning so I really need global coordinates one way or another.)

Edited August 13, 2021 by TobiasW

[ivan.popelyshev]

> The colorful background is repeating global-coordinate-based UV coordinate

https://github.com/pixijs/pixijs/wiki/v5-Creating-filters#conversion-functions

but for screen you need something else, you have to pass world container transform in a uniform and translate this thing.

> Since I have no idea how I'd write my own AnimatedSprite shader I thought the easiest would be to write a Filter.

no, the easiest is to write mesh-shader https://pixijs.io/examples/#/mesh-and-shaders/shadertoy-mesh.js . You can copy animated sprite code regarding textures and strip to your mesh, its just changing texture, its not difficult  

The only problem is texture frame, and you can get it through texture matrix uniforms

"texture.uvMatrix = new PIXI.TextureMatrix(texture)" , call update() sometimes, put it in uniforms like here: https://github.com/pixijs/pixi-heaven/blob/eb554499afe3fcbb42bb4d4f77edb2ee0c27b7fa/src/twotint/mesh/DoubleTintMeshMaterial.ts#L179

in vertex shader: https://github.com/pixijs/pixi-heaven/blob/eb554499afe3fcbb42bb4d4f77edb2ee0c27b7fa/src/twotint/mesh/DoubleTintMeshMaterial.ts#L20

clamp in fragment: clamp (i use discard, but better clamp it): https://github.com/pixijs/pixi-heaven/blob/eb554499afe3fcbb42bb4d4f77edb2ee0c27b7fa/src/twotint/mesh/DoubleTintMeshMaterial.ts#L46

=============

As for sprites, we dont have shortcuts for those kind of elements yet, there are batch renderers which a a bit different - its hard to supply uniforms to them. No docs, nothing, only pixi-batch-renderer plugin example and pixi-heaven two-tint batch renderer.

===========

Why there are no docs on it? Well, if only someone could do it... If, for example, I could just type a doc instead - it would be , but i have to commit stuff, push it, negotiate, someone has to proof-read it, and i already have 10 plugins on me.

Please, if you make good solution there - share it with people.

P.S. filters for tiles will be slow as heck because they use extra framebuffer operations.

Edited August 12, 2021 by ivan.popelyshev

[TobiasW]

Ah, I see! I was kind of hung up on "The batching system is too complicated for me", but I guess making my own non-batching AnimatedSprite shader shouldn't be too hard. Thanks for the ideas and the pointers here!

One more possibly related question: If I want two meshes to have the same shader, but different uniforms, do I need to make two Shader instances? Or is there a way to share a Shader despite differences in uniforms?

[ivan.popelyshev]

yes, different Shader instances, but they have same program. Shader = link to program + uniforms

Two ways to achieve that:

1. vertexSrc fragmentSrc passed to Shader.from() are identical for both shaders

2. look in Shader.from - it has Program.from call. Just do it yourself, and then create shaders with new Shader(program,..). 

You can also extend MeshMaterial (which is a shader) and pass program in options.

====

even more, you can use same uniforms in those shaders if you want - just create UniformGroup , and pass it to both as an object inside "uniforms", that way you can modify group and it'll affect both

Edited August 13, 2021 by ivan.popelyshev

[TobiasW]

Ah, gotcha! I'm coming from Unity where it's "Material = Shader + Uniforms", so I was looking in the wrong places here. Thanks again for your help!

[TobiasW]

Hmm, but regarding performance... if I have to always alternate between rendering the Sprite for the block and my SpecialShaderAnimatedSpriteMesh, that would break all the batching of the Sprites, right? And sadly it gets even more complicated like this, because each tile can have multiple layers that obscure each other.

So in the end would I have to render all tiles layers (and objects on the tiles) with one self-made batch renderer after all, with e.g. vertex colors deciding whether it is a normal animation or my special overlay shader?

(And I assume filters would also break batching?)

Edited August 13, 2021 by TobiasW

[ivan.popelyshev]

filters aren't just breaking batching, they use temporary framebuffer to do layering.

Tell me how big is your shader , what it does and i'll tell you what is better to do in situation. There are too many alternatives  For example you can take https://github.com/pixijs/tilemap as a base, ctrl+c it, put your shader there, with extra attributes / uniforms that you need. Its just a mesh with many tiles inside. Actually, that's a good exmaple of how to put many things in same mesh, because , really, tiles are usually static.

[TobiasW]

I'll briefly explain the background shader first: It's water shader. It takes a repeating texture, draws it over the whole background and distorts its UV coordinates with running waves. Later it'll also draw some oil spill textures in certain positions.

Over that water background I draw my (sometimes animated) blocks. Those blocks should be halfway in transparent water though, and that's where the new shader comes in.

This new shader takes a mask (the part of the block that is in transparent water), and the does exactly the same as my background shader again: It renders the ocean, this time over the already renderer block, transparently where the mask is. It also shifts the UV coordinates up and down to emulate the motion of running waves, so my geometry will be a bit bigger than the mask.

There'll also be a lot of other things that need to be drawn, like decoration on the blocks or NPCs. It seems like a pretty daunting task to optimize, frankly. We don't need to optimize *yet*, but I'm already looking to find out what the challenges and options are.

----------

Ah, so... is tilemap just like a manual batch renderer for sprites, not just specifically for tilemaps? It's probably not well fit for our needs (we have animated isometric tiles on a lot of layers, and players/NPCs moving between layers), but it might be a great example, yeah!