Spritesheet example / support for instanced meshes?

[thrice]

Wondering if anyone has an example of using a spritesheet or texture atlas with instance meshes, or if this could be added to the roadmap as it seems like a pretty core/important feature? (Or if it isn't possible?) I've been having to do some pretty awful things in an attempt to work around the lack of support for that.

Currently all the examples of using a atlas/sheet that I've seen, do not work with instanced meshes, since when you createInstance on a mesh, it is using/must use the same material. So if you modify the (uOffset, vOffset, uScale, vScale), you are mutating the texture itself. - So I didn't think it was possible, but then I saw this post:

So I thought maybe it was possible. However it looks like if you setVerticesData of an instanced mesh, it mutates the source, playground below for example.

https://www.babylonjs-playground.com/#IZACWJ#6

So is it even possible to have instanced meshes with different verticesData to benefit from the merged draw calls or not?

--- ALSO: Does anyone happen to know if the math changed on setVerticesData calculations after babylon 2.3? I've copied code from the babylon-atlas library, but it's returning the wrong frame, and it's upside down (and the frame data is in the correct format). So hoping maybe just a simple change to get it working, but math is hard.

https://github.com/andyhall/babylon-atlas/blob/master/index.js#L198

 

 

[Guest]

Hello! so instances in babylon.js must share everything but the world matrix with the source mesh. In your case, you want to change UV so this will require specific work

The core engine will be able to deal with that but you will have to use a custom shader as our inbox materials only support WM changes between instances.

 

Does it make sense?

 

Regarding setVerticesData, the only thing I remember is that we switched parameter order to make it clearer

[thrice]

Hi Delta -- it makes sense in theory, however I am unsure how I would go about implementing/ tracking that. If I'm understanding correctly:

You are saying to use one shader material, shared by many instances. In that case, how would I let instance 1 know to use uOffset X and instance 2 use uOffset Y, for example? Because if I'm using a shader material, and I say for example,

material.setVector2('uvOffset', new BABYLON.Vector2(1,1)) 

or whatever, then I'm still going to end up modifying the shader material on all instances. Unless I am missing something?

(also, figured out the setVerticesData, I didn't copy over a false param from the constructor of texture as in library to invert it because I had no idea what it was doing (would love more named param constructors i.e. mesh builder style in future  ))

 

[thrice]

-- That said ^, I still think it would be really useful to have built in to the framework if it's possible, as an StandardInstancedMaterial or InstancedAtlasMaterial or something like that. Reasons being:

I tried using sprite manager, but aside from the fact that I couldn't use it anywhere where I wanted a non billboard mode mesh (which is probably 75% of the draw calls in my scene, i.e. cards which need to be able to rotate), there were too many issues. Different (worse) rendering quality when sprites are far from camera, having to track parent/position manually with onWorldMatrixUpdated, translating parent size to scaling which is a hack in itself, etc. But the main dealbreaker /reason I can't use it at all, in most places in my scene, is because of lack of rotation.

If there was a InstancedAtlasMaterial or whatever, it would be hugely beneficial as not only could my draw calls be reduced by a huge amount (I would estimate my on average 100 draw calls I could cut in half easily, and that's without caching what would be really large textures like my card images at all). But the biggest benefit would be having a consistent API to work with, i.e. not having to use a different mechanism entirely (i.e. SpriteManager, or gui texture), to optimize things.

also almost all of the examples I've seen people post of using Atlases/sprites with meshes (i.e. this library https://github.com/andyhall/babylon-atlas ) ,  seem fairly pointless to me (at least for a great many use cases, without something like ^), unless I am missing something. At least from my limited experience, drawCalls (in general), seem like the #1 key performance indicator for optimizing speed, and when using a library like the one above, there is a good chance you can end up with the same number of drawCalls, since you end up having to create a new mesh (or clone) for each use of the texture. (Which in almost all of my use cases, creating an instanced mesh with a new texture instead of an atlas, saves me way more draw calls than I would by using the atlas). -- Granted that's not taking into consideration complexity of the mesh being rendered, but in my scene 95% of my meshes are planes, and I haven't seen nearly the performance boosts from using clone+atlas compared to createInstance+nonAtlasTexture -- But point is, having support for this would allow for best of both world / total draw call annihilation.

[Guest]

From the point of view of a shader there is nothing to do to support instance except adding more attributes.

Here is for isntance how we get data from mesh instances in the default shaders: https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/instancesDeclaration.fx

As you can see, I'm just adding 4 new attributes to store a matrix per instance.

The difference is in the way these attributes are added to the mesh: https://github.com/BabylonJS/Babylon.js/blob/master/src/Mesh/babylon.mesh.ts#L1237

First you have to create a buffer to store the data: https://github.com/BabylonJS/Babylon.js/blob/master/src/Mesh/babylon.mesh.ts#L1234 and then you create vertex buffers on it.

 

So in your case nothing prevents us for adding a new option in this file: https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/instancesDeclaration.fx to store more data (like uv?)

 

Does it make sense?

[thrice]

Conceptually, it's starting to make more sense after looking into the source code.I need to read more about shader attributes as I don't think I really understood what they were doing until this point.

[jerome]

BTW, currently the spritesheet-like feature already works in the SPS that can be an alternative to instances : http://doc.babylonjs.com/how_to/solid_particle_system#colors-and-uvs