Slow instance performace -- Equivalent of threejs' InstancedBufferGeometry in BabylonJS?

[Nockawa]

I totally understand. My advice, keep it simple for now and polish when you will really need it.

Internally the GPU drivers already try to do everything to prevent wait due to lock/update of data. You can certainly tune things up, but considering the wide range of drivers if you target everything WebGL is running on, I'm not sure you'll win every time.

And as you said, WebGL 2.0 may certainly provide you the tools to do a faster rendering.

Last point: all your instanced are potentially updatable? Because if you can make two set of SPS, one with the static ones and another one that you update all the time, you would speed things up for sure...Now if it requires to update the static one once in a world, you'd still win out of it.

[BeanstalkBlue]

Thanks for that insight Nockawa.

Playing with 100 particle SPS and updating one SPS per frame, it seems like this might have acceptable performance for now.

The world will be made of say 100 million solid particles (or some large amount), probably represented by heightmap textures that stream to the browser as PNG images. Then I can only display say 3000 particles at a time as the user moves through the world. So I must partition the world into some number of particle systems, and load more systems into memory as the user moves around.

This is basically the problem.

Most likely something like a transform feedback particle system that reads some vertex height data from a texture is how things will end up, a year from now.

[Nockawa]

That's a very interesting project, I will follow your blog to get the update. Looking at what you want to achieve, I would suggest you to implement your own caching system of SPS, with a MRU/Gargabe collector kind of algorithm and to have X instance of the SPS at runtime with X being setup based on the available memory. It reminds me a project I did in the past, when you're dealing with such volumetry it usually works well.

[jerome]

I guess that DK meant sps.mesh.setEnabled(false)

The right question is : what do you intend to do exactly ?

You seemed to want to use something like a 3js bufferGeometry what is not dynamically updatable at the subpart level and you talked about a static height field from what I initially understood. Maybe I'm wrong. But I don't really get what you really want to achieve. Please let us know and we'll enjoy to drive you in the right direction.

Not sure you need more than one SPS.

The SPS allows you to update one particle, several particles, all the particles at will : on some event or condition, each frame (milllions vertices each frame will probably be too much for the CPU anyway), or when you want.

The philosophy is to call setParticles() to update the particles, or setParticles(first, last) to update them only from the first one to the last one. This can be called when you want, once or as many time you want.

What does it do ? it gets each particle logical status (position, rotation, color, uvs, bbox, etc) and computes it into the SPS global mesh geometry, updates this one for the next rendering.

You can set each particle logical status by your own, just by accessing to sps.particles ... or better : setParticles() calls for you, within its loop the function sps.updateParticle(p)

So can put all your particle logic in this function directly, this is the fastest way to do things

 Please have a read at this : http://doc.babylonjs.com/overviews/solid_particle_system  and let us know what is not clear for you.

Check also these demos :

- 3000 spherical (well, morphed) drops in a SPS colliding another SPS (torus knots) http://www.babylonjs.com/Demos/Facets/

- 1200 different polyhedrons (SPS) colliding a sphere mesh : http://www.babylonjs.com/Demos/SPSCollisions/

- 80K random buildings : http://www.babylonjs-playground.com/#2FPT1A#36

- 20K boxes and only some moving : http://www.babylonjs-playground.com/#16SA9C#9

- asteroids (immutable SPS) : http://jerome.bousquie.fr/BJS/demos/solidparticlesAsteroid.html or http://www.babylonjs.com/Demos/SPS/

- 5000 updated textured boxes : http://www.babylonjs-playground.com/#2FPT1A#91

 

A user named chicagobob123  did a project with 15000 container trucks some months ago (can't retrieve his demo). I know Iiceman and Temechon did also pro projects about hexagonal tiled terrains with the SPS some time ago. 

[BeanstalkBlue]

@jerome Well, I am pretty sure at a minimum I will have to divide the scene into several SPS meshes.

Something like this:

http://www.babylonjs-playground.com/#VHJYX#5

Performance of updating 100 particles in a single 5000 particle SPS is much worse than updating 100 particles in a 100 particle SPS (that playground I linked makes it easy to test that).

I think @Nockawa has the right idea, allocate a bunch of SPS instances and then update their vertex data as the user moves through the world.

Another consideration is that if you only use one SPS then you have a single mesh and can only have maybe 3 point lights in the whole scene (all drawn on that mesh). If you have 10 meshes visible on the screen then you can have max 3*10=30 point lights visible for similar performance cost (ignoring shadows).

I hope so anyway, if Babylon lets me specify which lights shade each mesh.

Another consideration is that if you only have one mesh then you can't really force the particles to be drawn in front to back order, so in some cases you will have a lot of overfill, I guess unless you do a depth only pre-pass or something.

Is there a deferred lighting path in Babylon? If not then I think many meshes is the only way.

9 hours ago, jerome said:

I know Iiceman and Temechon did also pro projects about hexagonal tiled terrains with the SPS some time ago. 

I think I found one of the projects you are talking about https://github.com/Temechon/bursion thanks for the tip!

 

Quote

I guess that DK meant sps.mesh.setEnabled(false)

Oh! haha, thanks 

[Nockawa]

Yep, you should deal with a cluster of virtual SPS, and allocated a fix amount of SPS and use/refresh them when needed.

There's not deferred shading path to this day, mostly because doing so would require the WebGL Extension to render multiple target in one pass, if you don't have this extension, deferred shading is pretty much out of the question.

This is something we talk internally from time to time but we all agree it's something to consider when WebGL 2.0 will be majorly spread.

[jerome]

Hi,

I still don't know what you're trying to achieve, I mean in concrete terms, something that could start from a PG prototype, so it's not easy to tell that one SPS is better than 10, or the contrary.

Quote

Performance of updating 100 particles in a single 5000 particle SPS is much worse than updating 100 particles in a 100 particle SPS (that playground I linked makes it easy to test that).

Wrong ... here's a very simple code using updateParticle() like explained in the doc : 10K cylinders and 200 of them updated per frame at 60 fps

http://www.babylonjs-playground.com/#VHJYX#6

 

1000 updated / 10K : http://www.babylonjs-playground.com/#VHJYX#7

3000 / 10K :  http://www.babylonjs-playground.com/#VHJYX#8

The error in your code is that you manually re-iterate over the whole pool of particles when setParticles()/updateParticle() already do it for you.

Please take the time to read the doc, everything is explained.

Temechon had another project with thousands of trees in a landscape, I can't remember the link

[Temechon]

Indeed, but not available online as it's under heavy work  But yes, I use SPS to render tiles, trees and stone on a map (~100*100 tiles)

[BeanstalkBlue]

Thanks for that correction @jerome, I see the error.

Your code seems to run slower than mine though (but maybe I am not benchmarking well). Try making both the same particle and update counts, but use larger numbers so that they aren't both at 60fps for you.

 

Anyway it doesnt matter because there will still be the lighting problem and draw ordering problem I mentioned in a forward rendered setup with only 1 mesh.

 

What I am specifically trying to achieve is that there is a game world that is a height map, visualized by these particles (like a terrain, but made of particles). The player can walk through the game world. The world is e.g. 10 million particles large. The player can only see e.g. 3000 particles at a time (his local part of the world). There will be various small point lights throughout the world e.g. torches that light paths.

 

So my idea was that you divide the world into tiles. 1 tile = 1 SPS. As the player moves through the world, you load world data into offscreen SPS and then make that SPS tile visible as the player moves toward it.

To reduce fill rate you sort the SPS tiles by distance from the camera, and draw close tiles first.

To allow 20 point lights on the screen at once, you assign 2 point lights to each SPS tile mesh.

 

I am not sure if I am explaining well. Am I making sense?

[jerome]

Understood ...

What I would do personally :

- to have a chuncked logical representation of the whole world (its geometry)

- to have a single big SPS displaying only what is visible on the screen, say 3000 particles

- to make the logical world "slide" under the SPS in function of the user movement, to pass it the current visible data from the current chunk and to update its particle according to these current data

Well, this is just my point of view and I'm quite sure to keep a decent FPS, to achieve to manage it with a SPS and deal with huge amount of memory required from your massive data

[BeanstalkBlue]

But this single SPS would only allow 3 point lights on the whole mesh, instead of say 30 point lights if there are 10 meshes?

Also if there are only 3 point lights covering 10% of the pixels then 10 meshes runs a lot faster because the 3 point light shader doesn't run on all the meshes.

 

Right?

[jerome]

why only 3 point lights ?

 

[GameMonetize]

By the way I read you blog (https://blog.scarlet6.com/babylon/), this is really unfair and pretty unkind (mostly when you talked about something you did not really know at the time of writing)

[BeanstalkBlue]

43 minutes ago, jerome said:

why only 3 point lights ?

 

Because if you have only 1 SPS mesh then if a point light affects only a single particle in the mesh, you still have to use the point light shader for every particle in the mesh.

If you have 10 SPS meshes and a point light affects only a single particle, then you use a 0-point-light shader on 9 meshes and a 1-point-light shader on only 1 mesh.

If you try to have 30 point lights with 1 SPS, then you need a 30 point light shader for your entire SPS? Or can you specify a different shader for different groups of particles?

[BeanstalkBlue]

3 minutes ago, Deltakosh said:

By the way I read you blog (https://blog.scarlet6.com/babylon/), this is really unfair and pretty unkind (mostly when you talked about something you did not really know)

Yikes, sorry I didn't mean to offend!

In my latest post I mentioned that the community here is really great. I'll try to modify the blog to reflect the positive experience I'm having here.

[GameMonetize]

It is your blog  I don't want to control what you write but it seems that the general tone of your blog does not reflect the experience you seem to have here.

[BeanstalkBlue]

@Deltakosh I agree with you though, so I have updated that post: https://blog.scarlet6.com/babylon/ The post was meant to be more about my own mistakes, not problems that Babylon has hah

[BeanstalkBlue]

Is it possible to apply a different shader to different particles in a Solid Particle System? If not that is ok, I am just deciding how many SPS instances I actually need.

[jerome]

nope, you can't. The SPS is a standard BJS mesh.

[GameMonetize]

@BeanstalkBlue thank you I appreciate the changes