Have a Three js app considering converting

[chicagobob123]

I started an app about 4 months ago in three.js as a proof of concept. 

Its not a game but a ship yard inventory system. It maintains the position of over 20,000 containers in 3 basic sizes, and reflects the movement of over 200 vehicles with 5 different types. The prototype app positions the vehicles and the moves the containers around the yard.

The vehicles and containers are using models in the js format. The containers are wrapped in about 15 different textures that have the company logo in 3 basic sizes and  along with a couple default textures when we don't have the a matching company logo. 

Now the problems. 

With Three.js I have noticed massive memory leak that I think is caused by the adding and removal of containers. For performance I placed all the containers in just a few concatenated pieces of Geometry to reduce the draw calls. The vehicles I left alone. To remove a container I modify the geometry on the fly. That being said, not sure if I should even do it that way and no place just to ask questions like can I do this, or does this work for three js. The other issue I have is it seems to be changing versions with internal alterations. As my code becomes larger modifying it becomes more difficult. 

Then I did a search on three.js vs ? and found Babylon read a few posts about it being more industrial strength and seeing a board that actually discusses the language I am more intrigued.

Does it make sense to change? Is the language similar or different.  I saw the multi-thousand spheres but they had no textures is it possible to apply image textures from an array. Does babylon have a js loader for models and png loaders for textures. Is there light that is like sunlight? 

Feel free to ask questions etc. 

thanks

 

 

[GameMonetize]

hello and welcome!

Language is the same: Still good ol' Javascript. But we also provide Typescript if you're more into more modern languages

Regarding your project I see several options:

- You can rely on instances (if your number of similar containers is high)

- You can use our ubber powerful SPS. Documentation here and probably a good example for you: http://www.babylonjs-playground.com/#1X7SUN#7

- We also support cloning (which is similar to sharing geometries in 3.js)

 

Whatever option you'll choose, you'll always find someone here to help you

[chicagobob123]

Thanks so much for your reply. Instances sounds interesting. Can each instance be altered by an indexed texture? Like here is a box it has texture 7 here is another box it has texture 2. My thinking is texture 7 and 2 provide me with different logos. 

Sorry, I will try to do some reading on how this stuff works and again thanks for the reply

[GameMonetize]

Instances are constraint to use exactly the same material but you can create two groups of instances: one for logo 1 and one for logo 2

[MasterSplinter]

Lol @Deltakosh

 

It's more organized and less buggy than three.  Although the assets make the repo huge

[GameMonetize]

Assets are on a different repo now

[Kesshi]

One thing you have to keep in mind if you do a conversion is that BabylonJs uses a left handed coordinate system and ThreeJs uses a right handed system. That means you may have to convert your models or otherwise they look mirrored because the z-axis is inverted.
I wrote a custom loader for our models and doing the conversion from right to left directly while loading.

[jerome]

The SPS (solid particle system) allows to have a different texture per particle, so per container in your case ... from the same single texture image file.

It will generate only one draw call. It can be seen as the equivalent of the buffer geometry in threeJS, although it provides far more methods than the buffer geometry.

[chicagobob123]

In reading the replies I dont know what repo is? 

.Also " from the same single texture image file. " Is there an example on how to create a texture map for a particle and access its components parts ?

I am going to have a go at it and convert. First the replies here are so refreshing and its fantastic to have people familiar with the code. This is sorely lacking in three.js, Just starting out at this its imperative to be able to get some sort of leg  up on what to use. 

I love the idea of using a Solid Particle system and only one draw call. The bulk of the containers are static.

About 5 every few seconds containers are either removed or added. I imagine the particle has a life and it can be removed this way. 

Thanks for everything so far this is most encouraging. 

 

[JohnK]

repo - repository (on github)

[jerome]

you can first read this about the SPS : http://doc.babylonjs.com/overviews/Solid_Particle_System

check out the online examples in the text

You can't add or remove the particles once it's created, but you can make them invisible by scaling them to zero. You have to generate the SPS with the final number of containers.

BTW, I will implement the per particle visibility so it will be straight forward for users.

Note that currently you can set a texture per particle but the way the texture applies to each particle is defined by the particle model., maybe a box in your case

To set each particle texture, have a read please to this part : http://doc.babylonjs.com/overviews/Solid_Particle_System#uvs

 

here's a very old example when SPS was still a prototype : http://www.babylonjs-playground.com/#2KSQ1R#38

The principle hasn't changed

the texture atlas used is this one : 

[jerome]

the SPS per particle visibility is just PRed

I'll do a little post to announce it

[jerome]

just my two cents about the initial post :

- We are here numerous people that have switched from ThreeJs to BabylonJs and noone of us did it with regret. You could search this forum about this topic, you'll see sometimes these testimonials.

- 3Js, BJS, or any other WebGL framework, they all are just WebGL. And I think that 20 000 solids on the screen is still a huge geometry for WebGL if you need to compute and update them each frame.

I'm afraid some optimizations will be required, some leads to think about :

do you need to update 20K meshes each frame ?

do you need to display 20K meshes ? the user won't make the difference if there are only 8K, 12K or 15K but the CPU will !

 

BJS instances or the SPS are good tools to lower the number of draw calls and to not reallocate the memory (so no GC activity), but they can't override the current JS/WebGL limitations ;-)

[chicagobob123]

do you need to update 20K meshes each frame ?

No, MOST do not move. There are only a few that do. (about 50 in motion at any given time)

The fact that particles can not be deleted or added to worries me a bit. Not sure how to work around that. 

do you need to display 20K meshes ?

It depends on the field of view which can be all or some. Think of it like google earth for a shipping yard.

FYI here is how I beat this in 3js by using some tricks.

1) I know that every yard has a physical size. So I subdivided the containers into large groups by location. 

2) I knew that I would only use so many corporate logo textures for the containers, so I created an array of them. The Meshes in 3js can use the index into that array. 

3) I merged the geometry of all containers groups, in 3js when you do this you can give it an index to the array of textures.

4) That created a group of large meshes from the geometry passing it the textures as well. 

5) Knowing exactly how many Faces and Vertices and FaceVertexUvs I modified the cached version of the mesh geometry

and the clone option to delete or add container geometry.  Then simply replaced it when something was modified. 

I get 30 to 60 fps without issue and only have to endure 25 to 50 draw calls on the large meshs.

Memory usage is insane.

I tried the SPS example in the playground raising the number of primitives to 20,000 total, that did NOT go well. I am not sure if its the loop that is holding it back? Or the 

actual amount of information. Going to check that next. 

Worse comes to worse I am hoping there is something here that can accommodate what I did last.

I love community and users which I hope to be more a part of once I can do more than draw a sphere.

 

[jerome]

waaaoow... you pushed the SPS to 20K and it worked as you expected ?

cool

with such a huge mesh (the SPS is a single mesh) if you don't call setParticles() every frame or if you don't update all the particles each frame, this should do the job

note that the solid particle can now be set to invisible

well, I would suggest for instance to use less solids anyway and to use what we call the level of detail (LOD) when the camera goes far away so when the field of view increases : http://doc.babylonjs.com/tutorials/How_to_use_LOD

and http://doc.babylonjs.com/tutorials/In-Browser_Mesh_Simplification_%28Auto-LOD%29

maybe could find also some ideas from this forum topic :

 

[chicagobob123]

SORRY that was a misquote. that did NOT go well. It bogged down and went slow. 

[jerome]

seems quite normal then regarding the number of instances

 

fyi : this one runs at 60 fps with 8K boxes in chrome on my computer : http://www.babylonjs-playground.com/#16SA9C#3

if I use particle invisibility for some of them (not yet in the PG) and even when adding shadows casted to the walls, I can run more than 10K boxes

 

[chicagobob123]

I will be trying some more stuff today. 

Thanks

[chicagobob123]

I modified your code to based on what someone else mentioned.

I created 3 particle systems at 8K each. 8K particles * 3 systems = 24,000 boxes and it

http://www.babylonjs-playground.com/#16SA9C#7

I only moved 8K of the particles. Which is way more than I would do in a normal world. 

 

[GameMonetize]

Works well here

[jerome]

if some of the sps are to never be updated, you can set them as immutable, this will still increase the perfs : http://doc.babylonjs.com/overviews/Solid_Particle_System#create-an-immutable-sps

but in this case, their particle won't move, neither change once created

[chicagobob123]

" if some of the sps are to never be updated. " Sadly that is not the case.

Although my mind is having all kind of thoughts. Maybe one particle set could be trees and or lamp posts.  Those would never move. Knowing that you can stitch together several of these creates a lot of possibilities. 

But as they say, "Brick by Brick". I now have to go back see if I can load a texture and give a pretty hard think on how to load the textures for the containers.

( I saw you can have multiple textures if you stitched them together into one texture and use the percentage of the UV. Thanks Jerome. )

 

[jerome]

another approach (what could be combined with yours) is to use the start and end parameters of setParticles(start, end, update) : http://doc.babylonjs.com/overviews/Solid_Particle_System#start-and-end-indexes-for-setparticles

Example : here are 20K particles in a single SPS, but each frame only the 100 to 500-th particles are updated http://www.babylonjs-playground.com/#16SA9C#9

the loop is done only on this subpart, so faster, but the whole VBO is still passed to the GPU

of course, you can choose to update only a given i-th particle by calling setParticles(i, i+1, true)

and you can also call setParticles() many times, for each wanted particle to update

setParticles(i, i+1, false); // updates only the i-th logical particle and not the VBO

setParticles(j, j+1, false); // same for the logical j-th

setParticles(n, n+1, true); // finally updates the logical n-th and updates the VBO only on this last call

[aWeirdo]

I know i'm late in this thread, but i'd like to add,

That babylonJs beats threeJs on many points, 

not just preformance, but up-to-date docs, examples and tutorials aswell, together with an active community always ready to help out where they can.

if you download the latest threeJs and attempt to run one of their examples locally, it'll more than likely error out and drasticly slow down the learning process of the framework, because they are not kept updated, and online, they often run on an older version of threeJs than what you download from their repo, the same goes for their docs, often outdated, and again just slowing down the learning process even more because you have to waste time looking around trying to find the correct method of doing something.

 

Where babylonJs's examples are run in the playground,  which uses the latest version of babylonJs and therefore so does the examples, and are more likely than not, also backwards compatible.

The doc's are ordered by version's and updated with any changes made & features added when a new  "full" version is released.e.g. 2.3, the current "master" version and 2.2, the previous master version. 

Furtheremore, i simply find babylonJs well.. simpler and easier to work with.

 

Feel free to correct me if i am wrong on any of my points, this is purely from personal experience with the two frameworks.

[jerome]

@chicagobob :  about your containers and SPS as I guess you are still investigating.

Assuming, you've got thousands of containers but only dozens are updated each frame, what would I  do would be to create two SPS :

- one with thousands particles, this one would be updated just on demand (not each frame) with all the logical optimizations available : target only the wanted particles in the array, enable/disable on demand the texture computations, etc

- another one with dozens of particles (the per frame updated containers)

then, it's up to you to pretend that some containers from the big SPS go to the small one and vice versa by switching the scales, textures, positions from the particles entering/leaving each SPS. This would just be a logical operation, faking that a particle is swapping from a pool to another one.