playing with analyser : my first audio ribbon

[jerome]

http://www.babylonjs-playground.com/#1LN6FS

 

the same, more nervous : http://www.babylonjs-playground.com/#1LN6FS#3

[jerome]

another one with lines and Time Domain (sound curve) instead of Frequency data (volume per frequences) : http://www.babylonjs-playground.com/#1LN6FS#2

[NasimiAsl]

very useful thanks

[jerome]

ooopps

there was a little error

this one is more conform to what I expected : http://www.babylonjs-playground.com/#1LN6FS#4

[GameMonetize]

Huge!

[jerome]

and at last, a real 3D sonograph (sound curve) : http://www.babylonjs-playground.com/#1LN6FS#5

 

 

more readable with a bigger sample delay : http://www.babylonjs-playground.com/#1LN6FS#6

 

(wait for the music evolution)

[davrous]

Very very cool! :-D Thanks for sharing that. The last samples are very slow on my machine (4 fps) but are very interesting.

[jerome]

4 fps only ???

really surprising ...

 

I've got 60 fps in chrome on my little laptop (cpu : i5-4300, gpu: intel haswell mobile, os : ubuntu 14)

[jerome]

btw, I guess I will implement per path and per vertex color for the ribbons (in january). This will allow to dynamically generate really nice shapes (landscapes, charts, analysers, etc)

[dbawel]

Sometimes I hate you guys - as when I see scenes like jerome has just built and posted, I want to stop what I'm doing, and take time out of my day to play with the new code and build something visually different and stunning - which I just don't have time for. I suppose I'm saying I enjoy developing using this framework too much, and want to make cooler stuff than what I need to build in my regular daily work. So I guess I'm jealous of those who can find the time to build cool stuff - but certainly greatful to be able to take a break to see what others are having fun with.

 

@jerome - very cool! 

[jerome]

still 60 fps : http://www.babylonjs-playground.com/#1LN6FS#9

iiihhhhaaaaa... I'm flying a F-22 under the music mountains !

[jerome]

@dbawel : I often feel the same about the others' scenes 

They all look so stunning, amazing and I say to myself : if only I could have the time to learn all the BJS tricks and do the same !

[dbawel]

@jerome - This last scene is awesome! Best yet, and I can't stop starring at the screen.

 

However, I'm not able to run the scene in the playground on my Android tablet. It is a Sony Xperia Z2 with a quad core 2.3 Ghz processor and 16Gb Ram, with and additional 2Gb video RAM for graphics processing. It is updated to the latest Android OS lolipop 5.1.1, and I've installed all available updates. This is not the very fastest Android tablet you can buy today, but it was a few months ago and is still faster than 95% of the tablets on the market - although I know this isn't about speed, but wanted to met you know that if the scene was going to play on Android, then this tablet woud certainly have no barriers to play it.

 

What happens is that if I load the playground scene, there is no audio or animation at all - which is expected since you don't appear to be triggering the playing of the audio file which is necessary for Android tablets. However, when I press "run", the audio begins to play, but there is no deformation (animation) of the ribbon. I thought I'd report this to you, although perhaps you didn't intend on the scene playing on mobile. However, in reading your script, I would expect the ribbon to animate, although as mentioned, I can see why the audio doesn't initiate play upon first load.

 

FYI.

 

DB

[jerome]

thanks for your reports

actually I didn't intend to make it run on many devices, I'm just learning how the analyser works, this is just an intellectual game

 

I just tested on my very old laptop (10 years), this runs at 53 fps with chrome !

 

Only the last link has a single sided ribbon (so twice less vertices) with no backfaceculling to improve the perfs.

The number of paths can be lowered : sonoSize at the line 23

and tweak the z scaling if you still want a long landscape (l 46)

 

The same about the fft (the sample rate, l 24) so the ribbon width, just use multiples of 64

[jerome]

Just can't stop playing with this : http://www.babylonjs-playground.com/#1LN6FS#12

 

a SPS fountain and audio analyser

 

Wait for the beat to start please (D-J)

[dbawel]

@ jerome,

 

Again, I'm impressed with the code driving the animation of the ribbon. However, I wasn't aware that using back face culling would require much additional resources, and possibly lower the rendering framerate. I assumed as I've experienced with other renderers in years past, that setting back face culling to true has generally improved performance, as there is less calculations of surfaces (polygons). So I would be veru interested in how back face culling might decrease the rendering performance and framerate - unless perhaps you're speaking of setting this to "false" - allowing both sides of the ribbon to be rendered.

 

Please let me know any experience and/or benchmark scenes where there is a reduction in performance in using back face culling to set to either true or false. Although I would expect setting to false would likely require more CPU or GPU operations, but if you are saying that using this in any case will recease performance, I'm very interested in understanding these cases further.

 

Thanks,

 

DB

[jerome]

short answer (because it's late here) :

 

If you set the parameter sideOrientation to DOUBLESIDE, the mesh is built with twice more vertices and indices than the same standard mesh (FRONTSIDE by default). So it's textured and reflects the light both sides. It's visible from both sides too, even if backfaceculling is true (no need to set it to false to see it from behind).

The computation to update the ribbon geometry is done in JS, CPU side.

So twice more vertices, twice more calculations, twice more data to pass to the GPU each frame.

 

In brief, doubleside vs frontside has a real performance impact on big meshes or if you need the CPU for something else in the same time.

 

Backfaceculling, or not, is only computed GPU side. I know it can have an effect on the performance for very big numbers of faces (I just can't retrieve a morphing ribbon demo with 50K vertices where I could feel, a little bit, the difference between w and w/o backfaceculling) but I guess the gain is really poor unless you deal with very specific huge meshes.

 

In my example, the real gain was to go from plain DOUBLESIDE+bckfcck to FRONTSIDE+no bckfccl : geometry size to compute (including normals) divided by two, ribbon still visible from behind, but no light reflection backside

[dbawel]

Hi jerome,

 

That's what I expected, as I wasn't sure how you were refering to back face culling. If you have time, I believe it would be very valuable to provide your definitions and implications of CPU usage if you define an object to render both sides. I haven't looked at the documentation for backfaceculling, however, I doubt it is as thorough as you have explained on this post. And, I doubt most people are aware of what this is exactly - and I feel it is essential for anyone working in graphics and rendering to know what this represents and how it is calculated in babylon.js specifically. If you don't hve time to write a good definition for back face culling, let me know and I will take the time to do so, as I have many years of experience in rendering.

 

Again, thanks for informing me of how babylon.js clculates back faces and normals - it appears very inefficient to me based upon your info and my experience with other renderers. But perhaps this is simply a limitation within WebGL.

 

DB

[jerome]

Hi Daniel,

 

I don't really know how the backfaceculling process works GPU side (I know what it does, but not how), so your experience on this topic might be far more pertinent than mine 

 

About the parameter sideOrientation, and as I coded the method, I can explain what it does what is quite simple :

If you opt for FRONTSIDE (default value), nothing is done

If you opt for BASCKSIDE, the mesh indices are re-sorted in the reverse order and the normal are inverted

if you opt for DOUBLESIDE, a backside set of positions/indices/normals is computed and added to the initial set of the mesh

 

This method is called by MeshBuilder.CreateXXX() just after the mesh geometry construction process

 

https://github.com/BabylonJS/Babylon.js/blob/master/src/Mesh/babylon.mesh.vertexData.ts#L2035

[Stvsynrj]

@jerome : sorry, can't resist

 

http://www.babylonjs-playground.com/#1LN6FS#13

[jerome]

don't resist, it's wonderful