Sound analyse and visualization

[Zarko Kaktus]

 

Hi Everyone, 

 

I was working on a Phaser based website and as a part of the integration I needed to provide some objects interaction based on the song frequency data.

To achieve this I needed to write a new Phaser plugin.

This plugin allowed me to create a new extended type of Phaser.Sound object that provide real time sound data analyse.

 

And now I want to share this plugin with all of you .

 

Here you can view the demo http://zarkokaktus.byethost3.com/Phaser/sound_analyser/index.php

You can GIT clone/download it from here: https://github.com/KaktusFlame/phaser-sound-analyser

 

In the background, this object uses AnalyserNode from Web Audio API. This node is attached to Phaser main sound context.

If Phaser doesn't use context and Web Audio API then you can set this object to provide fake data and still have the visualization effect .

 

Here is some simple explanation for all four new extended objects you will have with this plugin: https://github.com/KaktusFlame/phaser-sound-analyser#about-objects-in-this-plugin

 

In short, this objects are in the following relationship:

• We need to create Phaser.SoundAnalyse object so we can get sound analyse update on every game thick (only if sound is in play state)
• We need to have graphical representation of this sound analyse data, and for that we will use Phaser.BitmapDataSoundAnalyze.
  We need to add this texture to Phaser.SoundAnalyse so on every sound analyse update this texture will be auto updated.
  We need to set this texture to sprite object so we can make it visible on screen
• If we want to have all in one visual representation of the sound, then we can create Phaser.SoundAnalyseSprite that will handle creation and linking of all other sound analyse objects for us

 

This is a simple demo how you can use it:

    // create new Phaser game    var game = new Phaser.Game(800, 600, Phaser.AUTO, 'phaser-example', { preload: preload, create: create });    // function called on game asset preload    function preload() {        // load SoundAnalyse plugin script file        game.load.script('soundAnalysePlugin', 'js/plugin/SoundAnalyse/SoundAnalyse.js');        // load song file        game.load.audio('song1', 'audio/All_Time_Low__Time_Bomb.mp3', true);    }    // function called after all assets are loaded    function create() {        // create plugin instance        var soundAnalyse = game.plugins.add(new Phaser.Plugin.SoundAnalyse(game));        // create new Sprite sound analyse object        var soundAnalyseSprite = soundAnalyse.add.soundAnalyseSprite(        20/*x*/, 200/*y*/, 530/*width*/, 200/*height*/, "song1"/*sound key*/, true/*auto play*/, this._onDecodeFinish/*decode finish callback*/, this/*callback context*/);        // show frequency domain chart bars (this is only one of eight other visualizations)        soundAnalyseSprite.showFrequencyDomainChartBars(true);    }

What do you thing about this plugin?

 

Some of the ideas you can achieve with this plugin (not included):

 - You can create water surface that dance by the music played in your game.

 - You can make sprite object act or do staff at some sound frequency

 - You can make other type of visualization, bubbles that shrink and squeeze by the sound amplitude data for example

 - And many more

[jmp909]

I look forward to playing with this . Thanks for sharing!

[ekeimaja]

Really nice plugin!

[Zarko Kaktus]

Thanks all for your comments!

 

Phaser is a great engine and with a great community, I am happy I have chance to share something that is creation and part of it's usability.

I will post a lot of new interesting staff in near future

[rich]

This is fantastic

[Taribuna]

Hello Zarko,

great work!
Do you have any plans on porting this plugin to Phaser 3? 

Thanks