Wave Synth sound engine

[sssurikov]

I'm going to add a new synthesiser engine in Babylon.js.

 

- engine can play sound of hundred instruments and drums

- you can change any song parts in realtime

- engine is compatible with mobile browsers

 

see demo at

http://molgav.nn.ru/babylonsynth/

 

Any feature requests are welcome.

[totor]

hey, i have that vinyl!

is it limited to babylonjs or can we expect a more general js sound engine? With latest firefox, i have TypeError: SIMD.float32x4.loadXYZ is not a function and nothing more.

[RaananW]

That's an awesome demo!

 

Care to elaborate a bit more on how you actually do it? Is it real live synthesis, or audio loops running in the background?

[Dad72]

woaw. it's great all that sound. an orchestra. I like.

[sssurikov]

is it limited to babylonjs or can we expect a more general js sound engine?

 

 

SSSynthesiser.js has no dependencies. See non 3D examples at http://www.reddit.com/r/molgav/

 

With latest firefox, i have TypeError: SIMD.float32x4.loadXYZ is not a function

 

 

I need more testing. Send me log from browser console

 

Is it real live synthesis, or audio loops running in the background?

 

 

This is real live synthesis

[jerome]

EX - CE - LLENT !

[JCPalmer]

As far as the Firefox error.  Is this while running Firefox Nightly?  Firefox production does not have SIMD & did not error for me.  Nightly is not working right now for SIMD, for me at least.  Works on Android 5.  Not working on iOS 8. Looks like a picking issue.  Can drag around scene, but cannot click speakers on.

 

What are you doing as far as integration?  "Sounds" like it existed prior.  Are you planning to integrate with the sound API?

 

I have played with similar system on Java.  From Java 6 to Java 7, they switched codebases.  The one on Java 6 was proprietary, but I thought it sounded better.  Does list of instruments look familiar?

0 Acoustic Grand Piano1 Bright Acoustic Pian2 Electric Grand Piano3 Honky-tonk Piano4 Electric Piano 15 Electric Piano 26 Harpsichord7 Clavi8 Celesta9 Glockenspiel10 Music Box11 Vibraphone12 Marimba13 Xylophone14 Tubular Bells15 Dulcimer16 Drawbar Organ17 Percussive Organ18 Rock Organ19 Church Organ20 Reed Organ21 Accordion22 Harmonica23 Tango Accordion24 Acoustic Guitar (nyl25 Acoustic Guitar (ste26 Electric Guitar (jaz27 Electric Guitar (cle28 Electric Guitar (mut29 Overdriven Guitar30 Distortion Guitar31 Guitar harmonics32 Acoustic Bass33 Electric Bass (finge34 Electric Bass (pick)35 Fretless Bass36 Slap Bass 137 Slap Bass 238 Synth Bass 139 Synth Bass 240 Violin41 Viola42 Cello43 Contrabass44 Tremolo Strings45 Pizzicato Strings46 Orchestral Harp47 Timpani48 String Ensemble 149 String Ensemble 250 SynthStrings 151 SynthStrings 252 Choir Aahs53 Voice Oohs54 Synth Voice55 Orchestra Hit56 Trumpet57 Trombone58 Tuba59 Muted Trumpet60 French Horn61 Brass Section62 SynthBrass 163 SynthBrass 264 Soprano Sax65 Alto Sax66 Tenor Sax67 Baritone Sax68 Oboe69 English Horn70 Bassoon71 Clarinet72 Piccolo73 Flute74 Recorder75 Pan Flute76 Blown Bottle77 Shakuhachi78 Whistle79 Ocarina80 Lead 1 (square)81 Lead 2 (sawtooth)82 Lead 3 (calliope)83 Lead 4 (chiff)84 Lead 5 (charang)85 Lead 6 (voice)86 Lead 7 (fifths)87 Lead 8 (bass + lead)88 Pad 1 (new age)89 Pad 2 (warm)90 Pad 3 (polysynth)91 Pad 4 (choir)92 Pad 5 (bowed)93 Pad 6 (metallic)94 Pad 7 (halo)95 Pad 8 (sweep)96 FX 1 (rain)97 FX 2 (soundtrack)98 FX 3 (crystal)99 FX 4 (atmosphere)100 FX 5 (brightness)101 FX 6 (goblins)102 FX 7 (echoes)103 FX 8 (sci-fi)104 Sitar105 Banjo106 Shamisen107 Koto108 Kalimba109 Bag pipe110 Fiddle111 Shanai112 Tinkle Bell113 Agogo114 Steel Drums115 Woodblock116 Taiko Drum117 Melodic Tom118 Synth Drum119 Reverse Cymbal120 Guitar Fret Noise121 Breath Noise122 Seashore123 Bird Tweet124 Telephone Ring125 Helicopter126 Applause127 Gunshot

[dbawel]

This is very cool.  I don't get any errors, but on my PC running Chrome it takes most of my CPU once launched, and I had to force close my browser.  But I wonder if this is another Chrome issue running Web Audio (which I assume you're using?)  I suppose I should look at the source first, huh?  But I'm almost ready to give up on Chrome, as it's full of bugs these days - so if no one else has these issues with other browsers, please let me know - as I have too many issues in Chrome to even begin to troubleshoot or to wonder if it is indeed a Chrome problem with this app.

 

I definately look forward to the next version once it's cleaned up and optimized.  I've been working on a music room also, but recently got sidetracked on another app.  However, I hadn't introduced any sequencing, and I'm certainly impressed.

 

Also, is it running in mono for click events for everyone like it does for me?  Or is anyone else able to trigger more than one sample with a pick event (polyphonic)?

 

Looks like we have another star on the babylon horizon!

[sssurikov]

As far as the Firefox error.

 

 

I think this is Babylon.js error. See examples without Babylon.js at http://www.reddit.com/r/molgav/ There are no errors

 

Are you planning to integrate with the sound API?

 

 

Which sound API?

 

Does list of instruments look familiar?

 

 

Yes. This is MIDI instruments. SSSynthesiser.js uses some free soundbanks to play music. Ofcourse, free soundbanks has low quality 8(

 

but on my PC running Chrome it takes most of my CPU once launched

 

 

Chrome has strange issue with WebGL on some videocards. Sound engine take not so much CPU. See non-3D examples at link above

 

 

 

Also, is it running in mono

 

Yes. Mono requires two time less resources than stereo. Place feature request here if you need it. I'm trying collect all wishes.

 

 

 

 is anyone else able to trigger more than one sample with a pick event (polyphonic)

 

This is a small demo. May be i will create more complex demo later. May be you can create multitouch demo that uses SSSynthesiser.js?

_{^{Btw, don't use computer toys if you wish play piano, just use real piano.}}

[dbawel]

SSSurikov,

 

Again, excellent demo.  I would love to add polyphonic multitouch to your demo - or to integrate this into the simple music app I've been working on in spare time.  However, I haven't been able to locate where in your SSSynthesizer.js file I might allow more than a single note of a sample to finish its envelope while triggering another note or seperate sample.  It's easy enough to read through, but if you might point out where you're limiting the polphony of a specific sample, this would be very helpful.

 

I'd love to work further on this.

 

Cheers,

 

DB

[sssurikov]

 

I haven't been able to locate where in your SSSynthesizer.js file I might allow more than a single note of a sample to finish its envelope while triggering another note or seperate sample

 

 

 

 

SSSynthesizer.js doesn't consists any code for screen. This is engine for sound playing. You have to change code of the sssdemo.js

 

 

 

window.addEventListener("touchend", function (touchEvent) {stopHit(); //this don't care how many fingers pressed - change it/*var touches = touchEvent.touches;for (var i = 0; i < touches.length; i++) {var one = touches[i];pickPianoKeyAndStopSound(one.clientX, one.clientX);}*/}, false);

 

also you need listener for 'touchmove' event to watch if finger press a piano key and moved away from this key.

 

How you see this required too much coding for such simple demo. This demo about sound mixing not about touch events.

[JCPalmer]

I was thinking about the web sound API, but scratch that.  When I was just trying to get some sound effects for a Java program, I found the MIDI structure both very difficult & overkill.  Other than maybe Wingnut, it is unlikely many here with enough musical background.

 

I ended up building a simplified interface using a single instrument at a time.  There was a note object, & a verse object as I recall.  You would load verses into a "Jukebox", to be played on request.  Maybe the "Jukebox" here could be implemented as a web worker.  This would be beneficial for a gaming API, that is forced to be single threaded.  The request nature of a web worker also seems very natural anyway.

 

The key is, that since there is already a file based, looping alternative, this cannot be much more difficult, or no one will use it.  I doubt anyone is going to try play actual music inside a game with this.  I do still have what I did, if you think looking at it is worth while.

[sssurikov]

for a Java program, I found the MIDI structure both very difficult & overkill

 

 

.mid file is for store a music. You don't need it to play music through MIDI synthesiser. See my old project as example

https://code.google.com/p/easy-midi/

 

I doubt anyone is going to try play actual music inside a game with this

 

 

Why do you think about games? Market for music producing applications is big. See

https://play.google.com/store/apps/category/MUSIC_AND_AUDIO/collection/topselling_paid

for example. There are mature apps like FL Studio Mobile, toys like 'real' guitar e.t.c. Millions of users, thousands of applications.

 

Ofcourse you don't need sound engine for simple background music of game. Just use a .mp3 for it.

But you need a sound engine for interactive background music of game. For example you can change music in realtime while user in danger.

 

SSSynthesiser.js allow you create, change and play music in realtime.

I will try to publish another demo to show more features.

[JCPalmer]

Ok, I found SSSynthesiser.js through the console window while running your demo.  It is written in the Web Audio API already.    I did not notice your topic tag, or remember the name of the api, but once I saw the code knew what it was.

 

I never meant to imply that it was not useful, just that for games any sound solution cannot be too much harder than .mp3 or other format files.  BabylonJS as of version 2.0, also has Web Audio API implemented, but for pre-recorded sound files.  You can already do things in realtime with BJS.

 

While you should have a standalone version for non-games apps, there might be a way to put just the parts we do not have directly into the BJS framework.

 

In BJS there is a typescript class called Sound.  Basically it is the holder of an AudioBuffer.  It might be possible to make a class called say, SynthSound, which was a subclass of Sound.  All it would do is ensure AudioBuffer member was constructed, using as much as possible that already exists in BJS, like the AudioContext.

 

Anything a Sound class can do in BJS, and also be done by a Sound subclass with no additional work.

[dbawel]

Midi is a very simple language - and has been around since the early 1980s - when I was making records.  That dates me a bit, but I'm happy it's still the standard in recording. If you look at the stream of data, it is very simple.  It is a unidirectional (doesn't have to be) asynchronous bit stream at 31.25 Kbits/sec.  Each byte contains 10 bits - a start bit, 8 data bits, and one stop bit.) Within the 8 data bits is a note on, a note off, key pressure (polyphonic), channel pressure, pitch bend (change in pitch controller,) program change, and unique control change messages such as breath or other user defined parameters.

 

After 30 years, my studio still runs on MIDI as does every dedicated piece of hardware such as keyboards and drum machines - not os based instuments, although with a midi interface, they all follow the midi standard.  So if you're interested in creating music for your games with the lowest overhead, learn midi messages - it's way too simple to overlook.  And as sssurikov mentions, there is a HUGE demand for music editing and sequencing software.  Just take a look at the #s for Protools, Sound Forge, Cakewalk, Acid, etc.  Without midi as a standard, I would have never been able to make the records I produced.

[dbawel]

OK Jeff,

 

You hit the 666 mark in posts.  What does that mean?

 

DB

[JCPalmer]

DB,

Still batting 0 on warning posts, whatever those are, though .  This post will bump it to 667, so before posting this I celebrated by creating a new twitter account "whoopstopia".  The 2 are not really related, but that is sort of what that account is going for.

 

Jeff

[sssurikov]

I tested demo under Windows10/MS Edge

 

[Vousk-prod.]

That's pretty impressive, love that demo!

Great discovery, SSSynthesiser.js is just trully amazing!!

 

Hmm, but isn't a midi synth engine a bit too specific to be inserted by default inside a game engine?

 

JCP, I also have my own musical (home) studio and I confirm that MIDI format is THE standard format used everywhere and between every devices in musical production. It allows to record some kind of real instrument, saving the feeling you play, and map that to an other "instrument" (you can play trully beautiful pianos or drums on bad quality keyboards or electronic drums for instance). The interesting point in MIDI is that it's not bound to the final sound, it only records velocity, length, etc. MIDI synthetisers have awful sounds, but they are synthetisers, this is totaly different from the MIDI messsage itself, which can be plugged to marvelous sound banks.

[GameMonetize]

I think this is an EXCELLENT candidate for an official extension

[Vousk-prod.]

Can't agree more, that would be a perfect extension. I can already imagine the opportunity for HTML5 audio apps and all kind of VJing applications!

[pathogen]

Awesome demo!

[sssurikov]

Hi

 

No time to finish work, sorry, See https://github.com/surikov/SSSynthesiser.js for source and examples.

 

Features

- it plays music on desktop and mobile

- you can change any part of song in realtime

- you can create song from scratch or use about 100 000 songs from library

- you can use equalizer and some filters