Dynamic lightning

[NikoVonLas]

Hi. Sory for my english, i know it very bad.
How can i implement a normal map to sprites?
Im read about this on google groups(https://groups.google.com/forum/#!topic/melonjs/cccLhhyr67Y).
But i dont can implement this in melon 4.1
Anybody can help with this?

[Parasyte]

HI,

Do you have any experience with WebGL? Specifically with writing shaders and passing values as attributes and uniforms using the WebGL API?

melonJS comes with a very basic set of WebGL shaders that can draw textured quads, and the WebGL compositor is extensible, so you can add extra attributes (like normal maps and lighting). To do so, you extend me.WebGLRenderer.Compositor with your own changes. E.g. if you need to add another attribute, you can resize the WebGL memory buffer `this.sb` and the javaScript buffer `this.stream`, add a new attribute pointer, and finally overload the `addQuad` method to append the new attribute to `this.stream` and increment the stream buffer index `this.sbIndex` appropriately. The best way to understand these pieces is to go through the source code. And the following blog post describes how it works: http://blog.kodewerx.org/2015/02/melonjs-should-be-all-about-speed-part-6.html (It is in English. Maybe you can find someone to translate if you need help with that.)

I know, it's really not as easy to extend. The API could be more generic, and abstract parts of WebGL better. But if you're familiar with WebGL, then you shouldn't have any problems with it.

And one last thing, the compositor has not changed at all since that dynamic lighting code was written. So it should be a drop-in replacement. The only thing he changed was the quad fragment shader, and added a few uniforms to the compositor constructor:

this.quadShader.uniforms.Resolution = [800, 600];
this.quadShader.uniforms.AmbientColor = [0.8, 0.8, 0.8, 0.2];
this.quadShader.uniforms.LightPos = [0, 0, 0.075];
this.quadShader.uniforms.LightColor = [1.0, 1.0, 1.0, 1.0];
this.quadShader.uniforms.Falloff = [0.1, 7.0, 45.0];
this.quadShader.uniforms.LightSize = 5500;

And here's the fragment shader GLSL template (this is deconstructed from a compiled template, so it does not contain all of the original comments):

/*
 * MelonJS Game Engine
 * Copyright (C) 2011 - 2017 Olivier Biot, Jason Oster, Aaron McLeod
 * http://www.melonjs.org
 *
 * WebGL Fragment shader for quad compositing
 */

precision {{= ctx.precision }} float;

// Uniforms

/**
 * 2D texture sampler array
 * Maximum number of textures is determined at compile-time
 * @ignore
 */
uniform sampler2D uSampler[{{= ctx.maxTextures }}];

/**
 * New stuff added for lighting
 * @ignore
 */
uniform vec2 Resolution;
uniform vec4 AmbientColor;
uniform vec3 LightPos;
uniform vec4 LightColor;
uniform vec3 Falloff;
uniform float LightSize;

// Varyings

/**
 * Fragment color
 * @ignore
 */
varying vec4 vColor;

/**
 * Fragment texture unit index
 * @ignore
 */
varying float vTexture;

/**
 * Fragment texture coordinates
 * @ignore
 */
varying vec2 vRegion;

void main(void) {
    // Convert texture unit index to integer
    int texture = int(vTexture);

    // Temporary vectors for colors read from the diffuse and normal map textures
    vec4 DiffuseColor;
    vec3 NormalMap;

    /*
     * Dynamically indexing arrays in a fragment shader is not allowed:
     *
     * https://www.khronos.org/registry/webgl/specs/1.0/#4.3
     *
     * "
     *  Appendix A mandates certain forms of indexing of arrays; for example,
     *  within fragment shaders, indexing is only mandated with a
     *  constant-index-expression (see [GLES20GLSL] for the definition of this
     *  term). In the WebGL API, only the forms of indexing mandated in
     *  Appendix A are supported.
     * "
     *
     * And GLES20GLSL has this to say about constant-index-expressions:
     *
     * "
     *  constant-index-expressions are a superset of constant-expressions.
     *  Constant-index-expressions can include loop indices as defined in
     *  Appendix A section 4.
     *
     *  The following are constant-index-expressions:
     *    * Constant expressions
     *    * Loop indices as defined in section 4
     *    * Expressions composed of both of the above
     * "
     *
     * To workaround this issue, we create a long if-then-else statement using
     * a template processor; the number of branches depends only on the total
     * number of texture units supported by the WebGL implementation.
     *
     * The number of available texture units is at least 8, but can be as high
     * as 32 (as of 2016-01); source: http://webglstats.com/
     * See: MAX_TEXTURE_IMAGE_UNITS
     *
     * The idea of sampler selection originated from work by Kenneth Russell and
     * Nat Duca from the Chromium Team.
     * See: http://webglsamples.org/sprites/readme.html
     */
    if (texture == 0) {
        DiffuseColor = texture2D(uSampler[0], vRegion);
        NormalMap = texture2D(uSampler[0], vRegion.st + vec2(0.5, 0.0)).rgb;
    }

{{ for (var i = 1; i < ctx.maxTextures - 1; i++) { }}
    else if (texture == {{ i }}) {
        DiffuseColor = texture2D(uSampler[{{ i }}], vRegion);
        NormalMap = texture2D(uSampler[{{ i }}], vRegion.st + vec2(0.5, 0.0)).rgb;
    }
{{ } }}
    else {
        DiffuseColor = texture2D(uSampler[{{ ctx.maxTextures - 1 }}], vRegion);
        NormalMap = texture2D(uSampler[{{ ctx.maxTextures - 1 }}], vRegion.st + vec2(0.5, 0.0)).rgb;
    }

    /*
     * An implementation of the Phong reflection model
     */
    vec3 LightDir = vec3(LightPos.xy - (gl_FragCoord.xy / Resolution.xy), LightPos.z);
    LightDir.x /= (LightSize / Resolution.x);
    LightDir.y /= (LightSize / Resolution.y);
    float D = length(LightDir);
    vec3 N = normalize(NormalMap * 2.0 - 1.0);
    vec3 L = normalize(LightDir);
    N = mix(N, vec3(0), 0.5);
    float df = max(dot(N, L), 0.0);
    vec3 Diffuse = (LightColor.rgb * LightColor.a) * df;
    vec3 Ambient = AmbientColor.rgb * AmbientColor.a;
    float Attenuation = 1.0 / (Falloff.x + (Falloff.y * D) + (Falloff.z * D * D));
    vec3 Intensity = Ambient + Diffuse * Attenuation;
    vec3 FinalColor = DiffuseColor.rgb * Intensity;

    // Output the final color for this fragment
    gl_FragColor = vColor * vec4(FinalColor, DiffuseColor.a);
}

 

[ldd]

Hey! I'm the original poster of that forum topic!

I understand that webGL is very hard, and I just replied to your e-mail but I thought it would be nice to give a follow up here, where melonJS maintainers could help.

On the melonJS end (I might be wrong about some details):

1) melonJS, when I used it, used a default compositor found here. If you look at the history of commits on the file, there are some changes, but I haven't yet taken a look at them (here's the link for the changes).
 

1.5) I used a custom compositor that you can find here.

 

2) So you first need to be able to use a custom compositor. I would make sure that this is still possible in newer versions of melonJS (I'd have to look into it)

 

3) I really followed this tutorial and code to do my changes: https://github.com/mattdesl/kami-demos/blob/master/src/normals-pixel/readme.md

 

4) you need an image that contains, on the left, the normal pictures you need and on the right, the normal maps (you can auto generate normal maps using a tool like sprite dLight. (I don't have an example but basically http://i.imgur.com/gnztOT6.gif has 5 parts. I used part 1 on the left of an image, and part 4 on the other side. Once renderer, in game, the result looks somewhat like part 5.

 

Again, I didn't really have much time to look into it,  so if you are patient you can hopefully work your way through my explanation. 

If I were you, I would start by copying the code I mentioned in point 1.5), using it in the demo as the compositor, and see if any errors show up. Then I would add the image that has a normal map, and that's it.

 

I hope that I helped, and if I didn't, I think the guys over at melonJS are super helpful and they can help you figure out how to do it, and how to use a custom compositor. They are online on gitter

[ldd]

I hate to double post, but I got it working on 4.x melonJS

Like I said before, I just looked at the history of the file and obviously, the init method no longer takes three parameters, it takes only one.

here's the working compositor (that gives basic lighting): https://gist.github.com/ldd/d0fa1bb1fe1a3644abfdd123e0875b8b

and I just had to look at the updates in this commit: https://github.com/melonjs/melonJS/commit/68b1e3a66bafa9d52e6e291ae5f94842593efe16

 

Be warned that there is a lot of theory to understand how to use the lighting and use better defaults (I'm just providing this for reference. The tutorial that I mentioned is awesome and will tell you what you need to update.

[obiot]

awesome feedback @ldd !

[ldd]

No problem. By the way, I tested my changes using the updated platformer, so you can see the changes needed in this commit.

It will also tell you how your image will look once you've added the normals, and where to add the compositor reference

(Obviously, it now sits as a global, which you shouldn't do in your game, and the updates I made did not include the JSDoc updates, so make sure to clean your code too)