Get the alpha of a pixel in a sprite

[Yeferson]

Hi everyone,

This is a bit tricky and I don't know if it is possible.

I want to get the alpha of a pixel (Which is in an x and y coordinate) of a sprite.

What I want to do is the following:
I have several textures (PNG) one on top of another as layers, what I want to do is know which layer my mouse is selecting, but since they are all one on top of the other then it cannot be done by AABB collisions. What I came up with is that I could get the mouse coordinate and go through each of the layers to find out if the alpha of that pixel is greater than 0.

Thank you.

Edited August 3, 2021 by Yeferson

[ivan.popelyshev]

I saw this topic a week ago but i dont remember whether its on this forum or in pixijs issues/discussions on github

Here are the demos: https://github.com/pixijs/pixijs/wiki/v5-Hacks#pixel-perfect-interaction

however , either someone modified them, either they just dont work in latest v6. You have to understand the code and find out whats wrong.

[Yeferson]

Hello,
The first example already works:: https://www.pixiplayground.com/#/edit/VwRWHqOC_PNQPFR7tjHyl

I was doing something very similar but with the previously processed texture:
  - I converted my texture to alpha texture (Black and White) but this was not actually necessary.
  - Then I reduce the size of my texture (So that it does not weigh too much when loaded)
  - I save all pixels as bits
  - I save that buffer in a file
  - I load the file in pixi and I already have the hitmap.

I think I'm going to use something similar because I see in the example it goes through all the pixels of the texture (w * h) and it also uses canvas (drawImage and getImageData), and if this is run on mobile with many textures it can explode haha.

[ivan.popelyshev]

>  if this is run on mobile with many textures it can explode haha.

this is why we dont add any methods like this in pixi. Its an open problem and so far no one made good general-purpose plugin for that.

[Yeferson]

I tried to optimize the time and resources that are used to create the hitmap, and I think I achieved something a little better.

Since I don't need the hitmap to be perfect (And in most cases it's not necessary either) I just reduced the hitmap image to 10% of its total size. This improved his creation speed and reduced the number of bytes that are saved.

I did the comparison with an image of resolution 1473 x 1854:

Before upgrade: Pixi Playground Before

After the upgrade: Pixi Playground After

The cons of this is that the edges are not perfect, but for me this works.

[b10b]

Nice - yes reducing the hitmap size is sensible, 8x is usually ok for "fast moving mouse".

A very different approach to a boolean hitmap is to render the entire scene twice per update.  The first render is what you currently want to see.  The second render as a "Color Map" to an offscreen renderTexture using substituted textures (or a Shader) assigning a unique RGB value to each Texture which can be "looked up".  A ray scan of sorts.  Advantages of this approach are lower memory footprint and reduced pre-processing (no hitmaps needed), minimal overhead to do multiple hit tests, pixel perfect (if subpixel rendering is disabled) and will automatically handle all DisplayObject transforms (rotation, scaling, etc).  Disadvantages are that we need to render twice!  Some of that disadvantage can be reduced by removing non-essentials, reducing the render frequency on the Color Map, etc.

[Yeferson]

9 hours ago, b10b said:

Nice - yes reducing the hitmap size is sensible, 8x is usually ok for "fast moving mouse".

The 8ms is the duration of creation of the hitmap, to verify if a pixel is active is fast.
The disadvantage with hitmap is at the beginning, since it must process each pixel of the image to create the hitmap, but to verify if the pixel in hitmap is active it is very fast:

    console.time("Checking the hitmap")

    //if not continues from here
    
    // bitmap check
    const tex = this.texture;
    const baseTex = this.texture.baseTexture;
    const res = baseTex.resolution;

    const hitmap = baseTex.hitmap;
	// hitmap width
    const hw = baseTex.hpwidth;
	// hitmap height
    const hh = baseTex.hpheight;

    //check mouse position if its over the sprite and visible
    let dx = Math.round((tempPoint.x - x1 + tex.frame.x) * res);
    let dy = Math.round((tempPoint.y - y1 + tex.frame.y) * res);
    dx = parseInt(dx / baseTex.realWidth * hw);
    dy = parseInt(dy / baseTex.realHeight * hh);

    let ind = (dx + dy * hw);
    let ind1 = ind % 32;
    let ind2 = ind / 32 | 0;

    const result = (hitmap[ind2] & (1 << ind1)) !== 0;
    console.timeEnd("Checking the hitmap")
    return result;

 

9 hours ago, b10b said:

A very different approach to a boolean hitmap is to render the entire scene twice per update.  The first render is what you currently want to see.  The second render as a "Color Map" to an offscreen renderTexture using substituted textures (or a Shader) assigning a unique RGB value to each Texture which can be "looked up".  A ray scan of sorts.  Advantages of this approach are lower memory footprint and reduced pre-processing (no hitmaps needed), minimal overhead to do multiple hit tests, pixel perfect (if subpixel rendering is disabled) and will automatically handle all DisplayObject transforms (rotation, scaling, etc).  Disadvantages are that we need to render twice!  Some of that disadvantage can be reduced by removing non-essentials, reducing the render frequency on the Color Map, etc.

This is very interesting, I think I will develop it to study it.

Edited August 5, 2021 by Yeferson

[b10b]

On 8/4/2021 at 1:05 PM, Yeferson said:

This is very interesting, I think I will develop it to study it.

You got me thinking about this Color Map some more.  I think it'd be viable to include an index directly in the original texture and avoid the need to render twice to create a Color Map.  Let's say any given color value is RGB 255,255,255.  Let's sacrifice the last 3 bits of each channel.  Max is now RGB 248,248,248.  That gives us 111111111 as combined "spare" bits = 512 indices.  Add the index back to the RGB channels.  The eye will unlikely notice the loss of data.  Now any pixel value can reveal an index between 0...512 to identify its respective Texture.  This assumes no mipmapping, no inbetween alphas, no blendmodes, no sub pixel operations etc.  Maybe?

[Yeferson]

Hey,

I've been trying to implement the idea but I don't think I understood it the way I thought it was.
The first idea you were trying to say is to render the scene twice and the second render will save the pixels inside a renderTexure? But how would you access that information?

On 8/5/2021 at 7:53 PM, b10b said:

This assumes no mipmapping, no inbetween alphas, no blendmodes, no sub pixel operations etc.  Maybe?

Doesn't work with alpha or blendmodes?

[ivan.popelyshev]

I have added a uniform "scanPixel, scanColor" in pixi globally and in preprocessor shader, that allows me to draw only one time. Its just, all shaders render number of element instead of real color if its a mouse pixel  Very tricky, not easy to do, cant even share it without spending like 4 hours on docs so people understand how that works

[Yeferson]

That you tell me that it is complicated makes my curiosity increase haha. I love challenges, where can I get that code?

[ivan.popelyshev]

I can tell you where its used, but there is like 650k lines of client js code  https://www.facebook.com/thehousehold If you move mouse around very fast, you can see the scanning pixel on screen. You can also see it in any shader if you capture the frame with SpectorJS. I'm waiting for GoodBoyDigital to make good pre-processor for shaders for pixi, so i can put the hack there

btw, that game is biggest html5 2d game. It has custom pixi fork that emulates flash graphics.

Edited August 8, 2021 by ivan.popelyshev

[Yeferson]

I was looking for this game for two days: https://www.facebook.com/thehousehold

and i think you meant lol: https://www.facebook.com/playthehousehold/

[ivan.popelyshev]

im sorry, it happens  i didnt look where do i link stuff.

I recommend you to go through tutorial and then look in TOP players , that way you'll see why I implemented that interaction there