Pixel perfect click

[timetocode]

Anyone have recommendations for getting a pixel perfect hit (click) detection for some oddly shaped jigsaw puzzle pieces? I've already got non-pixi collision detection via SAT.js rectangles, but as jigsaw pieces are pretty irregular I'd like to use the SAT Box as a broadphase and then do a pixel perfect check.

I have a few ideas but I'm not sure if they are overkill.

One idea, since the jigsaw pieces are cut with bezier curves, is to export the bezier data along with the jigsaw puzzle spritesheets and then recreate the paths in a regular html canvas. Canvas has a isPointInPath function that sounds like it can tell if the mouse is in the bounds of the puzzle piece's bezier path... tho I've never used this on a complex multi-curve object so I'm not certain. I also have quite a bit of pixi-level scaling and zooming going on, so there would be much math. There's no rotation yet (and there might not ever be), but I guess these would hypothetically be reasonable to rotate. It also sounds like the path would need remade frequently, because in this game it is possible to pan/zoom and while the pixi sprites will be fine, these canvas-bound paths won't be coming along for the ride... they'll pretty much need to be re-pathed near where the player clicks if the view has changed at all.

A more pixi-centric idea is to use the SAT Box as a broadphase, and then convert the relative position of the mouse over the jigsaw piece to the coordinate of an exact pixel within the pixi sprite. So for example if the mouse was at 50,50, and the sprite was at 45,45 and was 20x20 pixels big, then some amount of math can deal with the offsets and convert the 2d coordinate to a 1d index and tell which pixel of the sprite is under the mouse.... and then check the alpha...? I'm guessing I should cache the pixel data per sprite. How do I do this in a way that works both with the webgl and canvas renderers?

Are there other good ways?

Just having a non-canvas bezier isPointInPath would fix everything too, if anyone happens to know a lib that does that.

Thanks

[jonforum]

a idea like this. 

i was vey love the system by this guy .
 it was a pixel detection.
if i remember, you create a (duplicate rgb) to detect collision.
look ont work of this guy https://quxios.github.io/plugins

 

i don't know if this can help you, but this great idea are creat a clown with rgb.
example here red are pixel collision.

in this demo , i just photoshop with red hexa #FF0000

and use pixel detection overlap with hex.

It a cool way to do it.

[timetocode]

I tried both ways.

I couldn't get the bezier curves to work fully... it was easy enough to draw them around the pieces by translating the context, but then that requires translating the context again for the collision checks which became a bit of a mess. Also Path2Ds aren't real javascript objects with clear cut properties.. they seem to have automagic properties that are part of the dom or the context or something else that is not conventionally accessible.

Here's the pixel approach via pixi extract (v4).

let pixels = renderer.plugins.extract.pixels(someSprite)

That got the pixels from a sprite, which I cached on every sprite. Then here is the pixel perfect collision check including a broadphase:

// x, y are the mouse coordinates coordinated to world space
pieces.forEach(piece => {
    // broad phase check, where the collider is an SAT Box that fully contains the sprite
    if (SAT.pointInPolygon(new SAT.Vector(x, y), piece.collider)) {
        // how far from the top left corner of the sprite we clicked
        let px = Math.floor(x - piece.x)
        let py = Math.floor(y - piece.y)

        // use the sprite.width here
        let width = piece.width
        // convert 2D x,y to 1D index
        let index = px + py * width

        // get the alpha channel of the pixel (*4 because RGBA)
        let alpha = piece.pixels[index * 4]

        // if not fully transparent, then this object is under the mouse
        if (alpha !== 0) {
            piecesAtPosition.push(piece)
        }                
    }
})
// skipped: logic to choose which of the pieces is closest to the mouse if 
// multiple are under the pointer

Haven't tried rotation yet, as its not part of the launch featureset. Presumably rotating the sprite, creating a render texture, and then overwriting the pixels array with the pixels from that render texture would enable pixel perfect collisions again.

I'd like to note that while I'm using SAT for collisions (this is a multiplayer game and I have other collisions that are server-only) most people would do fine with just pixi collisions for the broad phase.

It would be somewhat easy to add this to all of pixi in general.. though its really just for pixel-perfect picking, not for pixel-perfect collisions. Not sure if people would be interested or not. Rotation would be the hard part.

Zooming/panning turned out to be fine with this method so long as the mouse coordinates are converted to world coordinates.

For me this turned out to be a lot of math b/c the zoom is implemented by scaling the stage, pan by moving the stage, and then there is a "low graphics" mode that locks the canvas to a low resolution and then fills the screen via css. All this stuff needed applied to the regular mouse x,y to convert it to a position in the game world.

Attached is a gif show clicking through the gaps in one jigsaw piece and grabbing the piece behind it, also clicking in the hole of one of the pieces and missing it.

 

 

 

[jonforum]

so cool hop you you will find you solution.

But I'm take a try
hum if example your create a kind of mask rendering, and your use it [alpha] for detect rendering hex pixel ? it only a suggest , do you think it can work ? 
your context is rather specialized, I'm not sure that I can be of a good help.

 

[timetocode]

I think I'm going to stick with the extract.pixels technique, it is working nicely. With the broadphase and the caching of pixel data the performance is extremely fast.