New Idea

[Mezzorio]

So I have a challenge on the current build, I'm looking at if you set the camera lock along z and give it 10% movement on y, then only can pan along the X using mouse hover to move the camera, and keep the infinite effect so your camera view is locked to essentially viewing the boxes, panning left to right using different images but so when say position 40 ends number 1 starts again, also as some additional information i'm using high details images i've made up in photoshop, as PNG and I want to move from equal distance objects to build up a scene of characters and layer up the characters across eachother, like found in this live example of the type of build i'm working on for this part.

After you click the meet the team button after video:

https://www.g-star.com/en_gb/raw

Imagine that build but changing to characters and with there stats coming up and all that kind of stuff when you click on them, but they look like a team of people kind of thing, same sort of feel i want to create in my build.

[Wingnut]

Hi Mezz.  That's a nice effect... that g-star slow-pan-across-cutouts thing.   I think that is done with an HTML operation called image maps.  Notice that a click does not go-active when your mouse is over the outer edges of a frizzy-haired head.  The mouse needs to be a little bit further towards the person's head, and THEN the click goes active.  There are invisible "click-zones" defined... like a fenced-in area surrounding the person's body shape.  As soon as you cross the fence (when you are inside-of the image-map click-corral) with the hover, that's when the click goes active.

These image-map corrals (pre-defined clickable zones) are not easy to do in webGL, if possible at all.  First, all your images need to be transparent background, I think.  In 3D land, transparency is often called alpha.  You have likely seen textures used in playgrounds... that are set .hasAlpha = true; or things like .useAlphaFromDiffuseTextureRGB = true;, etc. These are things that pertain to images/textures... with alpha, or, in other words, with transparency.  I'm no expert, so you might want to read about alpha on your own, and listen-to smarter people than I.

Forum friends @Nesh108 and @JohnK have done some work with what some of us call "cut-outs" or "silhouette mesh".  So has @ulisse, working-on MRI slices.

They are not easy to create... but Nesh has done it as well as anyone has, with BJS, so far.  (I was one of the requesters of cutouts, so Nesh is a bit of a hero to me, but so is everyone here in BJS land.)  I was/am seeking cutouts like those seen in https://www.youtube.com/watch?v=Xxl-6HIGCuA between 1:05 and 1:20.  I love those silhouette cutout things... they rock. 

But Nesh's "cookie cutter" has some issues, such as https://www.babylonjs-playground.com/#9GPMUY#16 .  At least you can see what Nesh's code TRIES to do.   Essentially,  it tries to make a flat mesh whose silhouette/contours match the non-transparent parts of an image.  Maybe Nesh and the gang will visit, and show us more demo urls.  A PG search for "keepOtherPixels" should find more Nesh demos.

This is what you need, I suppose.  All we have in BJS land... is click-the-mesh, so if you want to overlap flat-mesh atop flat-mesh, and have their click-zones NOT overlap, you'll need to shape the mesh to match the pictures.  Perhaps Blender or some other modeler... has a "Cutout mesh to match non-alpha parts of image" feature?  Not sure.  Possibly... "Extrude non-alpha parts".  BJS has an "extrudeShape" system, thanks to @jerome and his predecessors (and perhaps @adam who, like Jerome, sneaks-around quietly improving thousands of BJS things).  But the BJS extrudeShape does not automatically "derive" the shape data (the coordinates used to define the silhouette) like Nesh's cookie-cutter, and it also has some problems "capping" oddly-shaped extruded mesh.  Deriving ('gleaning'?) the shape silhouette... is sometimes called "edge detection".  If you look at Nesh's demos, he has to code-hard to accomplish edge-detection... rendering the image inside another canvas/image element, and grinding-it-out via pixel-by-pixel analysis.  (I got a little brain tumor JUST by THINKING about that task.) 

Anyway, I thought I'd start the morning right... by crushing all your hopes and dreams.  It makes me feel good.  hehe.  (j/k, of course).  We just don't have a meshOnlyPickableInNonAlphaTextureZones = true;  ...yet.  Will you be the one to invent that?  Hmm.

Perhaps others will comment.  Mezz, you're starting to wander-into brain tumor land, ya know? 

PS: I think it was JohnK who was the first to "cap the bird".  This bird shape-data was a challenge for our extrudeShape feature... to properly "cap".  JohnK or SOMEBODY... was able to use the PolygonMeshBuilder and get some nice end-capping.  De-wireframe at line 59.  PERFECT cap, and some pretty texture-tiling, too.  But it sure required a crapload of zig-zaggy indices, didn't it?

[Wingnut]

I keep editing it, eh?  You can never finish reading it, if I won't stop changing it.  hehe.

(I write tutorials in forum posts, eh?  They gotta be as accurate and fleshed-out as possible, because lots of people will read these, someday.  We're making history with every word.)

[Mezzorio]

@Wingnut Im back at last! Iv'e uploaded a video on youtube of where my projects at using sprite images of people  

 

I hope now you can really see the effect of what I'm going for! I'm nearly there now I think, Just need to fiddle with positioning and things a bit more, but its still not quite perfect! Ready for some more tips from master Wingnut!

[Mezzorio]

@Wingnut So! I have been building a new environment as I'm unhappy with mine, I have this playground https://playground.babylonjs.com/#4GCPF4#2

I want to swap the spheres over with sprites, but it keeps breaking it without an error message displaying sooo i'm struggling and need your Genius!!

[Wingnut]

haha.  YOU need to work-at-it a bit more... in that playground.  Here's a starter:  https://playground.babylonjs.com/#4GCPF4#3

You can remove the spheres, and install a line of sprites.  Sprites are not mesh, so you'll need to position them using methods for sprites, not for mesh. Position them within a FOR-loop, so I can easily make the line be 2 sprites, or 2000 sprites.

Get me a line of sprites, send me the PG URL after you're done... and then we'll do animation testings.  Sound good?  Alright. 

Keep in mind... your images MUST have transparent background, or else you won't have nice image overlapping.  And click-zones for ONLY the non-transparent parts of the image... I don't know HOW you will do that - if needed.  Maybe others will have ideas.

[Mezzorio]

My images are already with transparent backgrounds and such, I just keep breaking things and it's making me sad!  I'l keep working at it and sending you playgrounds as i progress! your like my mentor at this stage!

[Wingnut]

Oh goody, thanks for choosing me.  hehe.  Do you know how to tell made-up pirate stories?  I like pirate stories... so... ahem. 

Remember WAY back, when I steered you AWAY FROM using sprites?  And now you want me to help you with a sprite playground?  hehe.

How are you at making pancakes or french toast?  I need to get some free labor out of you, somehow.  heh

I guess we HAVE TO ASK... if you will want "click on non-transparent parts of image".  If so... hmm.  Sprite images might not work for that.  You may need a button atop/near each sprite... to use as a clickable thing.

[Mezzorio]

8 minutes ago, Wingnut said:

Oh goody, thanks for choosing me.  hehe.  Do you know how to tell made-up pirate stories?  I like pirate stories... so... ahem. 

Remember WAY back, when I steered you AWAY FROM using sprites?  And now you want me to help you with a sprite playground?  hehe.

How are you at making pancakes or french toast?  I need to get some free labor out of you, somehow.  heh

I have found a way to make shadows with sprites! Bet you didn't expect that! For the depth effect i'm after, Sprites are proving more effective with the way i'm doing it atm! I did used to be a baker   However, as for pirate stories i'm lacking!

I have a rough plan for the overlays and clickable side of things I just cant get the environment in a way that I'm happy

[Wingnut]

  Yeah, I'd love to see sprite shadows in your PG, too.  YUM!  Nice job.  I didn't know that was possible.

Want to have some CORS-clear images to use on your sprites... in the playground?

https://tinyurl.com/yco9srpz

That is a link to a google IMAGE search for "person" and "transparent" with re-usage rights == full, and with a few annoying image sites... filtered-out.

Essentially, this is a way to get a HUGE pile of CORS-cleared images... for use in a playground.  Change the word "person" to anything...  like "face" or "bust" or similar.  All/Most of the images returned in the search... will work for playground images, and likely have transparent backgrounds.  (for prettier playground testing fun).

*shrug*.  Might be useful for ya.  Click on any "thumb" image returned in that search, then choose 'view image'... and then copy the URL at the top of the 'view image' page.  That URL should work for playground use.

[JohnK]

This appears to be a solution for meshes and it might work with sprites. On phone not PC until tomorrow or later so cannot give it a go (also I cannot say I understand it fully)

[Mezzorio]

@Wingnut Been making some good progress today, https://www.babylonjs-playground.com/#VEGKI1 is now ready for you to play with! any questions ask awayyy! Ill update that playground and the link as I make further progress! 

[Wingnut]

  https://www.babylonjs-playground.com/#VEGKI1#2

Let's get some faces out there. 

So, Mezz... do you know why SOME images are in the back (higher z-depth) and SOME are in the front (lower z-depth)?

Let's use only 2 layers, right now.  Let's label them depth1 and depth2.

I think the "effect" you seek... can be attained by having depth1 images... scroll past the camera at a slightly faster speed... than depth2 images.

But first, we must learn WHY some of these images are at different depths than others.  I actually don't know why.  Do you think it is related-to WHICH image finishes loading first/second/third, etc?

This is your next challenge.  Learn WHY some sprite images appear to be low-depth, and WHY some are higher-depth.  Then see if you can control those depths.  Fun challenge, yes?  You bet.  Perhaps other "in-the-know" forum helpers will show us some hints and secrets.

[Mezzorio]

Although i cant get it working on the playground this is where im at currently depth effect working as im positioning all my sprites nicely to form the https://www.babylonjs-playground.com/#VEGKI1#5

 

Edit: Playground works with working sprite of myself lol

 

 

[Wingnut]

Hey, that looks pretty good.  The sprites along the right side... are already giving you that multi-layer effect (low-depths scroll slightly faster).  You're doing great!

You have surpassed MY knowledge about sprites, that's for sure.  Excellent.  Keep on truckin', my friend.  I don't know what advice I can give... that would help.  (I'm not very good at sprites, or much of anything else). 

Other forum helpers may have more tips and advice.  But... you're doing just fine... very good.

[Mezzorio]

1 minute ago, Wingnut said:

  I don't know what advice I can give... that would help.

Other forum helpers may have more tips and advice.  But... you're doing just fine... very good.

Well, my plan after i position everyone is to have them clickable which makes the others disappear brings them forward and shows some text/information alongside them when selected

[Wingnut]

Do you think that the transparent parts of the sprite image... will be clickable?  That could cause problems.

Example... you click on the left-side of an image... (your right arm, for example).  If the image to the left... has a transparent area that is overlapping your arm of the right image... then the click will happen on the left image, even though you clicked on the left arm of the right image.

The transparent part of one image... blocks/intercepts a click... on a non-transparent portion of another image.

This is why... in previous posts... I mentioned that you MIGHT need a "more info" button/label, or possibly a name label... at the bottom of each sprite.  Then users click on THAT and not on the image itself.  *shrug* 

You'll discover if you have overlapping click-zone problems... soon, I suspect.  Fun challenge... lots of learning happening here... especially for me. 

[Mezzorio]

51 minutes ago, Wingnut said:

This is why... in previous posts... I mentioned that you MIGHT need a "more info" button/label, or possibly a name label... at the bottom of each sprite.  Then users click on THAT and not on the image itself.  *shrug* 

You'll discover if you have overlapping click-zone problems... soon, I suspect.  Fun challenge... lots of learning happening here... especially for me. 

I'm wandering the same, and I'm hoping it wont be clickable, but I am also thinking of using some html overlay stuff to make it all work, I'm not too sure yet how to approach it, taking it step by step at the minute!

[Wingnut]

Hey M... did you take notice of the link that JohnK sent in his last post?  You seem to be starting new threads about subjects... that have already been answered here.

https://www.babylonjs-playground.com/#1UCP5L#5

There's the playground that does what you asked.  Two images, both with transparency, overlapping.

 - Click-on transparent area of left image, background green, no pick on left image.

 - Click-on transparent area of right image, background green, no pick on right image.

 - Click-on painted area of left image, background red, good pick on left image.

 - Click-on painted area of right image, background blue, good pick on right image.

I think @ZackMFleischman had a good idea... when he suggested texture.getTextureColorAtUV(u, v) in THIS POST.

Possibly, a future feature on a pickingInfo class object... getPickedPointColor()... does all the messy stuff FOR us. 

That might be "contaminating" a pickingInfo object, though (using it for something that is too far outside-of its primary purposes).  hmm.  Thinkin'. 

[Wingnut]

(Wingy scratches the microphone spit-screen... to make sure it is ON) 

U/V are the same as X and Y... except they are used for textures, not mesh/space.  A texture has its UV of 0,0... at its lower left corner (as opposed to html elements and many other things, which have 0,0 at upper left).

So, once you have a U and V value, you have a "offset-into" a texture/image.  The U value is measurement of how far rightward from 0,0, and the V value is how far upward from 0,0.

This allows you to put the image into a non-displayed 2d canvas and image buffer, then grab the color4 value of the pixel that is U-distance rightward, and V-distance upward... from the lower left corner of an image.

In the other thread, Deltakosh showed you how to get a U and V value from a sprite's picked point.  Now look at https://www.babylonjs-playground.com/#1UCP5L#5

It is advanced programming, to a degree, but there are cool things in there.    Take note of lines 83-90:

	if (pickInfo.hit) {
		let uv = pickInfo.getTextureCoordinates();
		let pixel = getImageDataPixelAtUV(impactImageData, uv.x, uv.y); // chg uv.x to u, and uv.y to v
		if (pixel.a === 0) {
			// We clicked a transparent part, so "unpickable" this one and pick again.
			pickInfo.pickedMesh.isPickable = false;
			pickInfo = pickNonTransparentPartOfImage();
		}
         }

getImageDataPixelAtUV()... wow, what an inviting function name for a swell guy like Mezzorio, eh?   If (pixel.a ===0)... or in other words... if this pixel has ZERO alpha, then... it is a transparent pixel.  Easy, eh?  The 4th item in a color4 is the alpha transparency...  color4(r, g, b, a).  With me?  Alright!

In your other thread... deltakosh was telling you how to establish a "Where on the sprite texture... did the pick happen?".  Now it is your job to use various tools, and those UV values (UV = X and Y offsets into an image/texture... measured from lower left corner)... to test the color/alpha at a certain point on the sprite image.

UVs are sometimes in values between 0 and 1.  So, a U of 0.5 means 50% X-distance... or halfway across the image X axis.  So, the center of an image is often at UV of (0.5, 0.5).   BUT, I think Deltakosh's U and V values (from your other thread)... are NOT between 0.0 and 1.0.  I believe they are in pixels.

Okay, I just wanted to connect point A and point B for you, yet not interrupt your bothering of the boss-man in that other thread.    (j/k)

PS: Did you see where Deltakosh said that sprites suck for this, and you should be using planes for this, instead?  Ok, I didn't see that either.  heh.  But... look how hard he had to think... about how to get a U and V from a sprite pickedPoint.  He almost sprained a neuron!  Had they been planes, they would have given you the U and V values on a silver platter, delivered by limo, accompanied by dancing girls, a live band, and an open bar.    (line 84 - pickInfo.getTextureCoordinates() - so easy.)  Later, if you put all your sprite images into a single "sprite-atlas" image, then you're headed-for a brand new kind of hell.  (Finding the lower left corner of whatever image "portion" is being used on the picked sprite... is more of a challenge, but not too bad.)

[Mezzorio]

On 08/09/2017 at 3:28 PM, Wingnut said:

 

Have a look at this https://www.babylonjs-playground.com/#VEGKI1#8 = My current progress with the build! Managed to get the transparent click working! I am now looking how to position the text display on the screen as well as hiding the others from view when any one person is clicked and all return when they are clicked again

[Wingnut]

Wow... what a playground!

I'm getting some errors when clicking... not sure what's shakin' with that.  sprite._manager.pickSpritePixel is not a function

[Mezzorio]

3 minutes ago, Wingnut said:

Wow... what a playground!

I'm getting some errors when clicking... not sure what's shakin' with that.  sprite._manager.pickSpritePixel is not a function

I think its really buggy on the playground is the issue, im not working from playground im working locally and all works!!