2D Sprite Animation on a Box's face

[royibernthal]

I'm trying to have 1 face of a box display a 2D animation.

The idea would be the same as having it display a texture like in the link below, only an animation instead.

https://doc.babylonjs.com/tutorials/CreateBox_Per_Face_Textures_And_Colors

How should I go about achieving it?

[Wingnut]

Hi R.  Such things are not easy to do.  WebGL does not support animated gifs/pngs, yet... or maybe ever.  Sprites are drawn in such a way that they cannot be parented to boxes, and they are always billboarded (aimed) toward the camera.  Perhaps pros at matrix transforms... can transform the sprite positions each frame of the render loop... to match a face of a box.

I use the "place plane on side of box" -method (with parenting)... to put animation on one box face, because that's an easy way to do that activity. 

Check out this forum thread... talking about exactly the same topic.

There's another way... maybe 2.  You can take an "atlas" of sprites, such as http://playground.babylonjs.com/textures/player.png and "shift it around" on a plane.  Using either texture uOffsets and vOffsets, or mesh UV value adjusting... in the render loop... you can "simulate" a sprite animation on a plane.  I'll try the uOffset and vOffset method.

http://playground.babylonjs.com/#2GYRPB#1

I am changing the uOffset and vOffset every 10th render loop, and it uses micro values to do it.  Generally speaking, it sucks.  It has positional drift.

But, web searches about this issue still recommend that all the images get loaded, and then switch textures, maybe like I do... every 10th render loop.  You don't need to use an atlas like me.  You can just load a pile of separate images, and iterate through them, placing them one after another... on the plane material diffuseTexture or some other material channel (like emissiveTexture).

I wish it were easier.  But, it is still one of the larger stumbling blocks for webGL, and none of the webGL frameworks has solved the issue.  Mostly, they make texture-flipper helpers, just like I described.  If you keep changing the textures on the plane that is parented to the box... every 10 renders, that should work.    (and much better than my uOffset and vOffset kludge/mess).

The same could be done with Canvas2D sprite2D primitives within a worldSpaceCanvas2D whose node is parented/positioned at one side of the box.  You would do the same thing, swapping-out it's image with the renderloop, periodically. 

Sorry that the sprites in my demo are a little blurry.  Each image needs to be a bit higher resolution, perhaps.  And it would be best to use the assetsLoader to pre-fetch all the sprite cels (the images) into the BJS cache system.      Hope this helps.  Others may have better ideas.

[Wingnut]

Ya know, just now, I was needing a renderTargetTexture... and I thought about THIS topic.

Let's say... you positioned a camera way out at 1000, 1000, 1000... and in front of that camera, you did a simple little 3-mesh animation... maybe some colorful boxes orbiting around each other.  Using renderTargetTextures, you could "play" that live camera view... on a plane/texture.  Take a look at this beauty, with 4 renderTargetTextures "animating" on planes below.  Wow, huh?

So, if you needed "animations" on the side of your box... that could be produced "live" out at 1000 1000 1000 television studio... THAT would really be something, huh?  If you need certain specific animations, sure, this won't work.  But if you need ANY animation... just to make your project look cool... consider using some live camera feeds from RenderTargetTexture Studios,  1000,1000,1000 Out-Of-The Way, Burbank, CA, USA.

Aside:  http://www.babylonjs-playground.com/#1WROZH#5

Before Deltakosh helped me fix it, the camera that is located directly behind the main scene camera (leftmost small plane)... was showing the 4 planes, but not the view from that camera.  It was black... in that view.  I almost got a tumor thinking about it.  Thank goodness Deltakosh fixed it quick and I was able to avoid thinking about it ever again, until now.    Anyway, I think renderTargetTextures are one of the coolest features EVER... I do.  Security cams for your dungeon/castle/world.  heh

[royibernthal]

Unfortunately I do need certain specific animations, so this won't do in this case. Very cool feature though  I'll remember it.

The ideas in your first reply sound like good directions for me to explore. Thank you

[Wingnut]

A little update.  I was touring PG's today, and came across this thing.

http://www.babylonjs-playground.com/#FT2RY#49

Likely, this is by @NasimiAsl.  He's certainly got an animation playing on the side of a box, and that's worth mentioning here.    kbye agn.

[Pryme8]

Um, I can maybe make something for you... It will be a custom shader.

 

Is there a need for transparency?

I am almost there on a animated sprite shader, its just been really hard to trouble shoot it with no sprite sheets loaded in the CYOS

[royibernthal]

@Wingnut Thanks for the update, that looks like a far more simple implementation of animation, which also seems likely to have better performance (correct me if I'm wrong).

1) How would I make different animations instances using the same sprite be in different frames at the same time?

2) babylonx shaderbuilder typescript

a. It seems that the typescript file does not represent the Map function of the javascript file correctly - for instance the parameter passed in the PG is marked as an error.

b. It'd be nice to have a declaration typescript file (.d.ts) to avoid having to compile it every time.

Who should I address these issues to?

 

@Pryme8

On 8/12/2016 at 1:10 AM, Pryme8 said:

Um, I can maybe make something for you... It will be a custom shader.

Is there a need for transparency?

Transparency would be nice. How would the shader you intend to write be different from the PG @Wingnut linked to?

[Pryme8]

Its not I just noticed it, you should go with that solution, I have not had time to look at a sprite shader yet.

[royibernthal]

I see. Any idea about my follow-up questions?

[Wingnut]

Hi Roy/P8.

1 & 2.  Dunno.  I'm not a shaderbuilder pilot.  But I have another idea. More about that, soon.

@NasimiAsl is the author (and darned good driver) of shaderbuilder.  I have now pinged him, so he might answer your typescript->shaderbuilder questions, maybe.    I'm also not a typescript user. 

Ok, thinking about this playground again... we all know that I am sliding the spritemap sideways, each frame, a certain amount... to cause the animation.

But there's another way.  On each of the 4 corners of that box face... there are (or could be)... UVs.  They are vector2's, and they are likely, currently, set like this...

lower left - 0,0
lower right - 1,0
upper left - 0,1
upper right - 1,1

You can think of these as percentages... 0 means 0%, 1 mean 100%.  These values are used to map textures to that face.  Those settings that I show... indicate the texture will be mapped 100% tall and 100% wide... across the 4 face UV's.

So let's pretend you had a real wide sprite map... like a film strip... 20 frames of sprites... side by side.  The texture is one sprite tall, 20 sprites wide.

Now, what if... you replaced those 4 UV's on the box/button face... and made them be...

lower left - 0,0
lower right - .05,0
upper left - 0,1
upper right - .1,1

That would 'map' the first frame (of our 20-wide sprite film-strip)... across the entire face.  Can you see why?  Let's move to the next frame...

lower left - .05,0
lower right - .1,0
upper left - .05,1
upper right - .15,1

Now we've put the second frame... across the face.

See the pattern?  The x values of our UVs... only increment a tiny bit per frame.  0%, then 5%, then 10%, then 15%.

Instead of moving the sprite map sideways with xOffset (like I did in that pg), do -> replace 4 UV's, replace 4 UV's, replace 4 UV's, replace 4 UV's, etc etc.

In a way, you are turning the teeth of the gears that move the film strip ahead, one frame at a time.  Click, click, etc. 

I think I saw Raanan... divide-up a single videoTexture and place portions of it...  on many tiles of planes... using this "tiny UV increments" thing.  Worked great. 

I'm no pro, not even close, but UV-wrangling seems to be a Good Thingtm to know about, and to be an expert-of.    I wish I was. 

I think this is yet another good way of animating on a face.  It is similar to my xOffset method, but with zero positional drift.  In other words... if you make your sprite map accurately, the sprite frames will get tightly-locked to those UV's and should look great.

IF you master this... then yes!    You can have many materials on different panels and boxes, all using the same spriteMap file(s) that are nicely stowed in the BJS cache. (yum)  YOU control the UV's on each box/panel.  Start one panel animating 35% into the sprite-strip, start another at 65%, they'll both run fine and not get in the way of each other.  20 boxes, 200 frames of sprites, all playing different parts of the same strip, all playing different speeds, even!  NO PROBLEM!  You are a pro UV wrangler!  You might even build some UV wrangler helper funcs or your own UV animation manager.  The sky is the limit, and I'll take a copy of that manager when you have it completed, thanks.    

Turnin' the film projector sprockets on the fronts of boxes. ( micro-incrementing UV's )    Sounds like fun, right?!  'Bout 24 UV's on a box, I believe?  Can you find the correct 4?

But still, the shader-based systems might still be wiser.  *shrug*

[NasimiAsl]

hi all  - (thanks   @Wingnut for mentioned )

in Shader Builder you can play  part of frames so i explain source and make a sample for that

1. you need make texture Divided by the square of the same number of rows and columns like this (2x2 , 3x3 or ...)

2. add ther babylonx.shaderBuilder.js to header

3.initialize shader builder when you init your scene with this code  

BABYLON.ShaderBuilder.InitializeEngine();

4. Make your material

 material =  new BABYLON.ShaderBuilder()
 .Map({path:"http://i.imgur.com/BW0tmsz.png" 
, indexCount : 4  // number of columns or rows count
, animation:true  // 
, animationSpeed: 20 // your animation speed
, animationFrameStart :5 // start frame
, animationFrameEnd :10  // end frame
}) .BuildMaterial(scene); 

http://www.babylonjs-playground.com/#2BJAXD#0

you can play back too when you set this : animationSpeed : -20

 

Step 2 Sample of control Box face with diffrent Animation

1. we make a box and detect face with normal

for flat box this condition is fast way for choose face 

normal.x > 0   =>  face number 1

normal.x < 0   =>  face number 2

normal.z > 0   =>  face number 3

normal.z < 0   =>  face number 4

normal.y > 0   =>  face number 5

normal.y < 0   =>  face number 6

http://www.babylonjs-playground.com/#2BJAXD#1   1 face

http://www.babylonjs-playground.com/#2BJAXD#2  6 face

2. make your wanted material for each face

http://www.babylonjs-playground.com/#2BJAXD#3

http://www.babylonjs-playground.com/#2BJAXD#4

 

 

 

[Nabroski]

@NasimiAsl
Great man! 
This belongs under Shader tutorial.

[royibernthal]

@Wingnut Hey, it's a nice idea, I'd like to give it a try.

faceUV is an array of Vector4, I'm trying to modify only the x and y parameters passed to it in order to display different sprites from the spritesheet but I can't seem to figure out how it's calculated. Apparently it's not as you said it's calculated, or I possibly didn't understand you correctly.

Do you think you can show me how different values you pass display different sprites from the spritesheet correctly?

http://www.babylonjs-playground.com/#1V3CAT#170

 

@NasimiAsl Thanks for the information.

Let's say we have an animation of 20 frames. We create 2 instances of it - animationA and animationB.

1) We start playing animationA from its first frame.

2) After 5 frames, we start playing animationB from it's first frame, animationA should be be at frame 6, and animationB at frame 1.

3) 5 frames later, animationA should be at frame 11, and animationB at frame 6.

So in other words each animation needs to track its current frame differently, but they don't start at different frame of the animation, they both start from the first frame of the animation but at different times.

How do I accomplish that using ShaderBuilder?

Also, what do you say about the typescript issues?

On 8/16/2016 at 3:37 PM, royibernthal said:

babylonx shaderbuilder typescript

a. It seems that the typescript file does not represent the Map function of the javascript file correctly - for instance the parameter passed in the PG is marked as an error.

b. It'd be nice to have a declaration typescript file (.d.ts) to avoid having to compile it every time.

[Wingnut]

Hi guys.

I forgot that @jerome created that FaceUV thing, but yep... he built a way to do texture-per-face.  Pretty cool system. Nice find, R.

http://playground.babylonjs.com/#VNFRE

Ok, that's really bad.  I needed to use manual-updating of the faceUV... done by disposing old box and re-creating it (every 1/10 sec right now, via setInterval).  I don't know if there is a updateFaceUV() anywhere, so I needed to dispose/recreate the box... after each change to faceUV values.  And as you can see, I did not do well in choosing my values... but that's for later. 

Jerome took a little shortcut, here.  Those vector4's are actually TWO vector2's side by side.  Even though their values are labeled x, y, z, w, they are REALLY x, y, x, y, as best I can tell.

I'm certainly no expert on ANY of this, and maybe Jerome will visit and help us fix my crappy playground.  He might even be able to help us write an updateFaceUV() function so I don't need to dispose and recreate.  In fact, I know he can.  He's a coding God and geometry wizard, and the Per-Face docs was written by him. (He's a serious hero of mine.) 

There is also a method on Abstract mesh... called markAsDirty(some_property_on_this_mesh)... and MAYBE... the property could be vertexData.UVs.  I haven't tried that... but I will in a while.  MarkAsDirty CAN, under some circumstances, inform the BJS engine that an update is needed.

What I was describing to you in an earlier post... is a more "primitive" way of setting the box UV's for a single side.  Let's look at some methods on the Mesh class... starting here.  Scroll on down... notice all the get/set vertex data helpers... there are many.  Even though most of the data that shapes a mesh (its geometry)... resides on the mesh's vertexData object, the mesh itself has lots of getter and setter methods... to modify the data on its vertexData object.  One of the arrays on a vertexData object... is an array of UV's.  Let's see if we can retrieve it, modify it, and put it back.  I will turn-off the setInterval method of animation, for now.

http://playground.babylonjs.com/#VNFRE#3

It would take some experimenting and thought... to figure out how to step-along nicely through the atlas of sprites.  I'll let you (or others) figure that out.  The main thing is that you can see some animation working... using this method. 

There are 48 UV's on the box, not 24.  Sorry for the wrong info, earlier.  You need to manipulate at least 4 of them, but I set it up to do 8.  There's LOTS of comments in this PG.  Look at it for awhile, sniff the console, notice the repeat UV values (possibly for doublesided-mode). 

SO, if you never create the box double-sided, and/or have no reason to view the animation from inside the box... perhaps you only need to animate 4 UVs (8 total values  - you need an X and Y for each UV).  Still, this PG is prepared for 8 UVs.  The Y values are not activated in the render loop. 

Watch your console.  Lots of stuff sent there.

Holler if you have more questions.  Jerome might show up and help us, too... and we're certainly glad to have assistance from ANYONE.  If some genius (not me) built a UV Animation Manager... a good one... we could do this animation MUCH easier.  The manager could be told the spacings and dimensions of the atlas, which mesh to use it upon, would allow different frame# starting points for each managed mesh, and allow different animation speeds on each mesh, too.  That would be sweet!

BUT... perhaps it would be wise to ONLY allow this future UV Ani-Manager... to operate on planes and boxes. Got that coded yet, friends? 

Sorry for the long-winded and not-easy-to-teach situation.  That is not a very "finished" playground, either, but, I want to leave some of the fun... for others.   Hope this helps.  Welcome to the world of vertexData manipulating!  It's a powerful (yet fun) area of BJS.

[royibernthal]

Help me solve the updateFaceUV() and how to display sprites from the spritesheets properly and I'll write one hell of an animation manager

I could try to dig into understanding how UV works but it'd probably take me quite a while to learn everything I need to know in order to make this little calculation.

Hopefully @jerome can help with these issues as you suggested, if not - I have it added to my todo and I'll give it a shot as soon as I can.

[jerome]

For now, there's no way to change the faceUV values after the box construction (unless dealing with buffer vertex UV stuff). The faceUV property sets a part of the box geometry (how the UV are bound to the surface) and the box is not a morphable shape (parametric shape), so not updatable with its constructor method like ribbons, tubes, etc are.

[royibernthal]

That's a shame. Assuming that's not an issue as there's another solution for the updates as Wingnut wrote:

http://playground.babylonjs.com/#VNFRE#3

How would I calculate displaying a sprite of choice from the spritesheet?

e.g. how would displaying sprite #1 look like, #2, etc...

Hopefully you'll be able to solve this easily as you're already quite experienced in this subject.

[jerome]

However you can still use the faceUV to force only one face of the box to fit a single sprite from your sprite sheet and then ... just shift the texture for 1/6 (6 sprites per row) with uOffset : http://www.babylonjs-playground.com/#1V3CAT#171

and you will have your sprite animation on one box face 

[royibernthal]

Thanks  That's all I needed.

[Wingnut]

oh sure, where was @jerome when I did the exact same thing, but had X positional drift?  hehe

See how he used math (1/6) instead of my hard-coded distance?   (see line 37 in my pg - ugly guess-timated value)

yep, yep, yep.  I should have thought of that, too, earlier.  Dummy me.  Sorry R, and thanks Jerome.

[Wingnut]

  But we need to ALSO pay attention to what @NasimiAsl showed us a few posts ago.  What the hell was that all about? 

Is that animation happening on the GPU-side, Naz?  Nabs?  Anyone? I don't understand the GPU/CPU lines very well... but... I also saw Naz's continuous dynamic terrain thing... and I just fell in love with it.  Naz has a pretty blue home cylinder in it. 

And he also made that city-block-sized shopping plaza thing, too.... at melyon?  Yeah.  That is niiiiiiiiiice.

But... there's two things in any "fun world" that are maintenance loads, and that's keeping all the animations running, and drawing terrain dynamically.  DO them both at the same time, with at least SOME load taken-over by the GPU, and you got a nice starting platform. 

Shaderbuilder is excellent, but the more I watch Naz, the more I realize that he just writes excellent shaders.  He could do it without shaderbuilder, too.  But the guy knows shaders.  Phew.  I sure like having him around. 

[royibernthal]

I'm also interested in hearing more about his solution

Regardless, I created a class for UV animations - nice and simple and working as expected

I separated the animation logic from the box mesh, it extends BABYLON.StandardMaterial so that it can possibly be assigned to any mesh instance and even to different mesh types in the future - spheres, etc. (haven't tested other mesh types yet)

Also the same mesh instance could be assigned different animations. e.g. box.animation = animationA and afterwards change it to box.animation = animationB.

The only thing missing to make it perfect for a box is being able to modify the box's faceUV after it was created, so that it could be assigned spritesheets of different sizes and different colls*rows arrangements. I have no idea how to solve this issue.

@jerome @Wingnut There must be something more up your sleeve that can solve this

[Wingnut]

Cool!

Set mesh updateable, find those 8 UVs inside-of vertexData.uvs array, adjust them accurately using Jerome/math amounts, and write the uv data back into vertexData.uvs.  That will do the same update as a faceUV updater would do.  Can ya do it?  Bet ya can.  Maybe use a plane? (Plane = faster to change its vertexData.uvs).  Then position the plane against anything (like a box face)... attach it with Parent Glue... and party in the streets. 

You've just made a sprite that ISN'T stuck in billboard mode.  YAY!

Remember the earlier post... talking about 0%, 5%, 10%, 15%... traversing left-to-right across an image that is one sprite high and 20 sprites wide?  SUre ya do.  That SHOULD work for a 20-sprite-wide strip... as described.  It might all start with 2 you-code-em funcs... 

var tank = plane.getVerticesData(BABYLON.VertexBuffer.UVKind)
createNewUvBasedUponSpriteFrameNumber(frameNum, cornerIndex)  (run 4 times, one for each corner)
pushNewUvIntoTank(adjustedUV, cornerIndex)  (also 4 times)
plane.setVerticesData(BABYLON.VertexBuffer.UVKind, tank);

Uncle Roy's SpriteFlipper v1.0  Alright!  Easy,huh? 

[Wingnut]

Ya know... I've been thinkin'/testin'.  (oh no).

What would keep us from using a sprite2D on a worldSpaceCanvas2D?

Here is a nice-working screenSpace sprite2D... Pedro Pitfall.    http://babylonjs-playground.com/#20MSFF#35

And here's Wingnut trying to map it onto a worldSpace.  http://www.babylonjs-playground.com/#20MSFF#36

Yep, PG failures that YOU can count-on... that's Wingnut Playgrounds Inc. 

See the thin line just under the top red border... with animated sprite playing in that single line?  I do.

If we can get this wench working, this will be the EASIEST method to do animation on a customWorldSpaceNode (a plane) ... which can be parented to a face of a box.

Let's ping all the gods.  They love when I do that. 

Hey @Deltakosh (or anyone), can you think of an easy way to turn OFF billboarding for standard sprites... perhaps allowing them to track a plane's face?  Yeah, I know, a sprite is not a mesh in the classic way, and doesn't allow parenting.  But a sprite-tracked plane WOULD be parent-able. (thx)

Hey @Nockawa (or anyone), can you get this world space sprite animation to work?  (thx)

First one to do either... gets... well... Royibernthal's undying love and respect.    (Nice of me to give away your stuff for you, eh R?)  heh

[Wingnut]

http://www.babylonjs-playground.com/#20MSFF#39

I win.    Thanks anyway, non-winners.  heh.  (I had line 11 set to 0,0 and it needed to be set to 64,64)

Ok, this solution is not all that easy (takes a pretty good chunk of code), and the sprite is not going thru its FULL string of images (not the fault of canvas2d).  But still... a sprite without billboarding!  Yay!   I win.  Eat your hearts out, billboarded sprites.    You may now worship the water I walk-on.  hah

@royibernthal - when you first thought-up this challenge, did you EVER IMAGINE that we'd go thru this much hell... to get an animation on one side of a box?  Holy crapolas, eh?  Something SO used, SO needed, SO painful.  Phew.

edit:  line 11 really needs to be set 0, 64.  Please continue reading this thread for more information about that.