Loading animations from separate files

[Jorisshh]

Babylonjs has been great so far, but yesterday I ran into an issue that I can't seem to fix.

 

Problem:  I'm trying to load in additional animations from .babylon files, however, when playing the animations on the original character it seems the original character's blend pose interferes with the newly loaded animation.

 

The main character is a skinned mesh in a T-Pose without any animations.

 

I then load in the walking animation (code snippet below)

 

Loading the main character:

BABYLON.SceneLoader.ImportMesh("", "./assets/models/babylon/", "character.babylon", this._scene, 
    function(meshes, particleSystems, skeletons, animationGroups){
      
        // Init character
    });



// later

BABYLON.SceneLoader.Load("./assets/models/babylon/", "[email protected]", t._engine, function(s) {
	
    // Copy all skeletons			
	let children = t._player.getChildMeshes();
	for(let i = 0; i < children.length; i++) {
		if(children[i].skeleton) {
			children[i].skeleton = s.skeletons[0];
			// children[i].updatePoseMatrix(s.skeletons[0].getPoseMatrix());
		}
	}

	t._scene.beginAnimation(s.skeletons[0], 0, 59, true);
});

 

However, I get this result. The t-pose is blended into the character animations causing this super wide stance; Which would suggest the walking animation transforms are relative? Can I make them absolute?

 

Loading the [email protected] file at the start makes it behave as expected. I simply want to load it in at a later time. Preferably I wouldn't change the skeleton or bones but instead, copy over the animators onto the main rig, but this didn't seem to work either.

 

Any help would be greatly appreciated.

[Guest]

Hello and welcome!

So let me try to rephrase it: if you load both files and apply the skeleton before rendering anything it works?

 

If this is the case, can you try to call this code before affecting the skeleton:

mesh.updatePoseMatrix(BABYLON.Matrix.Invert(mesh.getWorldMatrix())

 

If this does not work can you make sure that there is NO animation or bones in the original file?

Also worth trying: set mesh.needInitialSkinMatrix = !mesh.needInitialSkinMatrix

 

If nothing works, I will need a repro in the playground

[dbawel]

@Joris Huijbregts

I've previously experienced problems starting some specific group animations using Chromium and Electron. My scenes and results were different from above, but I've observed issues in varying forms. I need to create a repro of a couple of problematic scenes and see if this is still the case. But I thought I'd make mention of this in case the problem is specific to a browser or OS. I'll try and test on my known scene(s), and report if there still exists any issues.

DB

[Jorisshh]

@Deltakosh Thanks for the quick reply.

 

This issue happens every time I load in an additional animation for the character, even if it happens before rendering.

I've tried

mesh.updatePoseMatrix(BABYLON.Matrix.Invert(mesh.getWorldMatrix())

as well as

mesh.needInitialSkinMatrix = !mesh.needInitialSkinMatrix

But no luck

 

I've created a playground sample scene where I replicated the issue (happens in both 3.2.0 stable and 3.3 alpha)

https://playground.babylonjs.com/#2FP2YG#9

[Jorisshh]

@dbawel Thanks for the suggestion. I've just tried out Edge, Chrome, and Firefox and it seems consistent between all browsers. I've created a playground example at https://playground.babylonjs.com/#2FP2YG#9

[dbawel]

@Joris Huijbregts

I've seen this all too many times, although I still played around with your PG scene to no avail... however I didn't expect to affect any change to meet your desired applied animation. What is happening is that you most likely are applying your animation(s) to a skeleton which already has transforms you don't want applied in your 'T-Pose'. So even though there is no animation on the 'T-Pose', you're applying the new skeleton animation to existing transforms. This is very common depending on how you create, import, set transforms on bones, animate, export, etc. in your 3D application. 

Without working directly with your 3D scene in it's native application, I can't personally correct. However, you can. I highly recommend that you first reset, and then freeze all transforms before exporting your mesh, skeleton, and all elements to the .babylon format. If I had to guess, I might assume you have a scaling value in your mesh and/or bones of a value other than 1.0. any other value will multiply the scaling value to affect any transform animation by the very same value. However, it could be a scaling value other than 1.0, a rotation value other than 0.0, and/or a translation value affecting your animation. Remember, all transforms are multiplied into any animation you're applying to your skeleton; including a scaling value on any part of your mesh or it's skeleton - as bones typically act as children to the mesh they are bound to. Only a few applications such as Houdini allow you to alter these relationships. Also, check all transform values on your animated skeleton, as this will also affect the bones transforms. 

So rule of thumb every time... first reset all transforms on your mesh(s) and skeletons (bones), and then always freeze all transforms before animating... but also always reset, freeze, and personally verify all transforms before exporting your mesh and bones to another application (and any additional skeletal animations) just for good measure. This should solve the behavior(s) you're currently viewing. If not, then I can look at your scene, and this should be simple to solve regardless. Working with transforms other than 1.0 (in scaling and translation) - and 0.0 (in rotations) is the most common mistake seasoned animators make, so we all live and learn and will continue to do so until we can no longer move a mouse across the mouse pad.

Let me know if you still have any issues, or if you have questions concerning this.

DB

[Jorisshh]

@dbawel Thanks, for your detailed answer.

 

The t-pose skinned mesh had some rotations on the arms and translation on the root node, so I removed these, however, this didn't resolve the issue. Shouldn't the newly loaded animations overwrite the entire transform of the bones? Why is the final result t-pose transform + new animation transform?

 

I've done the following to make the most simple setup possible, but it still doesn't work.

 

- Removed all translations and scalings from the skinned main mesh.

- Created a super simple animation for the character where he rotates his upper body.

- Loaded in the new animation

 

When loading in the new animation and swapping out the skeletons (see code snippets above), it still doesn't work.

 

Look around animation when loading the character directly (what it's supposed to look like):

BABYLON.SceneLoader.ImportMesh("", "./assets/models/babylon/", "[email protected]", this._scene, 
	function(meshes, particleSystems, skeletons, animationGroups){

});

 

 

When first loading the skinned base mesh and later loading the look around animation

// Loading base mesh
BABYLON.SceneLoader.ImportMesh("", "./assets/models/babylon/", "character.babylon", this._scene, 
	function(meshes, particleSystems, skeletons, animationGroups){



   // Loading animation
   BABYLON.SceneLoader.Load("./assets/models/babylon/", "[email protected]", t._engine, 
   function(s) {

        // Replace bones and play animation
		let children = t._player.getChildMeshes();
		for(let i = 0; i < children.length; i++) {
			if(children[i].skeleton) {

				children[i].skeleton = s.skeletons[0];
				// children[i].updatePoseMatrix(children[i].getWorldMatrix().invert());
				// children[i].updatePoseMatrix(s.skeletons[0].getPoseMatrix());
			}
		}


		t._scene.beginAnimation(s.skeletons[0], 0, 59, true);

   });

});

 

Again, loading in the look around animation (or walk, idle, etc) works fine. The problem only occurs if I first load the base character, replace the skeleton and then play. 

[dbawel]

What 3D app is your model exported from? If it is a blender file, please attach and I can look to see what the issue may be. Otherwise, there's no way of debugging.

DB

[Jorisshh]

@dbawel @Deltakosh I used Maya to export the characters. Is there a guide for Maya or some standard workflow that we could adhere by?

 

I decided to try blender, But I ultimately get similar results. I've attached the 2 blender files to this post; 1 is the t-pose and the other a sneaky walk animation that just doesn't play right (if loaded 2nd).

 

I don't think it's an issue with the animations though. Loading in the animation directly works just fine; Loading in the t-pose and then attaching the animation messes it up. All animations work as expected when directly loaded, just not when I assign the skeleton to the t-pose character (standard rig from Mixamo, on both the t-pose and the animations). Do I need to re-index the bones or something?

character.blend

[email protected]

[Guest]

Ok investigating!

can you confirm that if you export the animation within the first file it works?

if yes can you provide a playground loading that file?

[dbawel]

@Jorisshh

As @Deltakosh requested, if you can reproduce the problem on the Playground, I'm certain we'll find a solution for you. But without a PG scene repro, it's just guessing. I'm certain it's an additive transform issue, but as soon as I say I'm certain about this is when we'll find some unknown bug. It's rare, but does happen. I wish I had time to repro, but right now there's no time for sleep as I have to deliver a beta app and am already running behind. But will take the time to make sure we solve this as a community.

Cheers,

DB

[Jorisshh]

@Deltakosh I don't think the problem stems from the files themselves so much, loading in animations directly works perfectly fine. The problem only arises when I first load the t-pose and then load the animation file and replace the animation skeleton with the t-pose character.

 

Complete repo case:

• Load t-pose character (no animations, just mesh + skeleton)
• Load walk animation
• Swap out t-pose character's skeleton with walk animation skeleton (same skeleton / same bones)
• Play animation on walk animation skeleton (now attached to t-pose character) -> Causes strange interference of t-pose in the animation

 

I created 2 playgrounds

• Loading t-pose, then loading animation and swapping out the skeleton of the loaded animation (t-pose influences animation): https://playground.babylonjs.com/#2FP2YG#9
• Loading mesh + skeleton + animation directly (single file) (works as expected): https://playground.babylonjs.com/#IIGMMJ

 

@dbawel I greatly appreciate it. If there's anything I can do to help out please let me know.

[Guest]

Wait there is something wrong that I missed

When you call SceneLoader.Load, a new scene is created and not merged! You have to use SceneLoader.Append:

https://playground.babylonjs.com/#2FP2YG#10

And if I stop all animations, I see that the initial state (the exported state) of the mesh is not the same in both files: https://playground.babylonjs.com/#2FP2YG#11

 

[Jorisshh]

@Deltakosh I tried using both Load, LoadAssetContainer (with & without addAllToScene) and Append, but this doesn't seem to make a difference.

 

14 hours ago, Deltakosh said:

And if I stop all animations, I see that the initial state (the exported state) of the mesh is not the same in both files: https://playground.babylonjs.com/#2FP2YG#11

 

This is just the 1st keyframe on the walking animation though? So naturally, it's different. The rig is exactly the same, both come from Mixamo.

 

Is there any demo, existing concept of loading in animations that I can use? I really don't see how swapping out an identical skeleton can cause this behavior. If anyone knows about a source I can work from I'll gladly take it.

[dbawel]

Hey @Jorisshh

I've spent the past few hours trying to figure out why I'm not able to import any mesh using BABYLON.SceneLoader.ImportMesh(); as I always assume it must be an error on my part. However, I've found a deeper issue specific to my machine, server, etc. or some other element specific to me. To provide more specific info for @Deltakosh or @davrousto possibly provide additional advise, the error I receive is:

Quote

main.js:60 TypeError: Cannot read property '_addPendingData' of undefined
    at Function.SceneLoader.ImportMesh (babylon.3.2.js:67461)
    at createScene (main.js:15)
    at main.js:66

So I'm unable to test any scenes at this point relating to your animation transform oddity. I did find several potential issues with your scenes, and reset some incompatibilities in the setup of your armatures and t-pose transforms. However, I have no way to test, so I won't go into these unless they prove to solve any of the obvious problems you're having. So I need to figure out why I'm unable to import meshes at this time, as I'm able to do so on the Playground, but not on my personal server. I've attached two .babylon files I've modified and exported for you to test. If you are able, please test these in the Playground and see if there is any improvement. If so, I'll walk you through the changes I made to the fies. Regardless, I need to solve my own issues concerning the import of meshes. Then I'll be able to assist further if still necessary.

The two files are 't_pose.babylon' and 'sneak.babylon' both exported from Blender and generated from your 'character.blend' and your '[email protected]' files respectively. Please let me know when you've been able to test these, as I hope I've made progress which would assist others as well. I hope you're having a great weekend, and we'll get this solved sooner than later - once I've solved my own server and/or client-side issues.

Let me know if you have any immediate questions or need some advise in importing these; although you obviously have a firm grasp of the export/import process. Good luck, and it would ake my day if we can solve both yours and my own problems.But that's why this forum and community are invaluable; as we are all in this together and all constantly learning. You can rename the .log files to .manifest files if you want to avoid any error messages concerning missing .manifest files.

Cheers,

DB

t_pose.log

sneak.babylon

sneak.log

t_pose.babylon

[Jorisshh]

@dbawel Sadly these animations gave a very strange result:

 

 

I'm still looking for potential solutions; And even though I get 'closer' the animations never look a 100% correct  - Thanks for all your help; Starting to love this community already

 

As for your ImportMesh issue, I think the scene you've passed into the ImportMesh call is null, causing the error.

[dbawel]

@Jorisshh

The scene appears to contain null properties. However, I've not been able to find out how these 'null' properties are generated. I'll install the latest .babylon exporter, as I know the version I'm currently using hasn't been updated since 12/2017. I receive errors in the log file for the t_pose scene with the mesh, so I still believe it's a configuration issue. Hopefully I'll find time to look further into this, as I can't sleep when something's broken - as devs such as @Wingnut and @Dad72 can attest to since they find me up at all hours of the night. Let's keep this post alive and get this one solved.

DB

[Dad72]

Use this before creating the scene:

BABYLON.SceneLoader.CleanBoneMatrixWeights = true;

And this for the creation of your characters

character.updatePoseMatrix(BABYLON.Matrix.Identity());

This solved for me similar problem

[Jorisshh]

@Dad72 Thanks for the suggestion, but sadly this didn't fix it either

I tried updating the post matrix before and after loading in the new animations, but this didn't make much of a difference.

Here's the playground demo:

https://playground.babylonjs.com/#2FP2YG#14

 

I've tried both my own animations, as well as the ones @dbawel suggested.

 

 

EDIT:

A found a clue (it's something...). When enabling the skeleton debug mode I can confirm that the skeleton plays the animation as intended. The mesh just doesn't deform correctly with the skeleton.

[Dad72]

I look your files and I think there is a problem of conceptions. 

Watch how I did it here for animation loading on the fly :

http://www.babylonjs-playground.com/#GA8XI#12

You must also make sure that the bones of your first character are the same number as those in the file containing the animations.

Use : createAnimationRange and copyAnimationRange

 

[Jorisshh]

1 hour ago, Dad72 said:

Use : createAnimationRange and copyAnimationRange

 

 

Sweet baby Jesus  I got it working with the createAnimationRange and copyAnimationRange thanks for sharing your demo!

Not sure why a skeleton swap doesn't work. If I find out I'll be sure to report back

[Guest]

Thank you @Dad72! 

[Dad72]

Delighted to have been able to help

[saygidagerek]

Hi everyone,

Help me Pls about Babylonjs animation. I have a file with babylon extension. I want this file to move as animation only in X and Z positions.

 

BABYLON.SceneLoader.ImportMesh(
                "",
                "/assets/",
                "file.babylon",
                scene,
                function (meshes) {
                    meshes[0].position.x = -123;
                    meshes[0].position.y = 0;
                    meshes[0].position.z = -21;
                 meshes[0].scaling.scaleInPlace(10);
                 
                 const frameRate = 10;
                 const xSlide = new BABYLON.Animation("xSlide", "position.x", frameRate, BABYLON.Animation.ANIMATIONTYPE_FLOAT, BABYLON.Animation.ANIMATIONLOOPMODE_CONSTANT);
                 const keyFrames = [];

                    
                    
                        keyFrames.push({
                            frame: 10, //İleri
                            value: 120
                        });

 

                        keyFrames.push({
                            frame: frameRate,
                            value: -160
                        });

 

                        keyFrames.push({
                            frame: 10 * frameRate, //Geri
                            value: 112
                        });
                        xSlide.setKeys(keyFrames);

 

                        meshes[0].animations.push(xSlide);

 

                        scene.beginAnimation(meshes[0], 0, 3 * frameRate, true);

                });