solved Confused about rotate and then move (first person)

[timetocode]

I think I may be using rotation incorrectly, but I'm not sure what I'm doing wrong. I'd ask a more specific question, but I truly have no idea how to narrow things down. So either I have a small stupid bug somewhere (could be...) or just a fundamental misunderstanding of something 3D, in which case I hope someone in this forum can point it out.

I've got a multiplayer first person shooter where the game client sends its keystrokes and camera rotation to the server. This is used to rotate the player's mesh (which is just a cube with a red face for forward) and then move the player around. The player is not limited to moving along the ground, and can in fact just look up into the sky and press forward and fly around... so I guess they're a little more like space ships that can strafe than  true first person shooter characters. Everything looked like it was working perfectly until I started comparing the client predicted positions to the server authoritative positions. Now I don't mean to drag anyone down the rabbit hole of netcode, and I strongly suspect my bug is something very basic relating to rotating a mesh and then trying to move it forward (or left, or right, or backwards).

To rotate the camera, I'm just capturing mouse movement and multiplying by a mouseSensitivity

input.onmousemove = (e) => {
    if (input.isPointerLocked) {
        camera.rotation.y += e.movementX * mouseSensitivity
        camera.rotation.x += e.movementY * mouseSensitivity
    }
}

I'm not sure if that is correct, but it looks and feels like a first person shooter.

Every frame the client produces a command that looks like this:

{
    forward: <Boolean> input.frameState.forward,
    backward: <Boolean> input.frameState.backward,
    left: <Boolean> input.frameState.left,
    right: <Boolean> input.frameState.right,
    rotationX: <Float32> camera.rotation.x,
    rotationY: <Float32> camera.rotation.y,
    rotationZ: <Float32> camera.rotation.z,
    delta: <Float32> deltaTime
}

This contains the state of the movement-related keys, and how long they've been held down (determinism note: holding 'W' for 0.187 seconds at speed 100 is a deterministic calculation which can be repeated by server and client to get the same result). It also contains the camera rotation, which I'm wondering if this correct..? I'd like to note that I've used this type of command structure in several games and that it is truly deterministic, but that I've never done this in 3D, so perhaps my error is just the rotation.

The actual movement logic

 move(command) {
    // rotate the mesh to the same rotation as the camera
    mesh.rotation.x = command.rotationX
    mesh.rotation.y = command.rotationY
    mesh.rotation.z = command.rotationZ

    // unit vector of our movement
    let unit = BABYLON.Vector3.Zero()
    if (command.forward) { unit.z += 1 }
    if (command.backward) { unit.z -= 1 }
    if (command.left) { unit.x -= 1 }
    if (command.right) { unit.x += 1 }
    unit.normalize()
    // rotating the unit vector to the context of this entity
    let heading = mesh.getDirection(unit)

    // full vector, movement and magnitude
    let velocityCoef = speed * command.delta
    let velocity = heading.multiplyByFloats(velocityCoef, velocityCoef, velocityCoef)

    // move
    mesh.position.x += velocity.x
    mesh.position.y += velocity.y
    mesh.position.z += velocity.z

    let y = ground.getHeightAtCoordinates(mesh.position.x, mesh.position.z)    

    // added a little padding to keep the cube off of the ground
    if (mesh.position.y < y + 1) {
        mesh.position.y = y + 1
    }
}

So the mesh gets rotated instantly to the rotation specified in the command. The command's rotation is just the camera's rotation. Is this a mistake..? Is copying the values of the camera's rotation vector TO the player mesh rotation not in fact what I should be doing to get the player to face in the same direction as the camera?

After that I do that unit vector stuff which encompasses what I consider to be first person-style movement with strafes: forward, backward, left, and right. The direction the mesh actually moves when all is said and done is calculated by mesh.getDirection, and then multiplied by the velocity. 

I worry about both the above ideas, because I've been combing this forum looking for babylon first person shooter examples, and they're all different from this and from one another. None of them copy the camera's rotation vector on the player, and none of them use getDirection. I see other people using matrices, local/global thingies, getfrontposition, and other properties of cameras. I feel like I'm missing something. And yet, my game SEEMS to work pretty well. It wasn't even until I started integrating the clientside prediciton layer that I started to notice that there were tiny inconsistencies with the positions of objects.

When moving the mouse and rotating the camera, there are no clientside prediction errors. If the client predicts that rotationX is 0.1234, the server always agrees. This is no surprise because the camera rotation is in fact client authoritative. 

When hitting W A S D or the arrow keys in any combination without rotating the camera, there are no clientside prediction errors. The length of any key press is known. The client produces its own local version of the movement. The server then receives the identical command and moves the player by an identical amount. At some point the client will receive a snapshot of server data that contains the state of the gameworld after having factored in the aforementioned command, at which point the client will know if the movement it predicted a few frames ago lined up with the server authoritative movement. This is the actual hard part of clientside prediction and I'm pleased to say it is working and even supports collisions with the terrain mesh. It isn't even the case that it sometimes has a mild error -- it literally never has an error, and this should be expected given that there are no collisions between players, so there's never any unknown state to the client that could potentially cause it miscalculate a position.

However when moving the mouse AND holding any of the movement keys, the predictions are incorrect every frame! Because of the way this engine is put together the position gets reconciled, and it recovers. But what is throwing off the deterministic calculation? A.) rotate a mesh to a specific XYZ, B.) find a forward vector C.) move a specific distance along that vector. I'm clearly not doing  A->B->C otherwise it would be deterministic and there would be no error.

I tried to recreate the movement code without a network, and got even more confused because setting the xyz of a mesh.rotation and then using mesh.getDirection to attempt to move it forward produced no turning in the BJS playground, the sphere just kept moving in one direction: https://playground.babylonjs.com/#LD0IDK#2. There are two spheres, one remains stationary, the other one is moved 3 steps by 3 commands each of which rotates the mesh before moving it forward. It ends up going straight, which I don't understand given that it is being rotated and that it moves in the direction it is facing (or that was my intention). So perhaps therein lines the issue, though if this playground doesn't work how does my game even do anything at all...

Here's a gif of what the camera controls and cube players look like (all of the visible cubes are bots that are sending random commands tho, so never-mind they crazy directions that they're facing).

 

 

Edited May 8, 2018 by timetocode

[timetocode]

After trying multiple ways of expressing the camera direction and the movement of the player, I think I've found a combination that allows for deterministic rotation+movement of the player.

Summary:

• camera.getForwardRay().direction instead of camera.rotation
• player mesh.lookAt a negative version of the camera forward ray instead of mesh.rotation = camera.rotation
• mesh.locallyTranslate(velocity) where velocity is a representation of WASD/arrowkeys instead of mesh.getDirection

This produces something that looks and feels very similar to what I had above, except that moving around while also changing direction now works without triggering any clientside prediction errors. I can't say that I know why, but I do suspect that what I had in my first post just sorta worked by accident, possibly because the network layer was reconciling the small differences in client/server state before they got big enough to create a visual defect (maybe...), and that this new implementation is more technically correct and does not need any reconciliation.

Here's the newer code

Clientside camera and sending input to server:

let cameraRay = this.renderer.camera.getForwardRay().direction

// sending data from client to server
this.client.addCommand(
    new PlayerInput(
        input.w, input.a, input.s, input.d,
        cameraRay.x, cameraRay.y, cameraRay.z,
        delta // this is the deltaTime for the frame in which the input occured
    )
)

The movement code (used both for the clientside prediction, and the server side movement):

this.mesh.lookAt(
    this.mesh.position.add(
        new BABYLON.Vector3(-command.rotationX, -command.rotationY, -command.rotationZ),
        0, 0, 0
    )
)
// unit vector of our movement
let unit = BABYLON.Vector3.Zero()
if (command.forward) { unit.z += 1 }
if (command.backward) { unit.z -= 1 }
if (command.left) { unit.x -= 1 }
if (command.right) { unit.x += 1 }
unit.normalize()

// full vector, movement and magnitude
let velocityCoef = this.speed * command.delta
let velocity = unit.multiplyByFloats(velocityCoef, velocityCoef, velocityCoef)
this.mesh.locallyTranslate(velocity)

let y = this.scene.ground.getHeightAtCoordinates(this.mesh.position.x, this.mesh.position.z)

// added a little padding to keep the cube off of the ground
if (this.mesh.position.y < y + 1) {
    this.mesh.position.y = y + 1
}

I'm going to be building two games out of this, one where the player moves around a bit like a spaceship (xyz), and another where the player moves around in a standard first person way (xz). For the spaceship I'm just going to add controls that allow for z rotation of camera (pitch,yaw, roll, w/e its called). For the first person shooter I'm just going to zero out the y component of the movement and make sure the unit vector purely moves along x, z with only the terrain and a jump affecting y. So if anyone has any general suggestions or tips about movement in general, I'm all ears .