controlling avatars using camera-like controls

[Srisub]

I really like the built-in camera controls in BJS (compared to Three), but what i really need is this - and it could be a really cool extension to BJS:

- Create an avatar with head -> face -> eyes
- Attach the camera to the eye
- Have controls that do this to the *avatar* parts, not to the camera. Camera will move implicity because its rigidly attached to the eyes
  - trackpad/mouse: move head up and down (and eyes, and camera along with it), move body left or right 
  - arrows - move body fwd/back/left/right

Thoughts on how to achieve this easily?

[Srisub]

In other words, its something like this:

Controller( avatarMesh, headMesh, eyesMesh ) 

and have configs that can attach various input mechanisms to rotation/translation to one of these meshes.

In my case, i want to do this:

- Trackpad/Mouse left/right - rotate avatarMesh left or right. Others may want to move the head
- Trackpad/Mouse up/down - rotate headMesh up/down
- Arrow keys - translate avatarMesh fwd/back/left/right

This, btw also lends itself really well to VR, where eventually there will be eye-tracking etc that can affect eyeMesh.

[Srisub]

Sorry for the stream, but here's what i can do as a simple measure for now:

- Set FreeCamera's initial position to avatar's eye
- Let the user do their input thing and camera matrix changes
- In the rendering loop, check if camera's position/rotation changed, apply that to the resp. meshes instead
- reset camera matrix, so its always pointing out straight from eye

[JCPalmer]

I sort of understand, but you seem to be using non-BJS terminology (maybe 3js?).  In the long run this is going to hurt you.  You attach the camera to a scene.  You can set the target of a camera to a mesh.

Getting past that, maybe.  First you have 2 eyes.  You cannot really use the target facility of the camera, here.  There is a also a billboard property of a mesh that can be used.

I did a test scene where eyes followed you.  Is that what you want?  You have so many words here, it is hard to tell.  Here is a thread I did to show the test scene.  FYI, I have still not fixed the random eye movement, so do not try it.

[Srisub]

Sorry for not being clear. 

I am building a 3-d chatroom with avatars. When a given user controls their camera, i want his avatar to change (rotate head up/down, rotate whole body left/right, translate whole body, etc), and that change be transmitted to the others in the room. For example, if the user moves camera forward, i want his avatar to move in others screens. If the user rotates camera up, i want the head of the avatar to look up in others screens.

In the Three.js version i wrote my own controller for the transforms for each of the avatar's components, and the camera which was stuck to the eye, moved passively with the changes to the avatar. Clearly, need to do something different in Babylon.js, but ideally with similar effects described in above para.