Search the Community

Basic support for Google Daydream controller PR was merged into BabylonJS, so should be in alpha11. Adding Daydream was more enjoyable and super easy compared to Gear VR. Sorry it's a long rambling post - it took me more time to write this forum post than the controller code itself! TLDR: Daydream controller has basic support (buttons work) - there's additional info here if you wanted to work on that or add a new controller. If you want to test your Daydream controller enable two flags in chrome://flags. As announced last month, BabylonJS added basic support for Gear VR. I put some instructions there for debugging GearVR as well. Setting up your phone for Daydream VR. I have a Pixel 2 and even with the Pixel 2, surprisingly you need to change some settings to get a good experience. Before I enabled these flags it was not an immersive experience even full screen (as I was testing it). That's unfortunate, but I think these features will be enabled by default in the future. Open up Chrome and visit chrome://flags to enable feature toggles for Gamepad extensions and WebVR: The nice thing about the Daydream Headset is that you can use it while plugged into USB for ADB over USB. For Gear VR you needed to debug over WiFi by design. You will need to follow instructions to enable USB debugging on your phone and the ADB instructions are in the GearVR post - just remember that WiFi is optional for Daydream. Should end up looking like this on your "Remote Devices" tab on Chrome on your desktop: The first time I plugged in the headset basically shows what the fallback (Generic) controller will output. This is helpful for you to figure out what indexes the buttons are in the Gamepad API. Here's the output: Pressing the home and app buttons did not output anything, so that lets you know that they aren't connected to the Gamepad API. The buttons do work on your phone, but for built-in OS features. What I did was grab the ID of the controller by adding a console.log and then create a new controller class. Here is the full output of what is generated from the Gamepad: What I want to do here is also outline steps to follow to add your own controller. There was a post about an unsupported 6DoF controller that led to a github issue (future feature) tagged "Help Wanted" (https://github.com/BabylonJS/Babylon.js/issues/3514) to basically allow devs to dynamically register support for new controllers and it will probably work similar to how the Model Loaders work, so your registered controller class would get asked if it supports: ie: "Daydream Controller", but we are not there yet. Not to get too side tracked, but the cool thing about that idea is we could use NPM to create and load new controllers; maybe even all VR and make BabylonJS core even smaller. I think also a notion of priorities may be needed to override defaults vs. custom vs. generic, but I digress... Right now you need to manually edit some files to allow your custom Controller to run instead of the Generic controller. Looking at the PR you can see there's only a couple of lines of code to wire this controller in - it only needs to be detected before the generic fallback to be used. Assuming that your new controller supports 'pose' in the gamepad API then you could hook it up here: https://github.com/BabylonJS/Babylon.js/blob/master/src/Gamepad/Controllers/babylon.poseEnabledController.ts#L46 An astute reader who hasn't fallen asleep this far into the post may have noticed that 'hasPosition' is false and 'position' is null. That's basically saying that the controller does not support it's physical location in 3D space (it only knows gyro rotation). Typically when discussing 3DoF the position is available and nothing else (vs 6DoF with position & rotation) - here we are talking about rotational data being available and positional info not. ie: Vive has Lighthouse tracking and WinMR tracks controllers from HMD for accurate position of the controller in space and are both 6DoF. If you look in the BJS source and come across this in the WebVRCamera you would notice that the hasPosition property is "missing": https://github.com/BabylonJS/Babylon.js/blob/master/src/Cameras/VR/babylon.webVRCamera.ts#L12 That's because VRPose and GamepadPose are different. Here is the spec on GamepadPose (not in BJS source, but should be at some point because right now the parameters are typed as 'any'): https://developer.mozilla.org/en-US/docs/Web/API/GamepadPose OK, so I'll come back to that in another post as I want to in the future make this 3DoF controller a 6Dof experience. Some very naïve code that does not work would look like this, but I feel the changes belong in WebVRController and not in each Controller... this.capableOfPosition = vrGamepad.pose || vrGamepad.pose.hasPosition === true; if (!this.capableOfPosition) { // Device not capable of position (ie: 3DoF rotation only). // Position estimated from VR experience helper default height 1.7m. let controllerPosition = new BABYLON.Vector3(0.5, -1.0, -0.25); if (this._defaultModel && this._defaultModel.getScene().useRightHandedSystem) { controllerPosition.z *= -1; } controllerPosition.scaleInPlace(this.deviceScaleFactor); console.log('setting default controller position:', controllerPosition); this._defaultModel.position = controllerPosition; } So, next steps are to create the 6DoF experience to get away from rays coming from your eyes (think superman heat vision). It will be based, I think, by guessing first where your elbow is. The elbow is following your head rotations as your HMR rotates (and torso). Probably moving up a bit with your HMD and backwards when you look down. Then with the known gyro rotation of the controller you just position the controller where the hand would be based on that rotation and elbow position and forearm length. I'm going to try a few more experiments within the Daydream and Oculus environments to see how they position the elbow and when they guess your torso is rotating (I think camera intertia will play a role in that movement - like ignore low inertia maybe). I have to look again, but I don't think the default height and model scaling are available from VRExperienceHelper, but they need to be - they're definitely not in the 'update' method, but I think those need to be taken into account for a good experience. Controller models are still needed (also for GearVR). Luckily google VR has made models available under Apache 2.0 (one is low poly with texture) and I think we can bring those in - I'll let the license experts decide: https://github.com/googlevr/gvr-unity-sdk/tree/master/Assets/GoogleVR/Models https://github.com/googlevr/gvr-android-sdk/tree/master/assets/controller Anway, first goal is reached that both Samsung and Google phone controllers buttons can interact through BJS. Of course, anybody else is fully welcome to implement those changes if they wanted. Let me know here so we don't do double work. I'll do another post later on when I start on the 6DoF stuff, but pretty busy for next 3-4 weeks. Cheers.