solved Oculus Go first impressions & questions

[olsibob]

I borrowed an Oculus Go from work, here are some first impressions/ questions.

Overall I'm really impressed with the hardware. It's comfortable to wear, resolution and FPS seem fine to me. My only other VR headset experiences have been cardboard and a Samsung one (forget which it was), Oculus Go is better than either of those.

As a dev device, I've not downloaded any SDK or looked at any dev doc, I just point the device's browser at http://<my laptop's IP address>:8080 and off I go (no JS console tho!)

It's amazing how much the following one line of code gets you:

const vrHelper = scene.createDefaultVRExperience({ createDeviceOrientationCamera: false });

it gives you a VR icon on the screen. When you tap it, Oculus Go launches into fullscreen VR. Looking around looks great, works seamlessly out the box.

Worth noting that Oculus Go is a 3 DOF system, rather than 6 DoF. Meaning that it only tracks orientation of the headset (and the controller too? Need to double-check this), not its position.

Where I have run into some difficulties is with the controller. I'm currently trying to implement so that the user can point at a mesh with the controller, and pull the trigger to select a point on that mesh. I'm going to list some of these in this thread. Bearing in mind I've only had the device for a few hours. If anyone has suggestions for these issues, would be great to hear.

1. getting PickingInfo

So far I've only been able to get PickingInfo with

vrHelper.onNewMeshPicked

this only fires when the pointer first moves over the mesh, regardless of whether the user is pressing one of the buttons or not. The callback for picking up button changes

controller.onButtonStateChange

returns info about the button, but no pickInfo. I guess I need to call scene.Pick inside the button state change callback, haven't had time to try this yet.

2. customizing the controller mesh

When you enable interactivity with 

vrHelper.enableInteractions();

The default mesh for the controller is a long, thin cylinder that stretches from where you're pointing, right up to your eyes. It's really in-your-face, and I haven't found a way to modify or disable it yet. This doesn't have any effect:

vrHelper.displayGaze = false;

And neither did trying to apply a custom mesh with

controller[0].attachToMesh(mesh)

I've got to be offline for a few hours now, but I'll keep experimenting. If anyone has questions, tips, please do post.

 

[brianzinn]

8 hours ago, olsibob said:

So far I've only been able to get PickingInfo with

Did you try the using scene observables for pointer events? This works for me for mouse and then controller trigger in VR mode (on meshes and GUI):

scene.onPointerObservable.add((evt) => {  // evt: BABYLON.PointerInfo
  if (evt && evt.pickInfo && evt.pickInfo.hit && evt.pickInfo.pickedMesh) {
    let mesh = evt.pickInfo.pickedMesh

    console.log('you picked', mesh);
  }
}, PointerEventTypes.POINTERDOWN);

 

8 hours ago, olsibob said:

The default mesh for the controller

For each of the controller types, you can override the meshes used by setting static properties.  For the Oculus Go, you will also be overriding the mesh for GearVR.  Here is the link:

BABYLON.GearVRController.MODEL_BASE_URL = '/assets/' // or http://www.site.com/assets/
BABYLON.GearVRController.MODEL_FILENAME = 'oculus-go.babylon'; // or wherever

It's a bit counter-intuitive and buried, but the Oculus Go is using the Gear VR, because I think the person who added Oculus Go did not want to duplicate code unnecessarily.  The side effect is both GearVR and Oculus Go share the same mesh and Oculus Go uses "GearVRController".  Here is where that association is made, if you wanted to make changes there:
https://github.com/BabylonJS/Babylon.js/blob/master/src/Gamepad/Controllers/babylon.poseEnabledController.ts#L92

Above is if you want to change the controller mesh for Oculus Go.  If you want to change it for all controllers then you should check out the fruit ninja game that uses sabers:
https://playground.babylonjs.com/#FAXLY2
search for switchControllersModelsToLaserSaber()

8 hours ago, olsibob said:

and the controller too? Need to double-check this

The gear VR and Daydream controllers are also 3DoF (orientation), so the position is inferred.

I wrote a post on how to debug GearVR over WiFi - not sure if it will help you out on the Oculus Go if you are still having problems with controller buttons then it may be a trigger mapping - I should add that I have never used an OculusGo.
http://www.html5gamedevs.com/topic/35003-basic-gearvr-controller-support-and-wifi-debugging/

 

[olsibob]

@brianzinn thanks for that, loads of things for me to try out there. On your first point, I do have scene.onPointerObservable set up, and it doesn't seem to be firing when in full VR mode (though it does when in "browser" mode, when you're looking at a virtual screen showing the web browser), I'll need to double-check that it isn't firing.

[olsibob]

Yeah, scene.onPointerObservable definitely doesn't fire on the Oculus Go when you're in full VR mode (but does fire in browser mode).

That annoying cylinder I see poking me in the eye can be turned off with controller.mesh.setEnabled(false)

What's weird, is that when I go to the Fruit Ninja playground, that same controller mesh is an actual model of the controller (looks a bit different, maybe its a model of the GearVR), kind of like what you see elsewhere in the Oculus Go UX. How come the controller.mesh I see is just a long thin cylinder, but the one in the playground is an actual model of the controller? I'm running on the npm version of BabylonJS 3.2.0, could it be that the playground uses a slightly different version, with different bundled assets?

[olsibob]

Oh wait a minute I think I know what it is. Logging those MODEL_FILENAME that you posted, they all point to .babylon files. Maybe its the same issue I'm having on the .env thread of not being able to serve certain file types, and that long thin cylinder is a fallback.

[brianzinn]

The long cylinder is a ray - I think we show that so that the controller is usable when you ie: have no internet to download the model.  I think if you sort out the mime types that it may just start working

edit: actually that model isn't loading from your server by default, so that should not be why you are not seeing the mesh!  I don't think you need the 'babylonjs-loaders' npm for .babylon files.

[olsibob]

Findings so far:

1. PickingInfo

Sadly, pointerObservables don't work on the Oculus Go when in full VR mode (apparently other headsets do map the controllers to pointer events). However, you can get pickingInfo using the controller's forward ray, eg:

vrHelper.onControllerMeshLoadedObservable.add(controller => {

 

controller.onButtonStateChange((controlIndex, buttonIndex, button) => {

const ray = controller.getForwardRay(48);

const pickInfo = scene.pickWithRay(ray, mesh => { return !mesh.name.includes("laserPointer"); });

If you're showing the laser beam, you'll need to exclude it from the ray's mesh predicate.

If you're using a gazeTrackerMesh though, you could just use the position of the tracker mesh, and save yourself an extra raycast.

2. customizing the controller mesh

Babylon attempts to load a controller model from a .babylon file. This fails for me, possibly because of a mimetype issue with .babylon files. As a fallback, a long thin cylinder is shown. This is quite ugly though, as the origin of the cylinder is your head, so it pokes you right in the eye. controller.AttachToMesh has no effect in this case, the cylinder remains visible (unless I've misunderstood the API this doesn't seem like the desired behaviour, I think I'd like to raise this as a bug). To get round this, I converted the controller.babylon file to gltf using a playground https://playground.babylonjs.com/ts.html#D1N0MZ#4 put the GLTF files in my project, and override them as the default controller the method suggested by @brianzinn:

BABYLON.GearVRController.MODEL_BASE_URL = "assets/geometries/vrController/";

BABYLON.GearVRController.MODEL_FILENAME = "controller.gltf";

It feels like I should be able to use controller.AttachToMesh to use the gltf model rather than overriding these static values.

The next thing I need to work out is:

3. how to offset the position of the controller

Oculus Go is a 3DoF system, meaning that the position of the controller is somewhat arbitrary. Throughout the rest of the Oculus UX, the controller appears in the lower-right corner of your field-of-vision, as if you're holding it in your righthand at around waist height. In WebVR mode though, it's position is right in front of your face. So far I've just been manipulating the pivot matrix of the controller mesh to move it down to the corner. This isn't the correct way to do this though, because if I rotate the controller in my hands around its forward axis, the onscreen controller will loop around to the top of the screen, because of the large pivot offset I've given it. I'm sure there must be some attribute somewhere that I need to set (I thought I might be able to set controller.devicePosition but it doesn't appear to have an effect), I'll continue digging.

Massive thanks to @brianzinn for all his help so far!

[olsibob]

Latest update: All my issues with the Oculus Go controller have been fixed by moving from babylonjs 3.2.0 to the latest beta 3.3.0b4. I no longer need to supply my own GLTF controller model, or mess around with the pivot point to reposition it. 

While googling, I found this PR which addresses the issue I was having above with the controller not being down by your right hand side and so on https://github.com/BabylonJS/Babylon.js/pull/4289/files , then realised it was merged in July, after the 3.2.0 release.

[brianzinn]

oh - unlucky you had to go through all of that, but sometimes lots in learned in those experiences.  I also didn't realize that support was added after 3.2.  I think you were using the fallback "generic" controller.  So, the position of the controller was {0, 0, 0} - on your eyes.  That was how the GearVR controller worked before it was supported.  The WebVR spec properties like hasPosition and hasOrientation can be used to support 3 DoF by default, but I don't expect that will be done soon.  WebXR will probably be the next push now that it has partial support.  Glad you got it working!

[olsibob]

Definitely learning a lot! Thanks for your help with this

[Guest]

This is really cool! Do you think this could be worth adding a specific classfor the OculusGo controller?

[brianzinn]

I don't know if you are asking me, since I mentioned some specifics. I think the second part of https://github.com/BabylonJS/Babylon.js/issues/3514 would make it easier for new controllers to be added and customised

[olsibob]

I don't have a GearVR to hand, so I don't have a sense of how similar they are (GearVR is described as "powered by Oculus" and it's also 3DoF). I'm very happy so far with how the Oculus Go controller works in babylonjs 3.3.0b4, it feels very close to the native controller UX on the device.

[Guest]

@brianzinn I asked everyone  But yes this would help..We just need to find a volunteer to tackle it. But I wondered if it could make sense to have an OculusGoController (perhaps helping at least with discovery?)

 

[brianzinn]

I think there is enough code in that class now that what you gain from discoverability you perhaps lose in maintainability, so I don't have an opinion.  If all of the 3dof logic was handled in a common class then I would have a stronger opinion.  For example, the daydream controller functions nearly identically and did not gain from the various improvements that GearVR did, so I wonder if the same would happen with a new class diverging 

[Guest]

well we could use inheritance to fix that

DayDream and OculusGo could simply inherit from GearVR