VideoTexture not working on ios

[Asagi]

Hi!
I have a code:

var myVideo;            // Our Webcam stream (a DOM <video>)
                var isAssigned = false; // Is the Webcam stream assigned to material?

                var videoMaterial = new BABYLON.StandardMaterial("texture1", Game.scene);
                    videoMaterial.emissiveColor = new BABYLON.Color3(1,1,1);

                    // Create our video texture
                    BABYLON.VideoTexture.CreateFromWebCam(Game.scene, function (videoTexture) {
                        myVideo = videoTexture;
                        videoMaterial.diffuseTexture = myVideo;
                    }, { maxWidth: 256, maxHeight: 256 });

                    // When there is a video stream (!=undefined),
                    // check if it's ready          (readyState == 4),
                    // before applying videoMaterial to avoid the Chrome console warning.
                    // [.Offscreen-For-WebGL-0xa957edd000]RENDER WARNING: there is no texture bound to the unit 0
                    Game.scene.onBeforeRenderObservable.add(function () {
                        if (myVideo !== undefined && isAssigned == false) {
                            if (myVideo.video.readyState == 4) {
                                Game.videoLayer.material = videoMaterial;
                                isAssigned = true;
                            }
                               
                        }
                    });

 

But video not playing in ios browser (safari). Access to the camera was authorized. What i do wrong? Thanks.

[Sebavan]

could you reproduce in a playground ? this would help troubkeshooting.

[Asagi]

I getting code from this example. This example not work on IOS (safari) to .

https://www.babylonjs-playground.com/#LIVJ8#0

[Sebavan]

i will take a look asap.

[Sebavan]

Could you try this out https://www.babylonjs-playground.com/#SQ5UC1#1 and let me know if it works ?

[Sebavan]

Or this one which is way smaller:

https://www.babylonjs-playground.com/#CHQ4T#1

You need to tap to start playing.

[Asagi]

 

Quote

 

Could you try this out https://www.babylonjs-playground.com/#SQ5UC1#1 and let me know if it works ?

 

Or this one which is way smaller:

https://www.babylonjs-playground.com/#CHQ4T#1

You need to tap to start playing.

 

These examples work.
The video-texture does not work from the webcam of mobile device.

[dbawel]

@Asagi

Unfortunately, Apple in their infinite wisdom has locked all external apps from utilizing any apple webcam. The only work around in real time is to hack an external webcam which for me is useless. Apple has allowed Skype as the only 3rd party I'm aware of to use their webcams in real time. Otherwise, the best solution I've found is to use one of the camera apps owned and maintained wholly by Apple such as iCam, and once the media is saved to disk, dynamically load it into an HTML video. There are other APIs you can use, however, I've personally had the best luck with HTML.

I like WebRTC, but it's resolution is limited to 640X480 as many apps are. Best of luck, as I've exhausted every idea I could come up with to capture dynamically on IOS. Only a saved and reloaded video will display on IOS.

Sorry.

DB

 

[Asagi]

 

I correctly understand that it is impossible to babylonjs? This example: https://jeromeetienne.github.io/AR.js/three.js/examples/mobile-performance.html works fine with the iOS camera, but only in the safari.

 

I would like to develop it in babylonjs.

[RaananW]

If it is technically possible (even using a different framework), it should be possible with babylonjs.

I won't express my opinion about the iOS browser, I will say that it is rather restrictive and that you need to understand what's wrong with the code in order to debug this.

Eventually, those are web APIs that we are calling, and they should be standard. Could it be that the example is using some sort of a fallback on iOS?

[RaananW]

I dont have an iphone (and so should you), but maybe someone with iPhone and a bit of testing time can check this code - https://github.com/jeromeetienne/AR.js/blob/master/three.js/src/threex/threex-artoolkitsource.js#L182 and see which part there makes iOS tick.

[Asagi]

 

I have to use a safari. For tests, I took chrome, mozilla, safari. Chrome and Mozilla do not give errors, but they do not work either. In BabylonJS safari requests access to the camera and that's it. As for access to the camera in the threeJS then everything there is working in a safari. Chrome and Mozilla also do not work with demos on threeJS where you need access to the camera. In babylonjs Chrome and Mozilla do not even request access to the camera. The problem is not only on one of my devices. I tested on different iPhones. 

UPD:

Now tested on Android and everything works. I think my question is closed, I'll have to use another device.

Thanks to all!

[dbawel]

If you hack the Iphone and can list a browser in the privacy>security setting, then you can provide permissions. Otherwise, for three.js demo, they used AR.js to capture each frame individually. Been there, and tried this, but too daunting for me.

DB

I tested in Safari on both Iphone and MAC IOS and nothing. However it did ask for permission. 5 different IOS devices and no result for me.

[Sebavan]

@Asagi  I am currently making a fix in Babylon which should solve your issue on IOS. ETA is end of day for the fix

 

[Sebavan]

@Asagi I have just published the fix : https://www.babylonjs-playground.com/debug.html#CHQ4T#108

A couple of issues where found in the webcam managenement.

From your sample,  on IOS maxWidth and maxHeight are not supported on IOS you need to use width and height instead with allowed resolution.

Hope that helps.

[Asagi]

That's great. Thanks! 

Can i change webcam source? For ex: front camera, back camera

[Sebavan]

you can try in the options: facingMode: "environment" as it should be a supported constraint : "https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/facingMode"

[Asagi]

Sebavan, can you give example in BabylonJS context? Not working for me facingMode

[Sebavan]

https://www.babylonjs-playground.com/debug.html#CHQ4T#110

[Asagi]

Sebavan, this is  awesome! Thanks!

[dbawel]

To all of you on this post:

I've spent weeks trying to crack this using WebRTC and the MediaStream API and have come up null. I'm in awe that you have been able to crack this. My gratitude goes well beyond respect in this case. Congrats for us all.

DB