JackFalcon Posted October 3, 2017 Author Share Posted October 3, 2017 Success... Ionic App -> STATUS: BabylonJS in Ionic App (small)dynamic sizing then begin AR APP test... TEST: will Ionic App Context improve AR Camera interoperability??? CREDITS: this link rocks: https://doc.babylonjs.com/tutorials/babylonjs_and_ionic_angular jokes removed.... :] Quote Link to comment Share on other sites More sharing options...
GameMonetize Posted October 6, 2017 Share Posted October 6, 2017 I think you should write your doc directly in our doc.babylonjs.com This could be useful for a lot of people JackFalcon and Dad72 2 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 10, 2017 Author Share Posted October 10, 2017 @Deltakosh, I will. Docs, ok. Look at that nice embedded tweet. Lol. I fav'd that link back then, [ and must muster artistic resolve to doc again : ].... so I will. UPDATE: -getting closer on multiple fronts r&d time ramping on: Babylon app about to go AR. (!~!) Want to see what happens!!! ... : ) I'll eventually re-doc this thread with results of the experiment: your awesome skybox is fully functional in Ionic App. So Nice. +1. a few wrench turns and an engine drop in... should give AR, but -> it is a different (physical) camera access instance. Right? So upcoming AR TEST, today... UPDATE: A bunch of different BABYLON Scenes on different Ionic pages: First, need a landscape... Hide skybox and set clear background color: // Set clearColor background to transparent. scene.clearColor = new BABYLON.Color4(0, 0, 0, 0); Next AR engine drop in... export class Ar1Page { BABYLON: any; ARjs: object; constructor(private camera: Camera) { this.BABYLON = BABYLON; this.ARjs = ARjs; } And init ARjs (after createScene() before engine.runRenderLoop): ARjs.Babylon.init(engine, scene, scene.cameras[0], document.getElementById('ar1Canvas'), engine, BABYLON) confirm (physical) camera not available. : ) Needs a native camera instance...? Or HTML5 permission? Or - what is needed for HTML5 getUserMedia()? (maybe only simulator... TEST) Dad72 1 Quote Link to comment Share on other sites More sharing options...
GameMonetize Posted October 10, 2017 Share Posted October 10, 2017 I think it could be a security constraint? Perhaps ionic needs to declare that you want to access the webcam? JackFalcon 1 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 10, 2017 Author Share Posted October 10, 2017 Indeed. : ) Following native (physical) camera path... returnRealCam ( ) or declared... (looking). thx... READING: https://forum.ionicframework.com/t/ionic-how-to-turn-on-camera/29703/4 - ionic plugin add cordova-plugin-camera - "The problem with desktop is that you can't use any Cordova plugins (like the Camera plugin)." - "typically a plugin is used to access the native Camera API, it is also possible to access the camera through plain HTML5:" - " testing app in the Android simulator not the browser, you can access the camera and use it in your local testing." DOCLINK: https://ionicframework.com/docs/native/ - Ionic Native: https://github.com/ionic-team/ionic-native import { Camera } from '@ionic-native/camera'; ... something like all that... Yes, HTML5 not simulators? Looking... getUserMedia()...? ~dives in ,<feet-first>... https://www.html5rocks.com/en/tutorials/getusermedia/intro/ (actual best link...) <in-progress>... One amazing use case for video capture is to render live input as a WebGL It is called Video Input to WebGL Texture... "this texture is special, it is a video. So it need to be constantly updated." - Jerome... : ) STATUS: (HTML AR into simulator-jungle): Stepped down to getUserMedia( ) catch (e) {e === 'PermissionsDeniedError'} . Ding! Few reasons for this... Tried adding permissions to an .xml -> (we still use xmllol). no change. Maybe different permission... found docs. (derp, restart ionic serve?) STATUS: trying simulator with both HTML5 and Native access TEST. Install Ionic Native Install Ionic Native core & Camera, and accessed camera object from Angular Provider... Great - there are 4 ways to simulate, likes USB the best... DONE. STATUS: Simulation, or USB debugging -> must I download the moon? Yes. <downloads... the moon> : ) Simulator worked (eventually). EPIC-AR-VIDEO-TEST-RESULT-FAIL -> it took a picture. : ) There must be a video stream around there somewhere... ... if getUserMedia can't cut it. In progress... Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 13, 2017 Author Share Posted October 13, 2017 Found it in the docs: https://www.npmjs.com/package/cordova-plugin-html5video - "Cordova Html5Video Plugin solves ... HTML5 tags for local video ... which is not functional in Cordova for Android." Another link, digging into HTML5Video Cordova https://cordova.apache.org/docs/en/2.7.0/cordova/media/media.html - permissions in xml file ... HTML5 Rocks: https://www.html5rocks.com/en/tutorials/getusermedia/intro/ camera stream piped into video src ... <cheers> navigator.webkitGetUserMedia('video', function(stream){ video.src = webkitURL.createObjectURL(stream); }, function(error){ console.log("Failed to get a stream due to", error); }); TESTS: - permissions in xml manifest - cordova Html5Video Plugin - native camera format "SECURE_VIDEO" -> assign to video.src. ... UPDATE: As usual DeltaK has the answer in under 10 words.... took me 200 lines of technobabble to get there! I found more great links... Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 19, 2017 Author Share Posted October 19, 2017 Success -> HTML5 Camera Access in App... The magic works... : ) ANSWER: (AndroidManifest.xml): <uses-permission android:name="android.permission.CAMERA" /> I think... Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 19, 2017 Author Share Posted October 19, 2017 Hey @raghu good question, ANSWER: step-debug the getUserMedia to see what it returns. Check out enumerateDevices() See solution above... (rebuild). : ) Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 20, 2017 Author Share Posted October 20, 2017 SOLUTION: Babylon & AR.js in Ionic App ~ with HTML5 WebCam... That was intense... ! Dad72, brianzinn and raghu 3 Quote Link to comment Share on other sites More sharing options...
brianzinn Posted October 20, 2017 Share Posted October 20, 2017 2 hours ago, aFalcon said: That was intense... ! The waiting is intense, too!! Can't wait to see some online sample or code! Thanks for sharing your experience. Dad72 and JackFalcon 2 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 20, 2017 Author Share Posted October 20, 2017 @brianzinn yep, I will doc latter side of journey. There are odds-and-ends issues...AR peculiarities arise! The white ion-card displayed between <canvas> and <video> at every z-index (because ionic CSS something) - blocking video with white <always>. Finally the fix: <ion-card style="background-color:rgba(0,0,0,0)"> , simple super-power++... There remains small-problems in the matrix-crossover. It needs a new-pattern, ... and someone will need to draw a diagram(ok). Stuff... Thinking, it is a three-part entity: visual-engine(s), ar-engine, app-engine. So .... stuff. The way it is currently organized is not the best way, so what is the best way? - could go angular "provider", but that is so baked in, so I'm thinking oldjedi generic namespace module, in ES6 Export/Import? Idk. But yeah, maybe next weekend! Anyway... The coolest things to encounter are: - enumerateDevices: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices - manifest permissions (thanks DK) :https://developer.mozilla.org/en-US/docs/Archive/B2G_OS/Firefox_OS_apps/App_permissions#video-capture - getUserMedia: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia Something else... https://blairmacintyre.me/2017/05/20/its-not-webar-yet/ this had some great information, counterpoint... ... and finally the answer, with an inside look into the wild-west saloon (3/4ths down). brianzinn 1 Quote Link to comment Share on other sites More sharing options...
raghu Posted October 21, 2017 Share Posted October 21, 2017 On 20/10/2017 at 2:50 AM, aFalcon said: Hey @raghu good question, ANSWER: step-debug the getUserMedia to see what it returns. Check out enumerateDevices() See solution above... (restart server). : ) Finally.. VID-20171021-WA0000.mp4 JackFalcon 1 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 22, 2017 Author Share Posted October 22, 2017 Sorry cannot click that link ... screenshot? Quote Link to comment Share on other sites More sharing options...
raghu Posted October 23, 2017 Share Posted October 23, 2017 11 hours ago, aFalcon said: Sorry cannot click that link ... screenshot? JackFalcon 1 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted October 23, 2017 Author Share Posted October 23, 2017 @raghu well done! There is code somewhere back in this thread to put a sphere at 0,0,0 with x,y,z as rbg dashed lines (if that helps). : ) Quote Link to comment Share on other sites More sharing options...
mkaouri Posted November 27, 2017 Share Posted November 27, 2017 Did someone tried jsartoolkit5 to implement it with babylonjs from scratch? Here is a sample code in github: https://github.com/mkaouri/jsartoolkit5/blob/master/examples/babylonjs_from_scratch.html Unfortunately, jsartoolkit5 supports only marker tracking objects such as patterns and bar code, and it doesn't support Natural Feature Tracking (NFT). There're some students made NFT tracker with pure javascript, here's an abstract: https://jensgrubert.wordpress.com/research/natural-feature-tracking-in-javascript/ Also, I'm attaching here their publication in pdf. I wish someone can implement it and share the code of NFT tracker in pure javascript. vr2012-poster_v1.4.pdf Arte and JackFalcon 2 Quote Link to comment Share on other sites More sharing options...
mkaouri Posted November 27, 2017 Share Posted November 27, 2017 I found a complete thesis of the NFT implementation in Javascript. Attached here with. Hopefully, it's useful to have an idea. 136880_oberhofer_christoph_2012.pdf Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted November 28, 2017 Author Share Posted November 28, 2017 @Mostafa Kaouri I like this very much... The implementation successes above were using jsartoolkit underneath AR.js, as I recall.... but looks you bring UPDATES... fantastic! That code looks really good. Screenshot of it working? A lighter footprint would be ideal on mobile... Quote Link to comment Share on other sites More sharing options...
mkaouri Posted November 28, 2017 Share Posted November 28, 2017 @aFalcon Great It's working with pure jsartoolkit5 and Babylonjs, without AR.js. Demonstration done on Google Chrome ver 62. For mobile devices, It should work on iOS Safari ver. 11 and Chrome for Android ver. 62 Way forward, I will make it with NFT detection (without markers). brianzinn, Arte, Sebavan and 1 other 4 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted November 28, 2017 Author Share Posted November 28, 2017 @Mostafa Kaouri Excellent! I will give it a run through over the holidays and postback results. Thanks for the update. Arte 1 Quote Link to comment Share on other sites More sharing options...
JackFalcon Posted November 30, 2017 Author Share Posted November 30, 2017 On 10/6/2017 at 4:53 PM, Deltakosh said: I think you should write your doc directly in our doc.babylonjs.com @Deltakosh - confirmed... many good things. New Years resolution. Quote Link to comment Share on other sites More sharing options...
mkaouri Posted December 3, 2017 Share Posted December 3, 2017 Very good news! I have developed the Natural Feature Tracking (NFT) tracker example with BabylonJs and JsArtoolkit5. It works like a charm. Find it here: https://github.com/mkaouri/jsartoolkit5/blob/master/examples/nft_babylonjs.html Clone the full project from here: https://github.com/mkaouri/jsartoolkit5.git Cheers... JackFalcon, brianzinn, Arte and 1 other 4 Quote Link to comment Share on other sites More sharing options...
Sebavan Posted December 3, 2017 Share Posted December 3, 2017 That is really cool, I guess we ll add a page in the doc as it is really cool !!! mkaouri 1 Quote Link to comment Share on other sites More sharing options...
Sebavan Posted December 3, 2017 Share Posted December 3, 2017 @Mostafa Kaouri could you push your changes to the main ArToolKit repo ? This would be better for highlighting the compat in the doc. mkaouri 1 Quote Link to comment Share on other sites More sharing options...
mkaouri Posted December 3, 2017 Share Posted December 3, 2017 @Sebavan Sure, my pleasure. For creating a mobile app on iOS and Android, I recommend to use phonegap. I tried it and it works with babylonjs on a web canvas overlaying native mobile camera. I will share this too after testing it with the integration of jsartoolkit and babylonjs. raghu, Arte and Sebavan 3 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.