Augmented Reallity

[MrVR]

Hi Guys Im looking for way to integrate augmented reality into the toolkit if any body has ideas about this please let me now.

so far I'm trying with trackingjs, I figure if we can track faces, color etc we can bring the video in vr mode camera and stick stuff in the 3d world using trackingjs syst.

any way I know there other libraries but I though this is a nice and clean way to do some augmented reality and it has nothing to do withe Babylon framework yet .

@MackeyK24 here my initial code attempt , I put the #video div  on the index.hmtl but I dont get the camera working  yet je je 

I need to render the camera's device into the canvas @Deltakosh is ther a way to do this already?

see the ~ref link

  public ready(): void {
            // this.scene execute when ready
            var scene = this.scene;
            this.scene.activeCamera.attachControl(this.engine.getRenderingCanvas());
            this.manager.enableUserInput();


            var tracker: any = new tracking.ObjectTracker(['face']);
            tracker.setInitialScale(4);
            tracker.setStepSize(2);
            tracker.setEdgesDensity(0.1);
            tracking.track('#video', tracker);
            tracker.on('track', function (event) {
                var canvas = this.engine.getRenderingCanvas();
                var context = canvas.getContext('2d');
                tracker.on('track', function (event) {
                    context.clearRect(0, 0, canvas.width, canvas.height);
                    event.data.forEach(function (rect) {
                        context.strokeStyle = '#a64ceb';
                        context.strokeRect(rect.x, rect.y, rect.width, rect.height);
                        context.font = '11px Helvetica';
                        context.fillStyle = "#fff";
                        context.fillText('x: ' + rect.x + 'px', rect.x + rect.width + 5, rect.y + 11);
                        context.fillText('y: ' + rect.y + 'px', rect.x + rect.width + 5, rect.y + 22);
                    });
                });
            });
        };}

 

[MackeyK24]

@MrVR You probably DONT want to render into the playing canvas but rather create a new canvas element and render into that... Check out the Virtual Joystick Implementation... @davrous does something like this to render the two virtual joystick controls ONTOP on the engine rendering canvas...

Also if you got the video element already no need to render anything into another element ... Just put #video element on top of canvas USING ABSOLUTE POSITIONING

take a look at some of the "Scence Controller" Html markup where i put things like "Score" on the game screen... is using a little SNIPPET of css above the actual html markup to show and position the element on the screen...

Note: The main page container is using ABSOLUTE positioning so YOU MUST also

[dbawel]

@MrVR -

Here is a PG with the camera streaming as a video texture:

https://www.babylonjs-playground.com/#1R77YT#4

DB

[MrVR]

6 hours ago, dbawel said:

@MrVR -

Here is a PG with the camera streaming as a video texture:

https://www.babylonjs-playground.com/#1R77YT#4

DB

Hey thanks for the playground !!