Project Video Onto Solid Particle System

[wartgun]

Hello! New Babylon user here--also new to 3D graphics and visual programming in general. I'm making a 3D visualization in Babylon and am a little stuck. Searched this forum for relevant topics but wasn't really even sure what to look for, tbh.

WHAT I HAVE SO FAR

A very simple visualization that uses the Solid Particle System. I add 1,000 shapes to it (flat triangles) and arrange them into 10 flat 10x10 (x, z) grids, all at the same initial y coordinate, and all initially invisible.

Every few seconds, I take an invisible 10x10 grid make all 100 of its particles visible.

My `SPS.updateParticle` just tests if `isVisible` is true, and if so, rotates it a little bit, and moves it down a little bit (decreases `position.y`).

It also tests if the particle's y position is below a certain threshold--if it is, it calls `recycleParticle` on it, which simply makes it invisible again and sets its y position back to original starting height.

It thus looks like an infinite series of 10x10 grids of flat triangles, appearing in thin air at a particular location, falling downwards, and then disappearing. At any given point there are maybe 6 or 7 grids visible simultaneously, all falling within the imaginary 3D rectangular prism defined by their (x, z) borders and their downward y motion.

 

WHAT I HAVEN'T BEEN ABLE TO FIGURE OUT:

I want to project a video onto the visible particles, like a moving, shifting projector screen. The projector could be at a fixed position, or it could be behind the camera, whatever's easiest at this point.

My instinct was to make each particle translucent, and just place the video on the opposite side of the particles from the camera such that it plays "through" the translucent particles. But I can't figure out a way to have the video ONLY be visible through the particles, and not just look like a TV screen with a bunch of triangles in front of it.

 

Let me know if any of that is unclear. Trying to get a playground up at some point but it's simple enough I thought I'd just ask. Thanks in advance for all your help, and for building such an awesome tool! I've had a lot of fun learning how to use it!

 

[jerome]

Well, you could also use a video texture and choose to set it only on the wanted visible particles.

[wartgun]

@jerome thanks for the suggestion! I used this as a template and tried:

 

let mesh = SPS.buildMesh()

// ...later, setting up Webcam

let myVideo
let isAssigned = false
let videoMaterial = new B.StandardMaterial("texture1", scene)
videoMaterial.emissiveColor = new B.Color3(1,1,1)

B.VideoTexture.CreateFromWebCam(scene, function (videoTexture) {
  myVideo = videoTexture
  videoMaterial.diffuseTexture = myVideo
}, { maxWidth: 256, maxHeight: 256 });

scene.onBeforeRenderObservable.add(function () {
  if (myVideo !== undefined && isAssigned == false) {
    if (myVideo.video.readyState == 4) {
      mesh.material = videoMaterial;  // Update SPS mesh material
      isAssigned = true;
    }
  }
});

 

But it displays my entire webcam image (so, my face) on EVERY particle. Instead, I'd like to project the webcam image onto a group of particles, more akin to this, if you imagine each ASCII char in that demo to be a separate SPS particle in mine. Is that possible?

[jerome]

set the particle uvs property to (0,0,0,0) on the ones you don't want the image to be projected

and to the value you need on the others ... http://doc.babylonjs.com/overviews/solid_particle_system#uvs

[wartgun]

@jerome thanks for pointing me in that direction! I'm pretty close, but may have misunderstood part of the documentation.

Made a playground demo here.

My understanding from this is the (uvs.x, uvs.y) represents the lower-left corner of the crop, and that (uvs.z, uvs.w) represents the top-right corner. So, to include the bottom-left quadrant of the webcam feed, you'd do:

particle.uvs.x = 0;
particle.uvs.y = 0;
particle.uvs.z = 0.5;
particle.uvs.w = 0.5;

...right?

But on my machine (and in the playground) this appears to include the upper-right quadrant of the webcam feed (see lines 69-79).

 

Am I just misunderstanding something basic about UV?

[jerome]

and what do you intend to do ?

[EDIT] about the uvs, they behave just like the standard textures behave regarding the uvs.

when your plane is not rotated, the texture is applied like this : https://www.babylonjs-playground.com/#2FPT1A#132

 

maybe should you use some invertY property on the texture instead of rotating the plane