Multiple problems using VideoTexture

[larsiii23]

Hi!

 

In my Babylon scene I have a room with a plane, representing a screen. My idea is to display a video on this screen when it's clicked and turn it of if it's clicked again.

 

Code:

var screen1 = scene.getMeshByName("Lerret");activateVideoTexture(screen1);function activateVideoTexture(mesh){    mesh.actionManager = new BABYLON.ActionManager(scene);    var action = new BABYLON.ExecuteCodeAction(BABYLON.ActionManager.OnPickTrigger, function(){        mesh.material.diffuseTexture=new BABYLON.VideoTexture("video", ["01_Tumor-Glioblastoma.mp4"], 640, scene, true);	mesh.material.emissiveColor = new BABYLON.Color3(1,1,1);    });    var action2 = new BABYLON.ExecuteCodeAction(BABYLON.ActionManager.OnPickTrigger, function(){	mesh.material.diffuseTexture.dispose();	mesh.material = new BABYLON.StandardMaterial("texture1", scene);    });    mesh.actionManager.registerAction(action).then(action2);}

I've got to major problems:

1: When the VideoTexture is activated/applied the screen only show the very top left corner of the video (i think). The screen is all brown, which is the color in the top left corner in the video. It seems like the plane just projects the color of the top-left pixel in the video

 

I also get this warning in the console "[.WebGLRenderingContext-05B3F9D8]RENDER WARNING: there is no texture bound to the unit 1"

 

So how do I make the video fit/stretch to the size of the plane it is applied to? I've set the size parameter to 640, which is the pixel width of the video. Don't know if it is this "size" is refering to.

 

2: When you click the second time I want the videoTexture to disappear, which actually works as it is now, except that the sound continues. How can I handle this?

[Wingnut]

Hi larsiii23!

I have been doing some experiments with this...

http://playground.babylonjs.com/#28ADCW

I don't have the correct video codec or some other problem... to view these likely-CORS-blocked urls.  But, I click on the plane, and I hear the sound play.

If you read the section about video textures, you can see that the size is set to 256 in the demo code... and in the demo playground.  So, 256 seems to be a good size to start-with, and your plane can be any size.  The video SHOULD map to full coverage if you have no scaling set.  But I am not an expert in these things, so I'm sure there are things to learn, yet.

Quote

Quote from that doc:  "The internal video DOM object is accessible via the VideoTexture.video property... which allows you to control some characteristics and monitor the status of the video (things such as play, pause, stop, loop, autoplay). See the link above for the full story."

Well, I was the one who wrote that line, and I got it wrong.  There is no stop() method on the DOM video object... just pause() and some other impertinent funcs and props.  Calling pause() will stop (pause) the sound.  Yay!  (bad way of doing it, though, eh?)

I actually tried to 'delete' the video object (an old near-dead JS command, I think)... and that failed to stop the sound.  You will see the pause() done to the DOM .video object... in line 31 of my playground demo.

I get an error as well, in my FF 30...

Error: WebGL: The canvas used as source for texImage2D here is tainted (write-only). It is forbidden to load a WebGL texture from a tainted canvas. A Canvas becomes tainted for example when a cross-domain image is drawn on it. See https://developer.mozilla.org/en/WebGL/Cross-Domain_Textures

and...

SecurityError: The operation is insecure.

We should probably put an assortment of test videos in h t t p://babylonjs-playground.azurewebsites.net/textures/videos/... so we can more-easily play with various videos... without cross-domain (CORS) security hassles.

Anyway, maybe some of this can help, and maybe not.  Others will surely have some ideas... I hope.  Maybe grab a zip of that playground, take it home, and use it on your video (01_Tumor-Glioblastoma.mp4) and see if your video... maps onto the plane better.  As long as you have no u/v/w scaling, wrap, offset, or ang on your video texture... it SHOULD map correctly.

If you still have problems with the "stretch", try to cause the same problem in the playground (please).  Modify my playground and hit RUN...  saving occasionally, and if you discover something, SAVE once more and send us the url... so we can see it.  We might need to put YOUR 01_Tumor-Glioblastoma.mp4 video into the playground /textures folder... so we can better troubleshoot.  We'll talk about that later, as needed.

Good luck!  Keep us posted.  Sorry about screwing-up the docs... regarding a stop() function on the DOM video object.  Obviously, I wrote that line without checking the facts, and that's a dumb thing for me to do.  :/

[larsiii23]

Thanks for your input, Wingnut. You definitely solved my second problem. I see this thread is raising much of the same issue as I am, without getting any clear solution. I will keep you posted on any progress.

[larsiii23]

If I create the plane in the javascript eveything works fine. The problem I described appears when I try to apply the video texture on a plane imported from a blender scene.

[Wingnut]

aha!  Good troubleshooting, larsiii23!  Do you think it has anything to do with the amount of subdivs/tessellations in the Blender-made plane?   Did you put a basic material on the exported plane... within Blender?  But it should work with or without a Blender material applied before export.

 

*scratch scratch*

 

Obviously, the Blender plane UV's (texture mapping) are somehow different than a BJS plane.  hmm.  I hope it didn't cost you lots of hours of work.

 

I'm not a professional Blender user, but many other forum readers are, and they will be interested in this discovery.  If you learn more about this, please share (thx).  If we can figure out why this is happening, we can add a note about it... in the video texture documentation.

 

Actually, we can add a note even if we can't figure out why. 

 

Thanks again for your investigation and info!  Well done.

[larsiii23]

First I tried with a simple plane (it was rotated 90 deg around x, and scaled to make make it widescreen), 1 face, 4 vertices. I applied a material in blender.

 

Tried again without applying a material, and without scaling. But I got the same result. The audio playes, but the plane is all in one color, which I expect is the color of the topleft pixel of the video of any time. The color of the plane change according to which color that can be seen in the topleft corner.

[Wingnut]

Interesting!

 

Here's a piece of code that you can run on both planes... to help see the differences:

console.log(  "\n uv: " + videoPlane.getVerticesData("uv") +  "\n pos: " + videoPlane.getVerticesData("position") +  "\n ind: " + videoPlane.getIndices());

For step 3, videoPlane.getVerticesData("indices") returned NULL for me, so I used the getIndices() function.  I'm not sure if that is important, but I suppose it COULD be.

 

For BJS plane... this returns:

 

uv: 0,0,1,0,1,1,0,1
pos: -0.5,-0.5,0,0.5,-0.5,0,0.5,0.5,0,-0.5,0.5,0
ind: 0,1,2,0,2,3

 

Know about that stuff?  uv are 2 values per point, pos are 3 values per point, ind are 3 values per face (there are 2 triangles in the plane.  If the indices are drawn in CLOCKWISE order, the triangle faces are frontfaced/backfaced in a certain direction.  If drawn COUNTERCLOCKWISE, backface becomes frontface and vice versa.). 

 

We could also check the "normal", but that is a lighting thing, and I don't think it applies to this issue.  There's also uv2, color (per vertex), and a couple of others... but probably not important here.

 

I don't know if anything can be learned from doing this to both types of plane, but, I thought I would tell you about it. 

[larsiii23]

Interesting. These are the values I get:

For the plane created in javascript (that works):

 uv: 0,0,1,0,1,1,0,1

 pos: -2.5,-2.5,0,2.5,-2.5,0,2.5,2.5,0,-2.5,2.5,0

 ind: 0,1,2,0,2,3

 

For the plane imported from the blender scene (that doesn't work):

 uv: null

 pos: 1,0,-1,1,0,1,-1,0,1,-1,0,-1

 ind: 0,1,2,3,0,2

 

 

So it looks like there is some problem with the uv mapping for the blender imported plane.

 

By then setting the uv coordinates manually in the javascript fixed the problem!

 

setVerticesData("uv", array_of_UV_coordinates, true)

 

Thank you so much for your help Wingnut!

 

Screenshot

 

[Wingnut]

Beautiful (screenshot)! 

 

Maybe Blender is not exporting the UV data because there is no .diffuseTexture on the plane, when exported.  *shrug*

 

I wonder if you put a basic simple texture on the plane before export, would there be some UV data on the Blender plane?

 

(Then replace that temporary texture with the videoTexture once its in the scene, of course).

 

If you could test that for me, that would great.  Once we test that, then I/we know what "note" to put into the videoTexture docs.  We can inform users about the 2 fixes... 1. put a temp diffuseTexture on Blender planes to make them have uv data, or 2. manually install uv's like our videoTexture troubleshooting hero... larsiii23... did. 

 

See the info about lighting and about setting an .emissiveColor to help light the video screen (in the docs)? 

 

That was put into the docs because myself and Temechon spent a few days troubleshooting a failed video-in-a-mirror test.  We had sound, but no video.  We thought about missing codec's, but I had the full KLite package installed, and I had properly played many kinds of video on my computer. 

 

It was caused by our failure to light the screen properly, in the scene.  We spent about 4 days in the swamp of befuddlement before deltakosh yelled "sounds like a lighting issue" from afar.  (He threw us a strong rope so we could pull ourselves to safety.)  hehe.  Fun! 

[larsiii23]

As I understand there is by default no UV mapping on planes in Blender, but it can be done manually in Blender. I have added textures and done the UV mapping on other planes in Blender, then exported to Babylon, and that works fine  . The problem was therefore not related to the VideoTexture in particular, but generaly adding texture to non UV mapped meshes imported from blender.

 

If any Blender experts find this statement incorrect, feel free to correct it.

 

So my conclusion is:

Adding texture in JavaScript to geometry imported from Blender does not work out.

 

But there are two ways of solving this problem:

Do the UV mapping in blender before export. 

Do the UV mapping of the imported mesh in JavaScript manually

[Wingnut]

Very good!  You are right.  This "note" would apply to all textures for imported-from-Blender mesh... mesh that have no textures at export-time.  (phew, what a sentence).

 

Yep, good conclusion.  Adding texture in JS - no workie.  CHANGING texture in JS - would work, I suspect.

 

Hunh.   I think a note about this needs to be put into our Basic Materials and our Advanced Textures tutorials.  Got any thoughts on that?  I sort of like the way you worded things in your conclusion.  If it's ok, I'll use some of those words in the tutorial notes.

 

Do you know how to write wiki markdown, larsiii23?  YOU can do the edits on those two tutorials, if you wish.  I'd put it near the bottom...

 

A Note About Imported Mesh: blah blah blah etc.

 

I can add it, if you don't want to.  No probs.  Thanks again for investigating this "stump in the road".  Good job.

 

We should probably find out the story for 3dsMax planes, too.  Hey forum users... does anyone know if 3dsMax exporting does the same thing as Blender... regarding this topic?  No mesh texture at 3dsMax export time = no mesh uvs after BJS import? (thanks!)

[JCPalmer]

As I understand there is by default no UV mapping on planes in Blender, but it can be done manually in Blender. I have added textures and done the UV mapping on other planes in Blender, then exported to Babylon, and that works fine  . The problem was therefore not related to the VideoTexture in particular, but generaly adding texture to non UV mapped meshes imported from blender.

 

If any Blender experts find this statement incorrect, feel free to correct it.

 

So my conclusion is:

Adding texture in JavaScript to geometry imported from Blender does not work out.

 

But there are two ways of solving this problem:

Do the UV mapping in blender before export. 

Do the UV mapping of the imported mesh in JavaScript manually

 

Yes, this is how I found it to work, however describing it as a problem is a slight mis-characterisation.  The pre-built BJS meshes add UV, so you do not have to.  These are small meshes.

 

Blender can build very large meshes, and if you wish to not use cover it with a texture material, it is just a waste of .babylon file space, down load time, CPU memory, & GPU memory.

 

Theoretically, could export script add UV to the mesh if mesh using a texture & no UV, but would cause "Do you want to save?" message when you left.  Also, not a priority for me, but you could PR the change, if you want.

[larsiii23]

Yes, this is how I found it to work, however describing it as a problem is a slight mis-characterisation.  The pre-built BJS meshes add UV, so you do not have to.  These are small meshes.

 

Yeah, I agree that "problem" is not the best word to describe it, but I couldn't come up with a better term at that moment 

[larsiii23]

Hunh.   I think a note about this needs to be put into our Basic Materials and our Advanced Textures tutorials.  Got any thoughts on that?  I sort of like the way you worded things in your conclusion.  If it's ok, I'll use some of those words in the tutorial notes.

 

Do you know how to write wiki markdown, larsiii23?  YOU can do the edits on those two tutorials, if you wish.  I'd put it near the bottom...

 

A Note About Imported Mesh: blah blah blah etc.

 

I can add it, if you don't want to.  No probs.  Thanks again for investigating this "stump in the road".  Good job.

 

We should probably find out the story for 3dsMax planes, too.  Hey forum users... does anyone know if 3dsMax exporting does the same thing as Blender... regarding this topic?  No mesh texture at 3dsMax export time = no mesh uvs after BJS import? (thanks!)

 

Good initiative. Am a little busy these days, and not a hell of a writer neither to be honest. If you don't bother adding it to the wiki that would be very nice. I can take a look and see if I got anything to supply. Not familiar with wiki markdown, but I'll guess it's not to hard to learn

[V!nc3r]

I up this topic because i have a strange issue. I use 3dsmax here, and that's the result i got on babylon :

My verticesData says :

 uv: 0.0001,1.0001,0.9999,1.0001,0.9999,1.9999,0.9999,1.9999,0.0001,1.9999,0.0001,1.0001
 pos: 0,-0.5939,-1,0,-0.5939,1,0,0.5939,1,0,0.5939,1,0,0.5939,-1,0,-0.5939,-1
 ind: 0,1,2,3,4,5

To do a check, i import some meshes of this scene into Blender, then reexport to babylon, and the video mapping works !

verticesData says :

 uv: 0.9999,0.0001,0.0001,0.9999,0.0001,0.0001,0.9999,0.9999
 pos: 0,-0.5939,1,0,0.5939,-1,0,-0.5939,-1,0,0.5939,1
 ind: 0,1,2,0,3,1

I don't understand what could be wrong on 3dsmax setup, it is just a simple plane. Can you enlighten me ?

Edit: and of course if i set a default material with just a diffuse map on this plane, uv are correct.

[GameMonetize]

Can you try to replace the diffusemap of your std material with your video then?

[V!nc3r]

I think this is already what i do, this is the babylon script i use :

scene.getMeshByName("tv_001").material.diffuseTexture = new BABYLON.VideoTexture("video", ["video.mp4"], newScene, false);

 

[Flomotion]

I have the same issue. For now I solved it by creating a plane with BJS with the video texture on it.

But it should be working on exported mesh as well. So hopefully an answer will be found :-)

[GameMonetize]

I don't see why 3dsmax change uv coordinates when exporting video textures Oo

[dbawel]

@larsiii23 - The conclusions you have determined are correct. In addition, along with the info that @JCPalmer provided about Blender should provide you with almost a complete picture of how to use video textures in BJS. However, as most all display screens these days are self illuminated, I generaly attach my video texture to the emissive channel in my material:

    videoDynamicMaterial.emissiveTexture = "your videoTexture";
    videoDynamicMaterial.useEmissiveAsIllumination = true;

Also, If you want the best performance as well as simple property setting to display the video in the correct aspect ratio, then set your texture size to be a power of 2 such as 512 X 512, or 1024 X 1024. Then use the uScale and vScale values for the texture to scale either the U and/or V property of the video to represent the exact and accurate aspect ratio of the original video. Or depending on what result you're looking for stylistically, it is also very simple to scale the X and/or Y values of the "plane" mesh to accurately represent the aspect ratio of your original video.

@V!nc3r - I have also had issues with the default UV coordinates which are natively assigned in Max - so I often avoid exporting primitive objects from Max, and simply create these in babylon.js. - as BJS primitives will always consistantly map simple textures such as video textures consistantly and correctly. Otherwise, I unwrap UVs in Blender and export these objects, even though I spend thousands of dollars each year for all Autodesk and other "professional"  applications.

And for everyone on the forum, I've spent COUNTLESS hours working with and testing various video formats - both compressed and not compressed - so if you have any questions reguarding video codecs, I might be able to help. Also there are various issues with browsers, OS systems, and playback and other controls depending on all of these factors including broser type and versions. Some you can work around and some you cannot. So if you have any further issues with these, please ask, as I don't wish for anyone else to tread through the muck of HTML video which has been my own personal hell.

Cheers,

DB

[V!nc3r]

@Deltakosh, in this case 3dsmax don't know that a video will be assign later on the material. I think the bugs comes when BJS get uv coords : as you can see abovementioned, 3dsmax write more uv coord than Blender for the same mesh... Why ? Who know's

@dbawel, you think that video texture have to be power of 2 ? May codecs doesn't like these particular sizes ?

[dbawel]

@V!nc3r -

There is no requirement for a video texture to be a power of two, and I often use non power of 2 video textures as I have built an app which has to allow any user to drag and drop their video into the scene at it's native resolution. However, if I am building an environment which uses a video texture only I need to manage, then using power of 2 resolutions will avoid many potential errors as well as improve performance. I also find them easier to manage since the math is simple to use U and V scale values or to scale a BJS primitive plane to display the video texture's original aspect ratio.

Also, if you aren't already using Handbrake, it is a reliable free video and audio converter which I personally prefer for compatible conversions.

But no, it is not required to use power of 2 video textures - only recommended.

Cheers,

DB

[GameMonetize]

@V!nc3r: I don't think this is a bug in the exporter as this works if you use a regular texture. Almost all scene from www.babylonjs.com comes from 3dsmax. In your scene seems like the texture of the bed's sheet seems to work right?

[V!nc3r]

Yes, aside from video mapping, all uv works great.

[GameMonetize]

So as this is the same code on the exporter side of the house, I'm guessing 3dsmax is doing something different on your specific plane