DASH video into Pixi.js

[danhett]

Hey folks.

 

So, this is probably insanely specific, but I'd love to find out if this is possible.

 

I'm building a web application that edits multiple streams of video, using Pixi.js as my route into WebGL, as I'll be doing lots of image manipulation and I want to use shaders and all that jazz (plus the workflow is lovely).

 

I've been using raw MP4's, streaming tons of them and grabbing imagery and everything - all great.

 

However: what I actually want to do is use DASH video (https://github.com/Dash-Industry-Forum/dash.js/wiki), as this will be used for editing a live stream in sections. DASH is essentially a wrapper for MP4 segments, using HTTP as delivery and adapting it's bitrate/quality/whatever to suit conditions.

 

Obviously, DASH doesn't seamlessley pull into a Pixi video texture as easily as a raw MP4 does. It uses a standard <video> element, but the src is actually just a manifest file, which is then used to serve small chunks of video.

 

My question then, is: is there any way I can easily pipe video footage from a <video> tag directly into a video texture in Pixi?

 

It's worth noting that this doesn't have to be pretty or performant, it just has to work - it's being used on the latest bleeding-edge Chrome and nothing else, on a beefy machine.

 

Can this be done?

 

Any and all assistance appreciated.

 

Dan

[xerver]

> is there any way I can easily pipe video footage from a <video> tag directly into a video texture in Pixi?

Have you tried using a VideoBaseTexture?

var videoTexture = Texture.fromVideoUrl('..url to video...');// Or if you have a video element already:var videoTexture = Texture.fromVideo(videoElement);// then use it in a spritevar sprite = new Sprite(videoTexture); 

[danhett]

Hi

 

Yeah, it doesn't work - with a normal MP4 is does, but Dash is a weird one. In the <video> element you link to a manifest file instead of a video, and Dash then pulls down tiny chunks of MP4 - this is how it stays adaptive, so it can drop the quality without skipping for example. The reason we're using it is that the tool is designed to scrub seamelessley back in time through live video, which Dash does perfectly.

 

I can't query the MP4 itself as it's always changing, all I want to do is grab the current footage and have it accessible as a texture in Pixi - even if it's a hacky route, or terrible performance-wise, it doesn't really matter as long as it works!

 

My other pondering was to be REALLY hacky and just continually modify the size and position of the <video> element itself with CSS, but taking this route means I can't do some of the more clever stuff I wanted to do with Pixi.

 

Any and all ideas appreciated!

 

Dan

[xerver]

Hmm, I don't know anything about dash. Sorry.

[danhett]

Neither do I - well, not until I needed to get this working! It's a neat format, I've just no idea how to pipe it into Pixi.

 

Cheers anyway.

 

Anyone?

[xerver]

Neither do I - well, not until I needed to get this working! It's a neat format, I've just no idea how to pipe it into Pixi.

 

Cheers anyway.

 

Anyone?

 

Using the video element directly (and not the url) doesn't work either? If that is the case the dash video can't be drawn to webgl context or canvas at all, which seems wrong...

[danhett]

Unfortunately not - the Dash embed looks like this:

 

<source id="dashvid" src="http://path.to.my.thing/manifest.mpd" type="application/dash+xml"/>

 

The dash.js lib then uses that manifest to swap out chunks of MP4. Unfortunately I can't query for these directly as they're obfuscated behind a TON of crazy logic that's constantly swapping chunks out so it's all seamless.

 

What I'm hoping to do is basically not both trying that, and instead figure out some way of grabbing the imagery from the running video - like, embedding the video into the page and then somehow doing a capture of it. No idea if it's possible though. Tearing my hair out!

[danhett]

Or, can I target the whole <video> element? I was using the path... I wonder if that would work?

 

I can see a Texture.fromCanvas call, is this how you do it? I guess I could draw the video into a canvas (like this: http://html5doctor.com/video-canvas-magic/) and then pull that into a WebGL video texture...

 

Or is there a smarter way to do this?

[xerver]

I meant the <video> element:

<video id="my-video">  <source src="manifest.mpd">  <source src="movie.webm">  Your browser does not support the video tag.</video> 

var videoTexture = Texture.fromVideo(document.getElementById('my-video'));var sprite = new Sprite(videoTexture);

.fromVideo() takes an actual video element, .fromVideoUrl takes a url to load the video from. Try just passing in the actual video element. Source swapping shouldn't matter then..

[danhett]

Ha, holy shit, that totally worked! 

 

I had no idea I could do that. You have saved my bacon, sir.

 

Let me double check everything and clean the code up, and I'll post it just on the off-chance anyone runs into this in the future (not likely, but hey).

 

Thanks again!

[xerver]

I said in my first post you can use a URL or the element directly

http://www.html5gamedevs.com/topic/15382-dash-video-into-pixijs/?p=87174

[danhett]

Yeah, I didn't realise you meant the actual HTML element - I was screwing around with the video directly and getting nowhere. All good.