VideoTexture Format Performance

[dbawel]

Hello,

In using various file formats for a videoTexture, I have an .mp4 file at 3896 KB, an .ogv file at 4421 KB, and a .webm file at 4421 KB. On my PC, they all load and play within 1 second, but on my Android devices, the .mp4 file causes my mesh object (a plane) approximately 15 -30 seconds to display a black surface as if it could not find the texture, and when I press my play button (a simple GUI element which initiates the play() function, it takes another approximately 15 or more seconds to begin playing the .mp4 video file. However, when the source is an .ogv or a .webm file both at a smaller file size, they load in approximately 1 second and play in approximately 1 second after pressing my play button. I have tried practically every setting available to format the .mp4 video file to improve performance on Android devices, however, nothing has improved the performance.

Any thoughts on such poor performance of .mp4 files on Android would be appriciated - as well as to identify the Android OS and avoid loading .mp4 files on Android - but continue to load .mp4 on PC and MAC OS for improved quality for file size.

Also, I've tried every possible way to stop autoplay of a videoTexture on Windows, but nothing appears to work. I thought that Wingnut had posted a playground scene which paused autoplay on PC, although I can only find a couple that chnge playback speed and pause on a pointerUp scene event. However, what I'd truly like to be able to do is pause every device at frame 0 or frame 1 (in .ms of course) using some functio utilizing (htmlVideo.currentTime). 

As always, thank you for your help with any of these questions.

Cheers,

DB

[JCPalmer]

On .mp4, I quickly scanned you message for the word codec.  Not found.  I am only aware of one specific .mp4 codec, H264.  You might try searching like "android codec issues" or "codec for streaming".  Some may be geared for streaming, and perform very poorly for random fast forwarding.  Where as others might be great for positioning, but geared for Blueray / movie theaters where that is just a 10 milli seek penalty.

I have attempted a manual source code transform of JCodec's Java source code (H264) to Typescript, but not completed it yet.  It was pretty elaborate in size reduction.

[dbawel]

Hi JC,

I read your response a couple days ago, but didn't have time to respond until now. Thank you for your advice, and I believe I've now exhausted all possible trials of video formats, codecs, and compression settings. I had previously tried every possible codec for .mp4, and primarily using Handbrake (but not limited to only this applcation), and I truly tested every possible format, codec, and configuration - which I have a background in video streaming and various codecs, but as the time delays I'm seeing on Android are outragously long, I sought out to find an answer or possible improvement in .mp4 performance on Android and other mobile devices. 

As is recommended by everyone online, compressing your .mp4 file using H.264 baseline should produce the most efficient .mp4 for playback on Android and other devices. However, I spent hours setting all possible conditions and combinations for .mp4 encoding, and I found either no playback or the same performance as before - even though I generally knew from past experience what should work best. As mentioned, a baseline H.264 profile should by far provide the best results, however, since I've exhausted every possible method, quality setting, framerate parameters, audio codecs and settings, etc., I've come to the conclusion that the .mp4 video compression format using H.264 or H.265 is only usable if the file is loaded into local memory, and not either downloading or streaming from a URL. However, I am (and have been sucessful) able to convert my video files to .ogv, and they play within one second consistantly on every Android and other mobile device I've teseted - even when the file size is substantially larger. It required at least 20 seconds for a 3896 KB .mp4 video and audio file to load onto an android mobile device, and as autoplay is not "true" by default, once playback is initiate by the user, it requires another 15 or more seconds to begin playback. However, as for .ogv and .webm files (I prefer .ogv for compatability across multiple platforms - although I've also had great success with .webm files) these formats for the same video content as my test .mp4 file - and at a file size of 10 -13 MB per file, load within approximately a second, and begin playback within one second afer the user presses play, I should also inform that my source is also 1280 X 720p and the quality and performance is excellent on all devices.

So I'm not familiar enought with HTML video playback (I don't have years of experience, but only months) to know if the results I've found are consistant with anyone else's testing, but I believe that I will begin supporting mainly .ogv files as the performance and quality are excellent on all playforms and devices I've tested - even if it is not a compressed file format. So as everyone most likely knows, an .mp4 file at HD resolution will provide the smallest file size whie still maintaining the best quality. Whaereas .ogv and .webm file formats are typically not compressed, and thus require larger filesize - which I would assume cause poorer performance overall compared to the efficient .mp4 file format. But if you are formatting video media for multiple devices and specificaly mobile, then I would definately chose .ogv or another format such as .webm, as they will provide far better performance across multiple display devices - and I find the they fuction well on every device I checked - which was everything from PC to Mac to Android, and on Ipad and Iphone. However, if I am only developing media for PC or IOS (not mobile), then I would certainly use .mp4 for video and audio as it supports high resolution and quality at the lowest file size - with no performance problems that I could find at all when compared to most every other video and audio file format, compressed or not. and now you have the next widely used standard upon us which is H.265 which is a highly efficient codec and compression algorithm. 

So unles your target is mobile or anything other than desktop and laptop running Windows or IOS, I would avoid .mp4 for video file format at all costs. If it's only audio, I don't know at what size and/or conditions .mp4 audio files lose performance compared to other formats, but I'm sure someone will find this barrier at some point in the future.

 

And by the way, I had asked a second question in my original post, which I'm sure was overlooked - but if anyone can advise me how to set autoplay:false on a video texture and actually have the video texture pause once loaded, I would be very grateful. And I know some of you believe that you can conditionally pause video in a video texture inside a BJS script, but I and others have tried every known HTML video function, and have not been able to stop atuoplay from playing the video once it is loaded. I hope you read far enough down into this post, and that one of the great brains out there might have of find a solution that works. I require the same performance on all devices which is to pause the video as it is automatically set to autoplay:false on Android. And I have spent countless hours simply trying to stop the autoplay function and/or set this to false in every conceivable way I could think of. So please help if you can.

Cheers,

DB

[JCPalmer]

You might wish to send .mp4 for all platforms but Android.  The script from this thread did not go thru the website upgrade well, but gryff did have a link for where he got it. 

No clue on autoplay.  My efforts have been on H.264 encoding, not decoding.  My planned use is for promotions, not replay in BJS.

[MasterSplinter]

264 is what I recommend.  Have you seen the webvr Google released a few days ago.  It's built in three on the s7 it gets really good performance.  However, anything less than that level of phone the thing falls off.  Also, their gorilla video is 1080 mp4 264...  When I have some free cycles I'm going to take a deeper look at their source and see if there is anything else they have under the hood.

 

Btw I'm using either hls or mpeg dash.  I haven't done test to see yet if their is a difference.

Also, I know this is kinda janky -- but you can definitely pause the video if you add it to the DOM.

Here is the link-- it was kinda buried

https://github.com/google/vrview

https://developers.google.com/cardboard/vrview#supported_formats

 

[dbawel]

Hi @MasterSplinter

I'm able to pause video as a videoTexture in JavaScript, but I cannot set either autoplay = false; or videoTexture.video.pause(); without attaching the second function to a DOM event which must be triggered by the user. So if you know of a method to set autoplay = false; I've tried every method I can think of. Also, thanks for the links to the vrview code posted by Google - although of course they are using three.js, there's alot of good examples for functions in the scripts provided by them  in these directories on GitHub.

@JCPalmer - thanks for the code example to identify devices and os - I've found similar scripts online, but haven't yet gotten these to work on all devices. The code example you provided originally posted by @gryff is similar to what I've been testing, but identifies more and different devices - so I'll give some of the examples from this a few tests, and see if I get better results.

Thanks,

DB