Video Encoding for HTML5 Video and videoTextures

[dbawel]

Hello,

 

This post is not a question, but is information that I would have found invaluable prior to my having problems playing video on mobile devices - practically all mobile devices. I wanted to create a seperate post for everyone to read, as the post where this dialogue has taken place is labled "console errors."

 

I have been working on an app which relies heavily on playing video files and specifically video textures. I found that almost 2 months ago, I had no problem playing video files and the associated audio on desktops and laptops no problem; but I could not get my video files to playback on most all mobile devices - both Android and Apple phones and tablets. Although at the time, I wasn't focused on inding a solution, as I had other features to build into the application to finish it, and believed that solving video playback on mobile devices wouldn't be much of a problem.

 

So over the past 2 months I spent time each week to try and solve the fact that I couldn't get any video files to play back on my Android tablet, without finding a solution. But now that I have most all features built in the app, I needed to find a solution to be able to take the app into beta testing now as the app is designed to be a mobile app and wouldn't run on most tablets and phones - practically none.

 

As I had used most all available applications to encode and convert video files to .webm and .ogv files for playback on mobile, I began to believe that it was either a bug in babylon.js, or a conflict in my lengthy scripts. So I worked for days trying to find the cause of the problem, and certainly went to the BJS forum where many highly experienced users attempted to help me troubleshoot and solve the video playback problem. However, neither the balon.js community or I could get any video file or format to play on most mobile devices - with the exception of audio playback on some files.

 

So when I was completely lost a couple of days ago, I thought I would focus on analyzing the video files and formats themselves, to make certain that I could rule out video encoding and file conversion. After spending more than 2 days, I finally found that the problem had nothing to do with my scripting or with babylon.js, but that it was an encoding and file format issue. Now I have video and audio playback working on every device I and my friends are able to test, as well as every browser and OS version.

 

Again, I wanted to post this as a new thread, since many users will most likely overlook this discovery and information, since the post where this occured says nothing about video in the title and tags. And I hope that no one else will ever need to face the problems of having no solution or available resources to solve the problem of video palyback on mobile devices. The following is the solution as a quote directly from the post where I and several other users spent considerable time and dialogue over several days with no solution we could find:

 

Finally - I have the video and audio correctly playing on all tested mobile devices, browsers, and OS versions. The problem was a video encoding issue. Although I used every application I could including Premiere, Sony Vegas, and so many others, and attempting to encode videos using literally every online converter to .webm and .ogv, none of these would provide proper encoding for playback on all mobile devices - yet none of these formats which I encoded in various applications and online played back both video and audio correctly on my desktops and of course laptops. And I had no idea if it was a conflict in my scripts, an issue with multiple canvas' and dynamic materials and textures, or some unknown bug I'd had the bad fortune to run into.

 

So the solutions I found are to use either Sony Vegas, Premiere (which I was already using to encode video and audio), although I now prefer and recommend the VLC player/converter for encoding .mp4 video files for the very best control and optimization for an HTML5 video or a videoTexture in babylon.js - within all browsers I tested today. So encoding and playback .mp4 video files using most video editors and encoders was never a problem on desktops and most browsers, but the best solution in my opinion for .mp4 is definately the VLC player/converter.

 

Now, for the main problem which I have been having playback issues for 2 months now, is that I could not encode a .webm or .ogv video file for playback on all mobile devices, and at best, only audio and rarely video playback worked - dependng on the mobile device and the browser on various tablets and phones - both Android and Iphone.

 

So in despair I kept at it, and after trying practically everything I could think of, I spent the past two days just working on encoding and analyzing the resulting video, compression, bitrates, etc. What I discovered is that if I encode both the .webm and .ogv using the FireFogg extension for FireFox, and using the correct advanced settings, I now have video files which so far have played back both video and audio correctly on multiple mobile devices, OS versions, and browsers without fail.

 

I found that I did not need to modify my scene (scripts and application) in any way - that the problem was only due to encoding and conversion of video files. I have read other posts in the past where users have had similar issues and there was apparently no solution, so I hope now that no one else has to go through the lengthy troubleshooting process which I had to do - along with all of the other seasoned users on this forum who aided me as much as they were able.

 

If anyone needs the general conversion settings and specifics I found to universally work for video and audio on most all devices, browsers, and OS versions, please let me know and I'm happy to post these. If you believe this is appropriate to add to the documentation site, I'll certainly update the site - however, these might change in future revisions of browsers and OS versions, but I doubt it.

 

So for me, this is a banner day.    Although, it has been one of the most painful periods I've ever spent troubleshooting any application, since there is no consistant documentation online for HTML5 video encoding for mobile devices - as nothing I read online worked. And as working in baylon.js is generally new to practically all of us and evolving rapidly, there was no basis for believing it wasn't in my scripting or usage. I simply tried everything and stumbled upon a very reliable set of encoding solutions.

 

Thank you to all those who responded to my posts and put in the time to provide help. I'm just very happy that I found a solution - not only for me, but for everyone who might now avoid any incompatabilities in the future.

 

 

I hope that this is valuable info for users now and in the future, and that no one else needs to spent any time troubleshooting video playback on any device or display.

 

Cheers,

 

DB

[Vousk-prod.]

So, to resume briefly :

 

To have HTML5 video working in every browsers, every devices, we need to :

. convert the video in .mp4 with VLC

. convert the video in .webm and .ogv with FireFogg extension of Firefox

 

Would love to know the settings you are using in those app dbawel, it's always a long quest to obtain the correct ones.

 

Other piece of info, Adobe Premiere has always been bad at exporting video, but the Adobe Media Encoder is really powerfull and efficient (but I've never tried to convert files for HTML5 video). Also, two years ago I managed to successfully convert my video files for every web format with Miro Video Converter, but I don't know if this software is still ok today.

[dbawel]

Hi VP,

 

I'm certain there are numerous applications to encode video to play on mobile devices. However, I have always found limitations to every application I've tested now except for VLC for MP4 and FireFogg for WEBM and OGV files. Finally, after many painful weeks, every device I've tested thus far palys back correctly using these two applications - thank God - finally.

 

I have meetings beginning now and through the afternoon, but later today I will post the settings I've found to produce the most compatible video files I've yet discovered. I should have listed these in this post already, but it was rather late last night when I wrote this post. But I will do this later today when I return home.

 

I've also tested the Miro Video Converter, however, once I stumbled upon FireFogg, I found what appears to be complete compatibility, so I personally will use this until I discover an incompatible device, browser, or OS. I'm sure there are other solutions, but what I discovered after a great deal of work has not yet failed on any device or configuration, so this is certainly what I would personally recommend since these have been as thoroughly tested as possible.

 

Cheers,

 

DB

[dbawel]

Hello Babylonians,

 

Actually, In my first post I meant to report that it is best to use Handbrake for MP4 video file conversion and compatibility, which is an extension of the VLC player - and not the VLC player directly.

 

For MP4 compatible file conversion -

You can find the download for Handbrake at:

https://handbrake.fr/downloads.php

 

So you won't be converting any files in VLC, but download the Handbrake application which is free, and use the following settings. Always keep in mind that many of these settings are dependant on the source video file, as all of the quality settings may not be available to you if your source video is of low quality for video and/or audio. So you will want to begin all conversions from the highest quality video/audio source available. Many users make the error of starting with a low quality video file, which will then limit them in the available settings for conversion.

 

1.  First, load your source file, and in the right hand panel, select either the "iphone and ipad touch" setting, or the "android tablet" setting - either is fine as both use the same default settings.

 

2.  In the Picture tab, either leave the width and height the same as the source, or I have had no problems setting a new width and hieght as needed. You should be able to crop your video as well if this is something you choose to do. Leave all other setting to default in this tab.

 

 

3.  In the filters tab, make sure all settings are off (they should be by default), and make sure Deinterlace is selected and grayscale is unchecked.

 

 

4.  In the video tab, leave most of these settings at their default, make sure the video codec is set to H.264 and the frame rate is set to 30 fps. In the quality section of this tab, select Avg. Bitrate, and you can use a setting of 1000 Kbps or greater - although 1000 Kbps is fairly poor quality video for desktops and laptops - however, if your target is mobile devices and displays, a Kbps setting of 1000 will display well with few visible artifacts on mobile phones and most tablets - and 1000 Kbps will keep your video file size very small and fast to load.

 

But you will most likely require higher video quality if people will be viewing your scenes on desktops and laptops, so you will want to increase the Kbps setting to greater than 1000 Kbps - which I recommend beginning at a setting of 2000 Kbps and then view (test) your scene at this bitrate on your laptop/desktop to see if this is good enough quality for your needs.

 

I also recommend that you check 2-Pass Encoding, although this will take slightly longer for processing, but ensures that your bitrate will be consistant for the entire length of the video. So I would always check this button before processing.

 

 

5.  In the audio tab, make sure the codec is set to AAC (avcodec), and set the bitrate to 128. If this bitrate vale doesn't appear in the pull down menu, it means that you are using a source video file of lower audio quality than 128 Kbps - which is generally OK, although your audio quality will not be very good. I recommend setting a bitrate of 128 Kbps, however, anything lower than 64 Kbps will definately sound like crap - and for video files of a few minutes or less, you will see very little difference in file size between 64 Kbps and 128 Kbps.

 

Next, set your Mixdown menu to either read Mono or Stereo. I generally use Mono for videoTextures which are simply playing on an object in a scene as background, however if you want more fidelity, then of course you can select Stereo - however Mono will allow you to more easily mix the audio of your screen into a global stereo mix for your scene, providing better directional audio if moving around in a gaming environment.

 

But if your focus is the video itself, then choose stereo, as you will certainly get better fidelity from this - especially if it is a song recorded in stereo. I would avoid using pro-logic or and more channels than stereo if you truly want compatability, as anything other than Mono or Stereo requires a codec to play, and also is more difficult to mix into a sound designed environment for directional audio mixing.

 

 

6.  Leave the subtitles and chapters tabs alone and don't change any settings in these tabs.

 

 

7.  Lastly, make sure your output settings are set to MP4, and you must check the Web Optimized checkbox. Then set an output path and filename using the "browse" button, and press "start" at the top of the application. That's it for fully generating the most compatible MP4 video files, and you simply have to wait for a few seconds (depending on the length of the video source file and quality settings for conversion.) Once completed, you'll find your newly converted file in the directory you specified for output.

 

 

For .webm and .ogv file conversion -

For full device compatibility, you will need to use the FireFogg extension directly in Firefox to generate both .webm and .ogv (ogg) video files. First, you must have Firefox installed on your computer, and then go to: http://firefogg.org/ to download the Firefogg extension. You must download and install this using the Firefox browser, or the Firefogg extension will not install.

 

If you have sucessfully installed the Firefogg extension in Firefox, when you go to http://firefogg.org/ you will see a green box in the middle of your screen which reads "Firefogg installed". If your Firefox browser does not say this, then you have not installed the extension correctly. This may be the case, as there is currently no valid signature for the installation of Firefogg at http://firefogg.org/ - so if you are not able to install as I was not you must do the following:

 

You will need to set the xpinstall.signatures.required preference to "false".​

1.  Type about:config into the URL bar in Firefox.

2.  In the search box, type xpinstall.signatures.required

3.  Double-click the preference, or right-click and select "Toggle" to set it to false.

 

You will now be able to install the Firefogg extension into Firefox using the Firefox browser. I recommend after installing that you set your xpinstall.signatures.required back to "true". for security.

 

Once you have the Firefogg extension installed and see the green box stating that it is installed, follow the instructions below to convert your source video to compatible .webm and .ogv video files for most all mobile devices. This process and settings are very, very simple to execute.

 

1.  In your Firefox browser, go to http://firefogg.org/ and click on " Make web video".

 

2.  Click on the "Select File" button and a file selection window will open. Make sure you select you high quality MP4 video file, and click "open". This should then automatically switch to the options menu, and display the name of your source file, your file specifications, and pull down menus as well as other buttons to click for defining your conversion settings.

 

3.  Set your output file to either WebM or Ogg (.ogv file format), and leave the preset menu at 480p (2 Mbit connection). 

 

4.  Click on "Advanced Options..."  and you'll need to fill in most of these dialogue settings. Make sure the "Video" checkbox is selected, and input the following:

A.  Your X and Y resolution for "size" (Width and Height).

B.  "Framerate" set to 30 (fps).

C.  "Aspect ratio" (which should be either 4:3 or 16:9)

D.  "Bitrate" which should be 1000 (Kbps) or greater - please read above, however, this can be lower than your MP4 file as both the .webm and .ogv video files will be dispalyed on screens of lower resolution than your MP4 video file. But if you need to reduce the size of your video file and improve load time, then a setting of 500 can work with a bit less quality. So I encourage you to test until you are familiar enough with your playback quality on multiple mobile devices. For .webm and .ogv video files I have found that If I'm using a value of 1000 Kbps for my MP4 file coversion in Handbrake, that to produce a similar quality for .webm and .ogv in playback, that I require a Kbps rate higher than what I'm setting for conversion in Handbake. So if I use a setting of 1000 Kbps for my MP4 file in Handbrake, to produce similar quality video playback on mobile devices, I need to set a Kbps setting of at least twice what the MP4 bitrate was set to - which is a value of 2000Kbps or greater, as the compression of these two video formats is greater than the MP4 video file format. So I recommend you experiment to familiarize yourself with these setting first.

E..  "Quality" which is a value between 0 and 10 with 10 being the highest quality setting and 0 almost unusable. Again, I encourage experimentation as this setting depends on how high you want the compression to be on disk and in playback. And again, if you want to lower the size of your video file, then a setting of 5 will also contribute to this.

F.  I set the "Keyframe interval" to a default setting of 64, as I have found this a reliable setting - although I have not had time to experiment with this setting as of yet.

G.  I leave all other video options as is, except to set 2 pass ending on as this will process your compression more evenly over the length of the entire video file.

H.  For the Audio options, I leave everything at it's default, unless I need to set my Channels to Mono to match my MP4 video conversion. Otherwise, like the MP4 conversion, I recommend only choosing either Stereo or Mono as the setting.

 

And that is it - just hit the "Encode" button and you are finished with either the .webm file format or the .ogv file format. Once you have encoded one of these formats, then repeat using the same setting to convert to the other format as you wil require both .webm and .ogv video files for full mobile device compatibility. The only other action you will need to do is make certain that the file extension for the .ogv video file is ".ogv" (ogg), as the file will most likely be named with an .ogg file extension, and you will be calling an .ogv video file in your script.

 

Now just make sure that in your script you are calling all 3 video file formats as in the following scipt example:

 

var videoTexture = new BABYLON.VideoTexture("video", ["textures/Dora.mp4", "textures/Dora.webm", "textures/Dora.ogv"], scene, true, true);

 

This should support video and audio playback and compatiblity for almost all desktop and mobile devices. It has worked on every device I and others have tested thus far, so I hope this helps and that no one else has to waste countless hours aimlessly troubleshooting and experimenting with conversion tools, video settings, and testing on devices as I have these past weeks. But please keep in mind that on many mobile devices, these settings have tested out - but file size is key - the larger your file size, the longer it will take to load and play. So wait for your object with the videoTexture to appear, and it will take a few more seconds to initiate play either from a command in your script, or a button in your GUI - just be patient, and the video WILL play. If anyone has any further questions concerning what I've learned in the process, please let me know.

 

Cheers,

 

DB

[Vousk-prod.]

Wow, that's precise instructions!

Thanks, that certainly will help many! (and not only "3D engine's people", I think this post will soon be a reference for everywhone in needs of HTML5 video)

[Dad72]

A very good video converter (for andoid too) is "freemake video converter" with which I have never had a problem and he converted a click with the 3 video formats html5 at once.

 

IT supports around 300 formats. this software is excellent.

 

Englich : http://www.freemake.com/free_video_converter/

French : http://www.freemake.com/fr/free_video_converter/

[RaananW]

Freemake would also be my choice. Easy to use, hassle free converter

[dbawel]

I'll try Freemake as well. but regardless of what aplication is used, for playback on mobile devices, it is a balance between resolution, quality (bitrate), compression, and other factors. The more control you have over these parameters and over your video file the better. And using the absolute lowest video resolution to achieve the required quality in your scene and the resulting file size determines if your video plays within 2 seconds or takes 30 seconds to load and play - and I've found that the differences in a video file don't need to be very different to cause a huge delay in playback.

 

So if you're experiencing a delay beyond a few seconds (depending on resolution and length of video, of course), then I recommend testing all settings which will provide you with a smaller video file size without sacrificing much quallity. So keep using whatever works for you - however, I have certainly come across many video applications which are not compatible with many or some mobile devices - such as writing .ogv and webm file from Vegas, although now I've been able to find settings using Sony Vegas to write an MP4 file which pays fine on mobile devices. But as the number of video converters and editors are too many to list, between the seasoned users on this post, hopefully others will read this and not have anymore "showstoppers" in scenes which won't play video files.

 

I might add some of this to the documentation, however it wouldn't be specific to any application, but would list and define the necessary parameter which require controlled settings and what those settings represent - as well as the relationship between these and the resulting video resolution, file size, and video format.

 

DB