Blender Exporter 5.1

[JCPalmer]

The Scene Exporter properties has changed to support 3 texture process methods: Legacy, Inline, and the new Prioritized.  (Another minor change was to now filter files with a .babylon when choosing)

A compressed texture is different from a lossy image format (think .JPG) in that it is not expanded by the CPU nor GPU.  This results in better GPU RAM usage for the same amount of textures.  A consequence of this is it must be supported by both the GPU and the browser's webGL, as well as BJS, of course.

This is where this change helps out.  It allows you specify these extra formats that you created from the image used by Blender, and in the order in which to attempt to load them.  The blender version is tacked onto the end of the list.  Without this new built-in fall back for BJS 2.5, compressed textures were pretty much useless.  You would have to know exactly what the customer's hardware / revision & browser was, because you only got one shot.

As DDS format is the only one currently implemented  in BJS for certain hardware, some of these others is for the future.

ASTC, is a WebGL extension, so it can be added to either webGL 1 or 2.  Many mobile GPU support it, and Chrome with --enable-webgl-draft-extensions flag.  Not sure what OSes, GPUs, or drivers are required now though.  I found one thing to convert files. 

ETC, is really ETC1 & ETC2.  I believe it is part of OpenGL ES3, which is on what WebGL 2.0 is based.  As far as a converter, I know of this.

[GameMonetize]

Really cool! (And I updated the PG with the support for texture fallbacks)

[JCPalmer]

@Deltakosh, is there some way to test for the ASTC extension in the PG?  I do have a fairly new Android Tablet.  If I can determine that it has the extension after turning on draft extensions in Chrome, I'll probably see it I can add BJS support for after Thanksgiving.  Actually adding an engine call to dump all extensions to console a return string would be quite useful, if that is even possible without knowing the name(s) of them in advance.

[GameMonetize]

We already are testing extensions there:

https://github.com/BabylonJS/Babylon.js/blob/master/src/babylon.engine.ts#L552

 

So I see no problem to add another test for ASTC. Do you know which extension to test?

[JCPalmer]

First, Playstore has an OpenGL Externsions Viewer. 

ASTC is KHR_texture_compression_astc_ldr  , spec

This tablet does not have it.  Since it is OpenGL ES 3.0 it supports both ETC2 and even another compressed texture format EAC.

ETC2 & EAC is WEBGL_compressed_texture_etc.  It is really a bunch of extensions grouped together.  It says that this is both WebGL 1.0 & 2.0.  I see all of the individual extension in the viewer.  Wonder if these are also exposed in Chrome?

[GameMonetize]

can you go there: http://renderingpipeline.com/webgl-extension-viewer/

 

[JCPalmer]

Thanks, yeah looks like Chrome supports WEBGL_compressed_texture_etc1, even without the drafts flag turned on.  I now also see WEBGL_compressed_texture_s3tc on my Linux machine.  Is that dds?

I am going wrap up QueuedInterpolation (by Thanksgiving), then look here.  I cannot afford to seed this advantage to my native app competition.  Now that I see what you did to implement this in engine, I now have enough info to improve it.  First is to make it file missing tolerant.

[GameMonetize]

yep! WEBGL_compressed_texture_s3tc   is DDS 

[JCPalmer]

Since I do not actually have the hardware to test astc, plan on adding pvrtc for iOS, and etc1 for Android.  The app I referenced above, show etc2 is available, but apparently not being exposed by chrome, even with --enable-webgl-draft-extensions flag.

Was just going to follow the pattern for s3tc as much as possible.  In EngineCapabilities:

public pvrtc: WEBKIT_WEBGL_compressed_texture_pvrtc;
public etc1: WEBGL_compressed_texture_etc1;

These error.  Is there a place you put WEBGL_compressed_texture_s3tc to get it to compile?  Could these just be any's?

In engine constructor:

this._caps.pvrtc = this._gl.getExtension('WEBKIT_WEBGL_compressed_texture_pvrtc');
this._caps.etc1 = this._gl.getExtension('WEBGL_compressed_texture_etc1');

For Engine.createTexture, I think I want to wrapper the old call without the array with a replacement createTexture.  It would perform each element of the array, first checking the extension existence, but also calling the original createTexture in a try block.

[GameMonetize]

the s3tc is in lib.d.ts (by default with TS).

Not sure there is the same for other extension. You will have to use any I guess

[gryff]

20 hours ago, JCPalmer said:

Since I do not actually have the hardware to test astc, plan on adding pvrtc for iOS, and etc1 for Android.  The app I referenced above, show etc2 is available, but apparently not being exposed by chrome, even with --enable-webgl-draft-extensions flag.

@JCPalmerWell that looks like a bit of a nightmare, Jeff. Hardware, OS, browsers and activating flags for stuff to be usable.

What hardware is needed for ASTC?

And then all the name variations - what is BC1-BC7, and DDS is "WEBGL_compressed_texture_s3tc".?

I guess you just have to include a default .png or .jpg. Just more work.

cheers, gryff

[JCPalmer]

The flag mentioned is only for testing stuff which is not yet production.  Systems showing ASTC can be found here.

I think I have made a mistake implementing this from an exporter though.  Would mean every exporter would need to implement.  Still time to move it out of Texture constructor & put everything in Engine.

Have begun work on an intelligent examiner of hardware to determine what machine supports in Engine constructor.

this._caps.astc  = this._gl.getExtension('WEBGL_compressed_texture_astc');
this._caps.s3tc  = this._gl.getExtension('WEBGL_compressed_texture_s3tc');
this._caps.pvrtc = this._gl.getExtension('WEBGL_compressed_texture_pvrtc') || this._gl.getExtension('WEBKIT_WEBGL_compressed_texture_pvrtc'); // 2nd is what iOS reports
this._caps.etc1  = this._gl.getExtension('WEBGL_compressed_texture_etc1');
this._caps.etc2  = this._gl.getExtension('WEBGL_compressed_texture_etc') || this._gl.getExtension('WEBGL_compressed_texture_es3_0'); // first is the final name, found hardware using 2nd

...

// Intelligently add supported commpressed formats in order to check for.
// Check for ASTC support first as it is most powerful and to be very cross platform.
// Next PVR & S3, which are probably superior to ETC1/2.  
// Likely no hardware which supports both PVR & S3, so order matters little.
// ETC2 is newer and handles ETC1, so check for first.
if (this._caps.astc ) this.texturesSupported.push('.astc');
if (this._caps.s3tc ) this.texturesSupported.push('.dds');
if (this._caps.pvrtc) this.texturesSupported.push('.pvr');
if (this._caps.etc2 ) this.texturesSupported.push('.etc2');
if (this._caps.etc1 ) this.texturesSupported.push('.etc1');

this.useHwBasedTexSelection = this.texturesSupported.length > 0;
if (this.useHwBasedTexSelection) {
     Tools.Log('Compressed textures formats HW supports: ' + this.texturesSupported);
}

There would be 2 properties to added to Engine:

// Hardware supported Textures (public for overriding after instancing)
public useHwBasedTexSelection : boolean; // set to false to ignore
public texturesSupported = new Array<string>();

You would have same workflow creating either .jpg or other IMAGE format.  This is the name of the file passed.  Based on what a piece of HW / Browser supports, files with extensions of compressed textures will be tested for / used first.  Need to ensure a 404 (file not found) is not a problem.

You would run tools which generate the other formats from the image format out of the export.  Not every scene merits this, but in order to provide ever more texture rich scenes on mobile hardware, not allowing this type of process, just a hardwired .DDS with no fallback is not competitive.

Another advantage might be load time.  QI uses a different process than .babylon to create Textures.  I noticed creating mipmaps from image formats takes a decent proportion of load.  Compressed textures have mipmaps in the file.

Here is a demo from PlayCanvas which shows their texture selection technology.  Run it on different hardware. This process should be very comparable to their's.

[JCPalmer]

Note to self, or anyone else this feature / concept is not compatible with databases /  manifests.  Databases assume that everything needs to be downloaded.  Only the BEST format for the hardware does.

In order to determine best fit, need to do quick header requests which are NOT Async.  Adding small function to Tools:

/**
 * This cannot currently be used with a database.
 */
public static doesFileExist(url: string) : boolean {
    var request = new XMLHttpRequest();
    var loadUrl = Tools.BaseUrl + url;
    request.open('HEAD', loadUrl, false);  // false, since this is NOT async
    request.send();
     
    return request.status >= 200 && request.status < 300 || (navigator.isCocoonJS && (request.status === 0));
}

 

[gryff]

1 hour ago, JCPalmer said:

Here is a demo from PlayCanvas which shows their texture selection technology.  Run it on different hardware. This process should be very comparable to their's.

@JCPalmer : Nice Demo Jeff

On my two desktops:  "VRAM 87.7MB, using DXT saving 440.3MB" .

On my tablet (Samsung Tab A -Android 5) VRAM 87.7MB, using ETC1 saving 440.3MB.

By the way Jeff, I like the idea of the Prioritized option and then adding the choices in the exporter.

1 hour ago, JCPalmer said:

You would have same workflow creating either .jpg or other IMAGE format.

Well from the information available it seems you could spend some time picking all kinds  of options to get a satisfactory level of "noise". And what graphics packages support all these different options ... or is that to come?

cheers, gryff

[JCPalmer]

I also get "66.8MB using PVR saving 459.2 MB" on iOS.  Sorry though, going to rip out that in Blender exporter.  I understand the desire to set the priority, but this really needs to be driven by what each individual machine can do.  Best to do it on the fly.

A developer need not provide all formats, or wait can till the end of a project to generate the compressed formats.  Selector will be tolerant of missing formats (or not yet provided) to make work flow easier.  As far as convertion tools,  there are a number of them.  This will evolve.  Yes, ASTC especially has a number of different options.  If you want to sell your game, you may want to play around more.

[JCPalmer]

@Deltakosh, I reversed the changes for passing an array in Textures.  This is my coded wrapper in engine.  It compiles, after some testing to see that we are back to working, is it alright to commit?  Have a tools.pvr.ts, but blows up.  Also need to both edit Blender & build a test scene in my blog repo.  Consider this to be the framework.  As formats get built, rip out those continues.  Allows others to work on a format who may have the hardware, or time, and decouples a little from 2.5 release.

public createTexture(url: string, noMipmap: boolean, invertY: boolean, scene: Scene, samplingMode: number = Texture.TRILINEAR_SAMPLINGMODE, onLoad: () => void = null, onError: () => void = null, buffer: any = null): WebGLTexture {
    var fromData = url.substr(0, 5) === "data:";

    if (this.useHwBasedTexSelection && !fromData && !scene.database) {
        var lastDot = url.lastIndexOf('.');
        var withoutExtension = url.substring(0, lastDot);
        var extension = url.substring(lastDot);
        for (var i = 0, len = this.texturesSupported.length; i < len; i++) {
            var testUrl = withoutExtension + this.texturesSupported[i];
            
            // code to allow the formats to be added as they can be developed / hw tested
            if (this.texturesSupported[i] === '.astc') continue;
            if (this.texturesSupported[i] === '.pvr' ) continue;
            if (this.texturesSupported[i] === '.etc1') continue;
            if (this.texturesSupported[i] === '.etc2') continue;
            
            if (Tools.doesFileExist(testUrl)) {
                return this._createTextureInternal(testUrl, noMipmap, invertY, scene, samplingMode, onLoad, onError, buffer);
            }
        }
    }
    // fallback to original type passed
    return this._createTextureInternal(url, noMipmap, invertY, scene, samplingMode, onLoad, onError, buffer);
}

I also made changes to compute useHwBasedTexSelection, edited above too.

[GameMonetize]

I'm fine with it

[JCPalmer]

I ran into problems with the doesFileExist.  Did not help extra complains in console that sync http calls are depreciated.  Changed so that you now need to call engine.setTextureformatToUse().  Will probably open a new thread where the status of new formats can be discussed.

/**
 * Set the compressed texture format to use, based on the formats you have,
 * the formats supported by the hardware / browser, and those currently implemented
 * in BJS.
 * 
 * Note: The result of this call is not taken into account texture is base64 or when
 * using a database / manifest.
 * 
 * @param {Array<string>} formatsAvailable - Extension names including dot.  Case
 * and order do not matter.
 * @returns The extension selected.
 */
public setTextureFormatToUse(formatsAvailable : Array<string>) : string {
    for (var i = 0, len1 = this.texturesSupported.length; i < len1; i++) {
        // code to allow the formats to be added as they can be developed / hw tested
        if (this._texturesSupported[i] === '.astc') continue;
        if (this._texturesSupported[i] === '.pvr' ) continue;
        if (this._texturesSupported[i] === '.etc1') continue;
        if (this._texturesSupported[i] === '.etc2') continue;
        
        for (var j = 0, len2 = formatsAvailable.length; j < len2; j++) {
            if (this._texturesSupported[i] === formatsAvailable[j].toLowerCase()) {
                return this._textureFormatInUse = this._texturesSupported[i];
            }
        }
    }
    // actively set format to nothing, to allow this to be called more than once
    // and possibly fail the 2nd time
    return this._textureFormatInUse = null;
}

 

[V!nc3r]

This texture-export change looks very interesting ! My work-rush will soon ended, i'm impatient to test this, especially on low-end hardware.

[ozRocker]

compression for iOS?? so a 4096 x 4096 texture doesn't have to take up 48MB of iPhone RAM anymore?  This is awesome!!! 

[V!nc3r]

For those who doesn't notice, normal map value is now correctly exported. You don't have anymore to do this tips.

[edit] Thanks to JCPalmer [/edit]

[JCPalmer]

12 hours ago, ozRocker said:

compression for iOS?? so a 4096 x 4096 texture doesn't have to take up 48MB of iPhone RAM anymore?  This is awesome!!! 

couple things.  In 2.5, only the frame work for doing platform based compressed texture selection is in place.

Also, if you mean 4096 x 4096 x 3, then you are forgetting about the space for minmaps.  For a 4k texture, you need to add space for a 2k, 1k, 512, 256, … version as well.  Not only that, but the time to build them really can slow down load time.  The minmaps for .DDS, .ETC, .PVR, & .ASTC are already in the file (if added), so they too are compressed, and probably load quicker then being computed.

I had started PVR, but I lost it due to my hardware circumstances.  Have a new motherboard, memory, & cpu arriving tomorrow.  Do not know when I will restart this.  Anyone who wants to try getting .ETC1 / .ETC2 working for Android, feel free.