How textures are handle by hardware in webGL ?

[V!nc3r]

On desktop I have a clear view of how it works, so I will start by asking how textures are manage on mobile hardware (tablet or smartphone) but of course if someone wants to ask something about desktop, this topic is open.

We assume that all textures size are obviously in power of two.

• If I have textureOne, jpg file, and textureTwo which is the same but in png, do they take the same amount of ram once loaded ? If yes, using for example tga will change only the download time and not the performance ?
• Could uncompressing a texture kill some FPS ? (example : jpg file to RAM ; png file to RAM)
• Is Mobile hardware have VRAM or only RAM shared between CPU and GPU unit ? It depends of the device I supposed ?
• Is there a way to use mipmaps or is this specific to dds format ?

Thanks

[GameMonetize]

Hey!

1. Yes as they are uncompressed and save as lain bitmap. Only DDS keeps data compressed (when DDS is supported)

2. Nope. This is utterly fast to decompress. Loading time can be the problem though

3. It depends. iPhone has really fast hardware for instance

4. Mipmaps are on by default with babylon.js and are automatically computed at loading time

[V!nc3r]

Ok great !

About the mipmaps, I was thinking that only dds allows it ; but in fact the benefit of the dds is "just" that the texture is keep compressed if I understand (which is a great benefit of course).

Out of curiosity, how works the process to compute mipmaps on the fly ? Depending on the camera distance, the full texture file is loaded, resize, then use ?

How BJS decide to use 128²px rather than 2048²px ? It depends of a ratio screen-pixels / texture-pixels ?

[GameMonetize]

mipmaps are generated on GPU side

if you load a 512x512 picture, the mipmaps will be 256x256, 128x128, 64x64, 32x32, 16x16, 8x8, 4x4

 

 

[Rodrix3]

Just to clarify, at *run-time* (after all assets have loaded) is there any benefit to using DDS textures?
Or is the benefit of DDS is just avoid a compression step and making loading time faster?

Would actually using DDS textures would get a scene have higher FPS than the same using PNG files?

If so, how can we create DDS textures? (What tool and what settings do you recommend)?
Thanks in advance @Deltakosh

[Guest]

DDS can contains compressed data but this will only work on Windows basically

DDS can save the time required to build the mipmaps but at runtime this will change nothing (once everything is loaded)

[Rodrix3]

3 minutes ago, Deltakosh said:

DDS can contains compressed data but this will only work on Windows basically

DDS can save the time required to build the mipmaps but at runtime this will change nothing (once everything is loaded)

Thanks so much for the reply!
So basically there is no benefit at runtime (on the render loop). 
The only benefit is the loading, correct?

This means I will see no performance boost  on the render cycle if I use dds textures. Right?

Thanks again!

[JCPalmer]

On 8/9/2016 at 4:32 PM, Deltakosh said:

Hey!

1. Yes as they are uncompressed and save as lain bitmap. Only DDS keeps data compressed (when DDS is supported)

2. Nope. This is utterly fast to decompress. Loading time can be the problem though

3. It depends. iPhone has really fast hardware for instance

4. Mipmaps are on by default with babylon.js and are automatically computed at loading time

Sorry, but not accurate:

1 No, (as long as you use the case sensitive extension ".jpg") a gl.RGB texture will get created.  A ".png" extension will get a gl.RGBA texture created.  There are also a whole series of compressed formats using the platform independent texture container KTX.  Inside can contain any number of formats:  PVRTC, ETC1, ETC2, DXT, & ASTC.  DDS containers can only store DXT.  The best format to use is determined at load time for a given machine.

3.  Yes it depends, but either way a compressed texture takes up much less memory the .jpg / .png.  This dramatically increases with the increase in dimensions, & can be important for using 4K textures on mobile.  Many people do not need 4k textures, but an example of this is taking your .blend with all your static meshes, merging to a single mesh, and force baking (turn on in source code).  Now you get maybe dozens of draws reduced to one & less JS overhead as well.

A 4k texture in PVRTC with mipmaps is 5,462KB (both in file size & RAM).  a .jpg varies in file size, but expands to 67,108KB in RAM:

50,331,648 (4096 * 4096 * 3) +
12,582,912 (2048 * 2048 * 3) +
 3,145,728 (1024 * 1024 * 3) +
   786,432 ( 512 *  512 * 3) +
   196,608 ( 256 *  256 * 3) +
    49,152 ( 128 *  128 * 3) +
    12,288 (  64 *   64 * 3) +
     3,072 (  32 *   32 * 3) +
       768 (  16 *   16 * 3) +
       192 (   8 *    8 * 3) +
        48 (   4 *    4 * 3) +
        12 (   2 *    2 * 3) +
         3 (   1 *    1 * 3)
===========
67,108,863

4. KTX & DDS containers have mipmaps right in the file, since they cannot be computed at run time.  This is controlled at create time.

[Rodrix3]

5 minutes ago, JCPalmer said:

Sorry, but not accurate:

1 No, (as long as you use the case sensitive extension ".jpg") a gl.RGB texture will get created.  A ".png" extension will get a gl.RGBA texture created.  There are also a whole series of compressed formats using the platform independent texture container KTX.  Inside can contain any number of formats:  PVRTC, ETC1, ETC2, DXT, & ASTC.  DDS containers can only store DXT.  The best format to use is determined at load time for a given machine.

3.  Yes it depends, but either way a compressed texture takes up much less memory the .jpg / .png.  This dramatically increases with the increase in dimensions, & can be important for using 4K textures on mobile.  Many people do not need 4k textures, but an example of this is taking your .blend with all your static meshes, merging to a single mesh, and force baking (turn on in source code).  Now you get maybe dozens of draws reduced to one & less JS overhead as well.

A 4k texture in PVRTC with mipmaps is 5,462KB (both in file size & RAM).  a .jpg varies in file size, but expands to 67,108KB in RAM:

50,331,648 (4096 * 4096 * 3) +
12,582,912 (2048 * 2048 * 3) +
 3,145,728 (1024 * 1024 * 3) +
   786,432 ( 512 *  512 * 3) +
   196,608 ( 256 *  256 * 3) +
    49,152 ( 128 *  128 * 3) +
    12,288 (  64 *   64 * 3) +
     3,072 (  32 *   32 * 3) +
       768 (  16 *   16 * 3) +
       192 (   8 *    8 * 3) +
        48 (   4 *    4 * 3) +
        12 (   2 *    2 * 3) +
         3 (   1 *    1 * 3)
===========
67,108,863

4. KTX & DDS containers have mipmaps right in the file, since they cannot be computed at run time.  This is controlled at create time.

Thanks @JCPalmer for the insight.

Some questions arise then for me:

• I am using 16K textures for ambient occlusion maps and I am running the app on a Windows computer with 32 GB of RAM and a 1080 GTX card (this is the target of the app, for experimental purposes).
  Up to now I was using .png files for the ambient occlusion maps, which seems like a waste since the RGBA channels are being loaded, and 3 wasted, since the occlusion map is black and white.
  What image format do you recommend I use for the ambient occlusion map textures (I cannot combine them with the metallicRoughtness map since they are 4K, and it is reused by multiple textures, which the only difference is the occlusion map).
• Diffuse textures are also in PNG format, but do NOT use alpha channel. Using jpg would be a mistake since it would deteriorate the quality. What file format would be recommended for this case?
• "4. KTX & DDS containers have mipmaps right in the file, since they cannot be computed at run time.  This is controlled at create time." 
  
  @Deltakosh mentioned that mipmaps are generated created on runtime. Are you stating the opposite, if I understood well, and you are mentioning that there is actually a benefit to doing it at create time?
• I see little mentions about DDS in documentation vs KTX. Is KTX more recommended?  My target is Windows, and I just want to achieve the highest FPS with lowest footprint possible (as of this moment the application is too heavy and is facing a bottleneck on the textures related pipeline I believe).

Thanks in advance!

 

 

[JCPalmer]

• 16 K, wow that is big even for a 1080 GTX.  For scenes strictly for one machine / type, first see what this says (probably going to load DXT).  For something that big, I would suggest a DXT, probably in a DDS.  The PVRTool, see Compressed Textures Doc link from before can build either DDS or KTX container.  Note DXT is shown as BCx in that tool.  Do not know whether BC1 or BC2 does not have alpha.  Both should work, check file sizes.
• Same as first answer.
• Compressed have mipmaps in the file, .jpg & .png do not.  It has nothing to do with benefit.  It is not in openGL API to create mipmaps for compressed types.
• If you did not have such a tight list of hardware, using DDS would die on mobile.