Getting black Pixi sprite from loading KTX2 file [TypeScript]

[Deoxyz]

Hi

I'm getting black textures when I'm trying to load KTX2 files using loaders.gl and use the UInt8Array to load a texture with from or fromBuffer and create a Pixi sprite with it.
It doesn't matter if I make a BaseTexture first or not.

Code with BasisLoader & Texture.fromBuffer:

import { BasisLoader } from '@loaders.gl/textures';
import { load } from '@loaders.gl/core';
import * as Pixi from 'pixi.js';

...

const loadOptions = {
     format: 'auto',
     containerFormat: 'ktx2',
     module: 'encoder',
};
const resources = await load(filename, BasisLoader, loadOptions);
const textureData = resources[0][0];
const texture = Pixi.Texture.fromBuffer(textureData.data, textureData.width, textureData.height);
const sprite = new Pixi.Sprite(texture);

Code with CompressedTextureLoader & Texture.from with BaseTexture:

import { CompressedTextureLoader } from '@loaders.gl/textures';
import { load } from '@loaders.gl/core';
import * as Pixi from 'pixi.js';

...

const loadOptions = {
	useBasis: true,
};
const resources = await load(filename, CompressedTextureLoader, loadOptions);
const textureData = resources[0];

const baseTexture = new Pixi.BaseTexture(
  new BufferResource(textureData.data, {
    width: textureData.width,
    height: textureData.height,
  })
);
const texture = Pixi.Texture.from(baseTexture);
const sprite = new Pixi.Sprite(texture);

Log output:

 

I also tried Pixi's CompressedTextureResource:

const baseTexture = new Pixi.BaseTexture(
	new Pixi.CompressedTextureResource(textureData.data, {
		format: textureData.format,
		width: textureData.width,
		height: textureData.height,
	})
);

But got this:

 

KTX2 Container is not (yet) implemented into pixi, that is why I had to try other ways to get it loaded into a pixi sprite. I'm not fully understanding all of the compression stuff yet, so I'm probably forgetting something.

To be sure you get enough information: I used KTXSoftware's toktx tool to create a KTX2 file from a png: https://github.com/KhronosGroup/KTX-Software 
I tried these commands and have loaded them all in:

ETC1S
.\toktx.exe --t2 --encode etc1s --clevel 5 --qlevel 255 book.ktx2 book.png
ASTC
.\toktx.exe --t2 --encode uastc --uastc_quality 4 book.ktx2 book.png
UASTC
.\toktx.exe --t2 --encode astc --astc_blk_d 4x4 --astc_quality 100 book.ktx2 book.png

 

So if someone could point me in the right direction, that would be awesome.
Thanks in advance!

[Deoxyz]

I just found out I was looking over the warnings, as I was was only in the error section of the webconsole. The warning looks like this:

 

[TobiasW]

Did you ever solve this? I've been seeing the same behaviour with version 6.

[Deoxyz]

2 hours ago, TobiasW said:

Did you ever solve this? I've been seeing the same behaviour with version 6.

I'm still figuring things out. Trying some different approaches, but I will post my solution here once I've found it.

[Deoxyz]

On 3/2/2023 at 9:05 AM, Deoxyz said:

I'm still figuring things out. Trying some different approaches, but I will post my solution here once I've found it.

So I think I'm a step further, but I get undefined from the BasisParser that I use from Pixi itself.
What did I do? I created and added an extension to load KTX2 files using Assets.load:

import { TranscoderWorker } from '@pixi/basis';
import { loadKTX2 } from '../loaders/PixiLoadKTX2'; // Custom extension
...

constructor() {
	Pixi.extensions.add(loadKTX2);
	TranscoderWorker.loadTranscoder(window.location.origin + '/basis_transcoder.js', window.location.origin + '/basis_transcoder.wasm');
}

public async create(filename: string): Promise<void> {
	const texture = (await Assets.load(filename)) as Pixi.Texture;
 	const sprite = new Pixi.Sprite(texture);
  	...
    
}

I added my extension code in the attachments, which is based on the loadBasis extension from Pixi itself. I also use KTX-Parse to read the ktx2 file from which (I think) I use the basis data to transcode it to an usable pixi texture:

import { read } from 'ktx-parse';
...

const response = await settings.ADAPTER.fetch(url);
const arrayBuffer = await response.arrayBuffer();
const ktxBuffer = concat([arrayBuffer]);
const ktxContainer = read(ktxBuffer);  // KTX-Parse read function

let textures: Texture<Resource>[] = [];
await new Promise<void>((res) => {
	ktxContainer.levels.forEach(async (level, index) => {
		const resources = await BasisParser.transcode(level.levelData); // Pixi basis transcoding
		
  		const type: TYPES = BASIS_FORMAT_TO_TYPE[resources.basisFormat];
		const format: FORMATS = resources.basisFormat !== BASIS_FORMATS.cTFRGBA32 ? FORMATS.RGB : FORMATS.RGBA;

		textures = textures.concat(
			resources.map((resource) => {
				const base = new BaseTexture(resource, {
					mipmap: resource instanceof CompressedTextureResource && resource.levels > 1 ? MIPMAP_MODES.ON_MANUAL : MIPMAP_MODES.OFF,
					alphaMode: ALPHA_MODES.NO_PREMULTIPLIED_ALPHA,
					type,
					format,
					...asset.data,
				});
				return createTexture(base, loader, url);
			})
		);
...

But I get undefined in 'resources' when transcoding the basis data. When I look further into the BasisParser code in my node-modules and add some logs to debug, I see that it stops at the worker trying to transcodeAsync():

...
await worker.initAsync();
// ---- Stops here
const response = await worker.transcodeAsync(new Uint8Array(arrayBuffer), _BasisParser.defaultRGBAFormat.basisFormat, _BasisParser.defaultRGBFormat.basisFormat);
// ----
const basisFormat = response.basisFormat;
const imageArray = response.imageArray;
const fallbackMode = basisFormat > 12;
...

The error:

tslib.es6.js:74 Uncaught (in promise) undefined

That's where I'm stuck at the moment.
I don't know if I use the BasisParser or even the KTX2 data correctly? Or that it could even work this way in the first place?

I don't know if someone from Pixi that is on this forum sees what I'm doing wrong?
I have a feeling that I'm very very close 😁

Anyway, thanks in advance.

PixiLoadKTX2.ts

[Deoxyz]

I have implemented the KTX2 container in Pixi.js myself and I might ask for permission to contribute this to the project. But I first want to test some more and make sure the code is good enough. 😁
Proof that it worked: