closed PBR on iOS

[Guest]

We know that DDS works when loaded with absolute Uri but not when loaded with relative Uri.

We know that DDS are loaded by XHR.

 

So my question was more: can you show the profiler when loading the DDS with relative Uri? (Or is it what you showed here?)

if you can reference Babylon.max.js, please set a breakpoint around a line with "DDSTools.GetDDSInfo(data);". There must be a call to _loadFile before which is where the xhr will be triggered

 

and try to check what is going wrong.

 

(and yes it is not fast as we are fighting against a platform bug)

[paleRider]

Hi DK:

Thanks again man. We are almost done (I want to believe).

As soon as possible (this same afternoon)  I'll try what you're asking for.

Best regards.

[paleRider]

Hi there:

Well, apparently, what is happening is what is stated by Ashley Gullen in the previously identified article (https://www.scirra.com/blog/ashley/25/hacking-something-useful-out-of-wkwebview), when he writes:

Quote

One of the first things Construct 2 does when loading is to load the game's main data JSON file via XMLHttpRequest. This does not work in WKWebView: it appears to treat local files as if they came from a remote server, even though they're in the app itself, and such requests are blocked."

 

So I think all this is being ran on WKWebView and not UIWebView, which is great for a improved WebGL speed, but very bad for XHR loaded assets, because it is making my plain URI "assets/textures/environment.dds" being changed by iOS in something like this:

That is, the O.S. is trying to retrieve an inexistent file from the same location the local copy of BJS engine is stored in my app. What a disaster!!

 

Moreover, I can't get more help on this by debug-tool (remote debug from Safari running on my macMini ), as the set breakpoint (marked line 69348) in never reached:

 

In contrast, following the trace of the environment DDS loading:

texEnvironment=BABYLON.CubeTexture.CreateFromPrefilteredData("assets/textures/environment.dds",_Scene);

 

Bring us to the CubeTexture object, and there to its constructor, where the execution exists at line 50557, having _this.texture a null value.

As onLoad is false at line 50570 the constructor definitely exists w/o loading no texture at all.

 

Coming back to line 50557, texture property is null because so is retrieved by BaseTexture.prototype._getFromCache:

I think that must be the problem but I can´t discover it.

 

Of course, personally I can see all these platform problems with iOS-hybrid-apps as a huge fiasco in my attempt to use BJS as a functional cross-platform tool. Any help or solution? ... as loading the environment DDS textures the same way the rest of textures are, that is without the help of XHR functionalities (with app-local-storage synchro-loading is fast enough by far), . 
 

 

P.S.:

Please note that the babylon.3.1.custom.js is only as I've named the BJS code downloaded from babylonia.com>download-generator-page, using: v3.1 stable, unminified and w/o components at all.

Finally, while not my main problem now,  I get (keeping on iOS-Cordova) another error in my console, identified in the article of Ashely, as I can´t have the audio (water falling SFX) working, because of:

 

[paleRider]

Hi again DK:

I've just realized you were asking me for a profiler screenshot when using online environment DDS path, here you are:

I hope this helps!

Thank you for your time.

[Guest]

I find you a bit mean here. It is a huge fiasco but this is not our fault. How can we fix the problem coming from Cordova? Whatever engine you will use will have the same issue. The platform has a bug with local XHR. How can we deal with that???

 

We need to use a XHR to load the DDS file as IMG DOM element cannot load it. Please tell me how to avoid that and I'll gladly fix it

 

[paleRider]

Hi Deltakosh:

Of course this is not BJS Engine fault, but.. please, understand me, I thought it was possible to have a WebGL cross-platform framework capable to achieve the requirements of my average projects.

In this direction, the inclusion of PBR materials in the engine was great news. Sadly, you'll need environment textures if you are going to use PBRs in a proper way. And so is how we got to where we are now.

Well, if XHR is needed but we can´t use it under iOS-Cordova, ... the available options here are:

1. store that DDS in a web server and include the need for Internet connection in order to use de app,
2. (maybe I'm wrong at all whit this approach) read the local file as an ArrayBuffer, then create a Blob URL from that, following these steps:

Quote

Blob URLs would do the job, but apparently createObjectURL doesn't work on the Cordova plugin's files - I think they are some kind of fake file object which can just be used in specific ways. Data URIs could work, but they tend to be inefficient (string parsing to load an image?!) and bloat the data size. However... we can read a local file as an ArrayBuffer, then create a real Blob from that!

function fetchLocalFileViaCordovaAsArrayBuffer(filename, successCallback, errorCallback)
{
	fetchLocalFileViaCordova(filename, function (file)
	{
		var reader = new FileReader();
		
		reader.onload = function (e)
		{
			successCallback(e.target.result);
		};
		
		reader.readAsArrayBuffer(file);
	}, errorCallback);
};

function fetchLocalFileViaCordovaAsURL(filename, successCallback, errorCallback)
{
	// Convert fake Cordova file to a real Blob object, which we can create a URL to.
	fetchLocalFileViaCordovaAsArrayBuffer(filename, function (arrayBuffer)
	{
		var blob = new Blob([arrayBuffer]);
		var url = URL.createObjectURL(blob);
		successCallback(url);
	}, errorCallback);
};

Yes, this probably involves copying the data. But it stays binary, and we can create a real blob URL to it.

Do you think this last focus of the problem deserves to be evaluated?

If so, what lines of BJS code must I investigate?

Best regards.

[Guest]

This should work:) I will make it work else

[paleRider]

Hi again Deltakosh:

I'm here again after a break, in order to (hopefully) finish this development.

Retaking the conversation where we let it, we had a DDS texture that Cordova-iOS implementation refused to load by means of BABYLON.CubeTexture.CreateFromPrefilteredData BJS API, because "...WKWebView appears to treat local files as if they came from a remote server, even though they're in the app itself, and such requests are blocked." (taken from Construct Engine / Ashley's Blog / https://www.construct.net/es/blogs/ashleys-blog-2/hacking-something-useful-out-of-wkwebview-932)

From here, following the Ashley's article, I have to implement the following functionality:

function fetchLocalFileViaCordova(filename, successCallback, errorCallback){
  var path = cordova.file.applicationDirectory + "www/" + filename;
  window.resolveLocalFileSystemURL(path, function (entry){
    entry.file(successCallback, errorCallback);
  }, errorCallback);
};

function fetchLocalFileViaCordovaAsArrayBuffer(filename, successCallback, errorCallback){
  fetchLocalFileViaCordova(filename, function (file){
    var reader = new FileReader();
	reader.onload = function (e){
	  successCallback(e.target.result);
	};
	reader.readAsArrayBuffer(file);
  }, errorCallback);
};

function fetchLocalFileViaCordovaAsURL(filename, successCallback, errorCallback){
  //convert fake Cordova file to a real Blob object, which we can create a URL to
  fetchLocalFileViaCordovaAsArrayBuffer(filename, function (arrayBuffer){
    var blob = new Blob([arrayBuffer]);
	var url = URL.createObjectURL(blob);
	successCallback(url);
  }, errorCallback);
};

So I must load my DDS file with this code:

fetchLocalFileViaCordovaAsURL("assets/textures/environment.dds",function(link){
    alert(link);
    texEnvironment=BABYLON.CubeTexture.CreateFromPrefilteredData(link,_Scene);
  },function(){
    alert("Error");
  }
);

Sadly, although the fetchLocalFileViaCordovaAsURL has success in generating the wanted blob (as the first alert shows in my screen), it seems BABYLON.CubeTexture.CreateFromPrefilteredData refused to load the DDS as the geometry is shown in black. Please note the final statement of the (excerpted) console:

2018-04-01 21:36:27.628173+0200 GRB Technology[1019:607207] Finished load of: file:///var/containers/Bundle/Application/4A5035B4-5E5A-49ED-8FF5-526CF779DC72/GRB%20Technology.app/www/index.html
>>>> frameSizeChanged = 4096
>>>> frameSizeChanged = 4096
2018-04-01 21:37:54.532608+0200 GRB Technology[1019:607266] void SendDelegateMessage(NSInvocation *): delegate (webView:runJavaScriptAlertPanelWithMessage:initiatedByFrame:) failed to return after waiting 10 seconds. main run loop mode: kCFRunLoopDefaultMode
2018-04-01 21:37:54.557541+0200 GRB Technology[1019:607207] Error in Success callbackId: File1392284593 : Error: Cannot load cubemap because files were not defined

I suppose the BJS loading method must to be changed anyway in order to work with the blob.

Any help with this?

 

P.S.:  As a second reference on this issue, I think it could be affordable to implement here the same technique that is already applied in the BJS Sandbox. Your colleague Davrous explains it in this same web-site (http://www.html5gamedevs.com/topic/18078-load-file-without-webserver/), when he says: "The implementation is fairly simple. I'm just taking the blob via the HTML5 File API, storing the texture as blog into an array and I've slightly modified the Babylon.js loader to load the texture from my blob array rather than using XHR to load them from the server."

public static LoadImage(url: any, onLoad: (img: HTMLImageElement) => void, onError: (message?: string, exception?: any) => void, database: Nullable<Database>): HTMLImageElement {
            if (url instanceof ArrayBuffer) {
                url = Tools.EncodeArrayBufferTobase64(url);
            }

            url = Tools.CleanUrl(url);

            url = Tools.PreprocessUrl(url);

            var img = new Image();
            Tools.SetCorsBehavior(url, img);

            const loadHandler = () => {
                img.removeEventListener("load", loadHandler);
                img.removeEventListener("error", errorHandler);
                onLoad(img);
            };

            const errorHandler = (err: any) => {
                img.removeEventListener("load", loadHandler);
                img.removeEventListener("error", errorHandler);

                Tools.Error("Error while trying to load image: " + url);

                if (onError) {
                    onError("Error while trying to load image: " + url, err);
                }
            };

            img.addEventListener("load", loadHandler);
            img.addEventListener("error", errorHandler);

            var noIndexedDB = () => {
                img.src = url;
            };

            var loadFromIndexedDB = () => {
                if (database) {
                    database.loadImageFromDB(url, img);
                }
            };


            //ANY database to do!
            if (url.substr(0, 5) !== "data:" && database && database.enableTexturesOffline && Database.IsUASupportingBlobStorage) {
                database.openAsync(loadFromIndexedDB, noIndexedDB);
            }
            else {
                if (url.indexOf("file:") !== -1) {
                    var textureName = decodeURIComponent(url.substring(5).toLowerCase());
                    if (FilesInput.FilesToLoad[textureName]) {
                        try {
                            var blobURL;
                            try {
                                blobURL = URL.createObjectURL(FilesInput.FilesToLoad[textureName], { oneTimeOnly: true });
                            }
                            catch (ex) {
                                // Chrome doesn't support oneTimeOnly parameter
                                blobURL = URL.createObjectURL(FilesInput.FilesToLoad[textureName]);
                            }
                            img.src = blobURL;
                        }
                        catch (e) {
                            img.src = "";
                        }
                        return img;
                    }
                }

                noIndexedDB();
            }

            return img;
        }

 

Thank you for your time.

[paleRider]

BTW I'm trying to tackle the problem also from the Cordova side.

There're are several plugins that promises to solve XHR issues within iOS environment (so this is a really hot matter).

The case is that, for example, cordova-plugin-ionic-webview works but its performance is very poor (maybe because I'm using JQM and not Ionic). Maybe that poor performance is due to the overhead of running a local http server.

On the other hand the more "framework agnostic" and not running any http server, the Oracle's pair:  cordova-plugin-wkwebview-engine (1.1.4) +  cordova-plugin-wkwebview-file-xhr (2.1.1) seems not solving the problem and instead refuses to load the .babylon and .PNG files.

Keep on investigating!

[Guest]

The CreateFromPrefilteredData should work seamlessly with a blob url

But in your code I do not see the blob creation which should look like this:

var blob = new Blob([buffer]);

var blobURL = URL.createObjectURL(blob);

[paleRider]

Hi Deltakosh, and thank you very much for your support in this "not-really-a-BJS-issue" issue. 

If I understand, I need to implement it (more or less) this way:

var xhr=new XMLHttpRequest();
xhr.open("GET","assets/textures/environment.dds",true);
xhr.responseType="arraybuffer";
xhr.addEventListener("load", function(){
    if(xhr.status===200){
      var blob=new Blob([xhr.response],{type: "image/png"});
      var blobURL=URL.createObjURL(blob);
      texEnvironment =BABYLON.CubeTexture.CreateFromPrefilteredData(blobURL,_Scene);
    }
  },function(){
    alert("Error");
  };
);

Isn't it?

Unfortunately, xhr.status===200 is never reached.

[paleRider]

Hi there:

As using the environment in a DDS format is revealing as an excesive time consuming task, I'm decided to keep the things simple and use the good old 6-sides method for the CubeTexture/CubeMap.

Thank you very much.

[paleRider]

As an epilogue, and thinking it can be useful for somebody, I finally managed to have my DDS local file loaded by means of CreateFromPrefilteredData, on Cordova-iOS, with the help of cordova-plugin-wkwebview-file-xhr.

Best regards.