Load file without webserver

[ovmihai]

Hello everyone,

I read some threads but my problem is not clear yet. Is there any option in the latest Babylon API to load external files(models, textures, etc) without using an external web server?

 

Thank you

[MasterK]

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --disable-web-security

 

Is that what you mean?

[ovmihai]

No, sorry maybe i did not explain my problem very good. I want to push data into babylon, data already loaded somwhere. Like a texture i have in binary texture data i would like to push it into babylon. 

[iiceman]

What exactly do you want to do? Like an upload field where you select a file from you PC and the instead of uploading you use it in the local client with babylon... something like that?

 

What have you read so far? I think we had that topic for textures at least somewhere in the forum... but I can't find it at the moment and don't have time to search. I don't remember anything like this for models, but should work the same way, I guess.

 

I think it was something like that: http://www.html5rocks.com/en/tutorials/file/dndfiles/

 

Edit: think I found it: http://www.html5gamedevs.com/topic/17653-how-to-load-textures-dynamically/?hl=readasdataurl

Example: http://www.babylonjs-playground.com/#VNVOU

[davrous]

Are you trying to do something similar than the Sandbox: http://www.babylonjs.com/sandbox where you can drag'n'drop content (.babylon & textures) into it to load a scene?

 

More info here: http://doc.babylonjs.com/search?q=sandbox

 

David

[iiceman]

Is that the sandbox code? https://github.com/BabylonJS/Samples/tree/master/Sandbox It looks so short... is that really everything?

 

Can you explain how the loading of local files into the sandbox works?

[davrous]

Well the github hasn't been updated. The up to date version is on http://www.babylonjs.com/sandbox and simply view the source with your favorite browser.

 

The implementation is fairly simple. I'm just taking the blob via the HTML5 File API, storing the texture as blog into an array and I've slightly modified the Babylon.js loader to load the texture from my blob array rather than using XHR to load them from the server.

 

You'll find the main parts in our code here:

 

- https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.filesInput.ts

- https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.tools.ts -> look at LoadImage function and I'm filtering on the "file:" moniker.

[reddozen]

You know... couldn't this be expanded to make "install" packages for users? So if you dont want to keep several gigs of data in your cache and also don't want to have to download it everytime to render the scene.... the server could keep track of "install point" on the users machine and direct the data pull from there instead of relying on downloading it again.

 

Any possibility of making a dedicated load function for local files that you feed it the root folder path and file names like loadMesh? This would be insanely useful for massive projects like mine where some players may not have great internet access to stream all the content, but they could play reasonably well if they could pre-download all the raw files.

[iiceman]

That sounds like a reasonable idea. I think that could be useful indeed!

[Vousk-prod.]

In fact, the meshes and textures loading system alteady put everything in the local indexDB browser cache by default (need to add the manifest file alongside the .babylon files), so people coming back later does not need to reload everything. Is this not enough ?

[davrous]

Yes, I can't see the difference between what you're suggesting reddozen and what I've already implemented via IDB: http://doc.babylonjs.com/tutorials/07._Caching_Resources_in_IndexedDB and my blog article: http://blogs.msdn.com/b/davrous/archive/2013/09/24/using-indexeddb-to-handle-your-3d-webgl-assets-sharing-feedbacks-amp-tips-of-babylon-js.aspx

[reddozen]

Because if they clear their cache, it's all gone. In some instances where you have exceptially large files, it can be more benificial to be able to download the packages and load them locally. With that I can provide them a compressed archive option if they have a poor internet connection to be able to get the files they need instead of waiting on the browser to pull down and cache everything.

 

I think the cache is great for dynamic content that will change regularly, or quick model swapping for events, or small demo's or games, but I'm expecting to be generating several gigs worth of map file data, player character data, and monster / NPC data. I'm converting an entire MMO world (failed game that didnt leave beta) into babylon a piece at a time. The original game is close to 6GB of compressed UT2k4 files, and we'll be expanding it further than where the beta left off. That's way too much to expect a player (especially with a weaker internet conection) to wait for the cache to pick up while sitting on a black / loading screne for hours. To me, this is better suited to an "install" type process. It can still use a manifest type file itself as well to make sure that their files are in sync with the server, but this would be easier to manage with a larger project.

 

I'm not being critical of what we have, just suggesting more options to handle huge projects in the future. I hope you can understand my thought process on this.

[Vousk-prod.]

In fact, no, if people clear their cache meshes and textures are still there. The indexedDB is not browser cache, it's browser "application data cache". To empty this special cache, in most browsers you need to do more sophisticated operations than just clear cache (that's even usually a problem when developping because you are constantly in the need of emptying everything ).

 

But I understand your point reddozen, a downloadable assets package that can be addressed afterward in the BJS application. Integrate this possibility directly in the engine could be a nice feature, but it's already possible to create that for now "outside of the engine" (since it is possible to specifiy the url of every assets you want to load, it's up to the dev to create such a package downloadable and set urls accordingly).

[jacquesr]

Hi, I want to support ovmihai here a bit.

The problem we have is the following:

We have a (sub)component of a bigger project that uses babylon js to display 3D scenes. The component should not ask the server for resources itself, instead, it should provide an api to put in textures, models, etc. We are most likely going tor some kind of a theming object that contains all the data.

So, we cannot call new Texture(some url) or a loader that loads models but instead we have to put in binary image data and already loaded .babylon json objects into the component through the api using e. g. the mentioned theming container objects.

In the end, of course the subcomponent is in the same domain as the container and of course, theoretically it could call the server, but it's rather an architectural design choice to prevent that from happening.

You could now say "well, the parent component can use the loaders" but well, then the parent component becomes dependent to babylon while it shouldn't know anything about it.

I hope the problem is more clear now.

[JCPalmer]

There are multiple conversations going on in this thread.  2 thoughts:

 

Why not just make an application? The customer / user downloads it, using (XDK, CocoonJS, or BabylonHX).  All are both mobile & desktop.  An application can still get some of its stuff from net, but all the bulk was downloaded at install.

 

Or, if everything must be in a .babylon, then put everything in.  FileLoader.ts accepts inline textures.  Right now the only producer of a .babylon that packs textures in a .babylon is Blender, but if someone wants it in coming from another producer, then they should modify it to do so (PR it, if you are so inclined).

[davrous]

Interesting discussions. Let me share my thoughts:

 

1 - IndexedDB has some limitations, you're right. Some browsers have quota (like IE11 at 250 Mo) and users can clear their cache but as they can also delete files themselves on the hard drive even if it's a bit different process

2 - I'm wondering if we're not reaching the limit of the web as a media to distribute a game with such a big size. Or maybe we don't have the proper web technologies to cover that. Let's imagine you're downloading your packed assets on the hard drive using whatever technology. How do you provide it to the game? You're asking to the user to drag'n'drop it into the game or using the File API to open the files explorer? The experience is not great. You have no garantee he will provide the proper files. And you can't access directly to the hard drive from your web app for obvious security reasons.

 

So, I perfectly understand your points here, it's exactly why I've been building the database layer trying to experiment with IDB. I know this is not perfect but I'm still trying to figure out if we have real better plan than that with today's available Web APIs.

 

Apart from that, service workers could also be a better approach to cover what some of the things you'd like to do. But it's currently Chrome only.  

[reddozen]

I was invisioning using file API, and having the player tell the website what their root "installed" directory is, and go from there using php to build dynamic load files from a database. All my scenes lately are dynamic PHP. I would know the file names and where they're located. if the file isnt found, notify them to update the directory location, and if it's not found, then either tell them to reinstall, or revert to indexDB. Could even present a new download file. All that handling is my own deal to handle. All we would really need is a method to read a given file. either by selection browse or known directory / file name (if it's even possible).

 

 

If RuneScape can go full HTML5 there's no reason we can't as well.

 

 

Example:

function readSingleFile(e) {var file = e.target.files[0];if (!file) {return;}var reader = new FileReader();reader.onload = function(e) {var contents = e.target.result;displayContents(contents);};reader.readAsText(file);}function displayContents(contents) {var element = document.getElementById('file-content');element.innerHTML = contents;}document.getElementById('file-input').addEventListener('change', readSingleFile, false); <input type="file" id="file-input" /><h3>Contents of the file:</h3><pre id="file-content"></pre>

[jacquesr]

I like your conversation guys, but it went off topic. We still have a question about how we can disconnect loading data like models and textures from babylon and put it in using our own top layer api

[Vousk-prod.]

You can create meshes via .obj, .stl loaders, or by importing vertex data the way you want and injecting the data in an empty mesh. You then make materials by creating directly new babylon textures by specifying the images URL and assigning them to your materials.

[iiceman]

@jacquesr, wahts wrong with the way davrous suggested? This seems to work well with the sandbox application... or why dos that not fit your request?

 

Well the github hasn't been updated. The up to date version is on http://www.babylonjs.com/sandbox and simply view the source with your favorite browser.

 

The implementation is fairly simple. I'm just taking the blob via the HTML5 File API, storing the texture as blog into an array and I've slightly modified the Babylon.js loader to load the texture from my blob array rather than using XHR to load them from the server.

 

You'll find the main parts in our code here:

 

- https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.filesInput.ts

- https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/babylon.tools.ts -> look at LoadImage function and I'm filtering on the "file:" moniker.

[Vousk-prod.]

I'm pretty sure they don't want to have their users to drag'n drop files. They already have a data loading API and wanna create Babylon objects on top of that.

But it's completely feasible, since we can inject vertex data in a mesh, and create material and textures from scratch from images files URL.

[iiceman]

Yeah, it's not about the drag n drop. The point is storing the file as a blob thingy and using createObjectURL. That can be done completely independent from any Babylon calls. And if you need to create a mesh or a texture from a file, you just reference that ObjectURL... at least if I understood the idea right.

[jacquesr]

@jacquesr, wahts wrong with the way davrous suggested? This seems to work well with the sandbox application... or why dos that not fit your request?

 

sorry, I totally missed that! Thanks a lot