Creating HDR cubemap with probe

[Rodrix3]

I see all demos have environment.dds.

Now that I have my final scene (which is a close doors image that does not require a skybox) I would like to render a cubemap out of the scene.

The question is: is it possible to render an HDR cubemap so I can use it as reflection texture for materials? If so, how?

 

Thanks!

 

p.s: manually setting the reflection texture overrides the general environment texture right?

[dbawel]

@Rodrix3

You can assign practically any reflection texture using various modes (only a couple to choose from currently) but it can be endless if you get creative with your alpha channels, gradient textures, emmisive textures, etc. An example might be: skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;

 

Here are some docs that should help with a couple basic but cool PG examples:

https://doc.babylonjs.com/how_to/reflect

 

If you need more info than this, just chime back in. There are also a couple of converters available - I used to use 'one released by '80 level' - it was worth the $25 cost at the time:

https://80.lv/articles/cubemap-converter-high-quality-hdr-images-at-lowcost/

 

Hope this helps,

DB

[Rodrix3]

Hi @dbawel!

Thanks so much for the reply.

 

But to be honest I am now more confused after reading Babylon documentation you provided.

 

It says:

>High Dynamic Range (HDR) images are panoramic images that cover an entire field of vision.

 

This is wrong right? An HDR image is an image of high dynamic range which means it has different RGB values for each pixel according to different levels of brightness. The fact that it covers an entire field of vision talks about the projection used (Spherical, cubic, equirectangular).

 

This means we could have an HDR equirectangular image (as shown on documentation) but also an HDR spherical or cubemap texture.

 

Therefore.. I am quite sure the documentation is wrong. Am I mistaken?

 

Second the tool you mentioned is for processing external images. Instead I want to generate an image using probes, and capture it in HDR format.

 

Is this possible?

Am I right... Or am I confused in some way?

 

Thanks so much in advance!

[Nabroski]

Hello

painless and fast forward, without going in detail. My generation, people in the 30ties and older, get used to watch RGB(A) 8bit Images. Unfortunately this is only a portion of what human eyes is capable of perceive, and how real world object interact in light when it comes to calculations on a math based understanding. That the simplify version, don't want to write an essay and also not saying something wrong. So to pack more information into images is where the High Dynamic Range as scientific field of research originated. Nowadays low cost HDR Monitors are sold on consumer market and also Cameras going digitally with better chips build in to process information. It is not only about the Image, is about the hole chain of technology needed to produce and receive such images. Camera with Super Wide Lenses with a WIFI build in, you can preview the picture you about to take on your Smartphone, very convenient. thouse are only base markers it this procedure. As i said, hole topic on this own, read that stuff up online. Psychically Based Rendering collect techniques that are matches that of the physical world. With this in mind you can(should) go and what a tutorial on YouTube about (any) Photography and how to take a HDR shoot and replicate you 3D Scene for a real physically based experience.

 

• How to:
• place a camera in the middle of you World Scene.
• rotate it 90 degrees:
  • Make screenshoot:
    • 4 shoots in four directions ( X, Y, Z, -Z )
    • 1 shoot up
    • 1 down.
• repeat the process 3 this in your scene
  • 1 time underexposed ( going for shadows )
  • 1 normal lit scene and
  • 1 overexposed ( highlights )
• process the 6 separate LDR images to 3 Cubemaps
• pack them as one HDR Cubemap
• you done

 

Their are tones of software online: To process LDR (3 Cubemaps) images and pack them as HDR also to create Cubemaps and DDS files.

Sounds scary at first. Never done it on a Babylonjs 3D Scene, so take this advice as the rough theory, for what I read and know so far.

 

So you the Pioneer. Congratulations.

 

[brianzinn]

I just want to explain further the HDR concept.  I used to do a lot of amateur HDR photography, so I'l take a risk to explain the basics and then back to how maybe we can do this in BabylonJS.

As we move our eyes from a shaded portion of a scene to a bright sky, our eyes will adjust automatically as we transition. In the shaded area we would see grass, if it were there, and blue skies on a sunny day.  A typical point-and-shoot camera from a few years ago would get one or the other (blue sky and black grass, or green grass and white sky) - typically we could choose which one mattered more by adjusting the exposure setting.  Simplistically, our eyes can see both because as we transition from one part of the scene to the other we make adjustments - ie: change pupil size for amount of light entering.  So, for a camera the easiest way is actually to let in the same amount of light in (changing the aperture on a lens would adjust light entering, but usually aperture priority is used and it is locked) then you just need to change the shutter speed.  If you have a DSLR there will be a multi-bracketing setting - it will have settings like 3,5,7.  The other major thing you can control is ISO, but I want to keep to 2 main concepts.  Aperture and shutter speed typically capture 3 images and then software is used afterwards to make a single image.  If you are still getting dark or bright sections then you have an image with really high dynamic range then you would maybe use the 5 or 7 in multi-bracketing - not important for what we are doing.

The reason that I went to all the gory details, is that it looks to me like you want to create an HDR cubemap of an existing BabylonJS scene.  I think the HDR cubemap is rooted in real photo usage.  I don't see how you can go from a typical screenshot to the same screenshots with different exposures (and I don't think that is possible).  If we, however, used post process and createRenderTargets with adjusted settings like shutter speed or aperture then we should be able to generate the different images!  We are really lucky as well, because we can freeze the scene - so, we don't need to worry about ghosting or tripod moving - like in real life

Now to see if there are post-process that do that - or am I wrong about what you are after?

 

[dbawel]

@Rodrix3

18 hours ago, Rodrix3 said:

This means we could have an HDR equirectangular image (as shown on documentation) but also an HDR spherical or cubemap texture.

Therefore.. I am quite sure the documentation is wrong. Am I mistaken?

No, you are not mistaken. You're not constrained to using cube maps within babylon.js, as there are other modes available such as planar and spherical. I haven't experimented with anything other than cubemap textures, but you are able to accomplish what you seek to do if I understand your original post correctly.

And as @brianzinn states:

7 hours ago, brianzinn said:

As we move our eyes from a shaded portion of a scene to a bright sky, our eyes will adjust automatically as we transition. In the shaded area we would see grass, if it were there, and blue skies on a sunny day.  A typical point-and-shoot camera from a few years ago would get one or the other (blue sky and black grass, or green grass and white sky) - typically we could choose which one mattered more by adjusting the exposure setting.  Simplistically, our eyes can see both because as we transition from one part of the scene to the other we make adjustments - ie: change pupil size for amount of light entering.  So, for a camera the easiest way is actually to let in the same amount of light in (changing the aperture on a lens would adjust light entering, but usually aperture priority is used and it is locked) then you just need to change the shutter speed.  If you have a DSLR there will be a multi-bracketing setting - it will have settings like 3,5,7.  The other major thing you can control is ISO, but I want to keep to 2 main concepts.  Aperture and shutter speed typically capture 3 images and then software is used afterwards to make a single image.  If you are still getting dark or bright sections then you have an image with really high dynamic range then you would maybe use the 5 or 7 in multi-bracketing - not important for what we are doing.

The reason that I went to all the gory details, is that it looks to me like you want to create an HDR cubemap of an existing BabylonJS scene.  I think the HDR cubemap is rooted in real photo usage.  I don't see how you can go from a typical screenshot to the same screenshots with different exposures (and I don't think that is possible).  If we, however, used post process and createRenderTargets with adjusted settings like shutter speed or aperture then we should be able to generate the different images!  We are really lucky as well, because we can freeze the scene - so, we don't need to worry about ghosting or tripod moving - like in real life 

The use of HDR textures in babylon.js and other applications is to try and replicate how natural light is projected and reflected from the light source onto the object and then registered on the conal arrays within the biological eyes. Although it's only a simulation currently, within 2 years, new projection systems are hitting the consumer market (not pro-sumer) which will be rendering natural light for AR displays; and this will change everything... including the way we render scenes and objects within the scenes to provide a real physical model of natural light and how it is registered in the human eye. So in my opinion it's good you're looking at these next steps now - even though they aren't new to many professional imagery tools and cameras. They're simply due to hit the consumer market sooner than later. thant's when everything changes; including the way we watch media (film, broadcast), as well as experience real-world elements; especially as GPUs become fast enough to handle the computation necessary to bring these new rendering methods to life in real-time.

But I'm stepping a bit outside of what you want to accomplish... which I'm certain you can do now; however as mentioned by Mr. Zinn, you may require a post process shader to generate the specific results you seek. Regardless, this all becomes the norm in just a short time. There is already media in production which takes advantage of these new displays and render methods which will have a HUgE impact on our and most every media industry. But your original question was? Ask specifically with a scenario, and this community will jump at the opportunity to assist... at least I believe they will have a great level of interest.

DB

[Guest]

Quote

p.s: manually setting the reflection texture overrides the general environment texture right?

Yes

 

So far the way to generate dynamic cube textures is to use the ReflectionProbe which uses a uint texture to store data so no HDR (which requires float)

BUT it is really easy to add a parameter to the constructor to switch to float here: https://github.com/BabylonJS/Babylon.js/blob/master/src/Probes/babylon.reflectionProbe.ts#L19

[Rodrix3]

AMAZING replies.

Thanks so much everyone for going in depth. I really love this community.

@Deltakosh got my idea precisely of why I want this.

I want to make the reflection probe capture HDR, that is a texture with additional information than just RGB. The reason I want an HDR cubemap (and not just a standard cubemap), is because I see that in all the demos that use a environmental mapping and PBR, an HDR texture is used. So I am assuming (which seems to be right) that PBR works much better with an HDR texture and gives more realistic reflections for all different lighting environments. This is why I want to capture an HDR texture in the first place.

In the second place, I want to write a small utility library that will let us (babyloners) add real reflections to all our scene with almost no setup, and without depending on external HDR textures which are hard to find, and also 0make little sense in indoor environments. There is a new feature from Babylon 3.2 which is called local reflections, which I am testing, and building into a library.

The first proof of concept is here:

...although it still needs much more work.

@Deltakosh, could we go ahead and add a new parameter to the constructor of ReflectionProbe as hdr=false so that if true, it would save an hdr texture (if I understood well).
 In this way I could add to my previous code that flag, so the probes capture in HDR (did I get all this right?).

On 7/23/2018 at 12:17 PM, Deltakosh said:

So far the way to generate dynamic cube textures is to use the ReflectionProbe which uses a uint texture to store data so no HDR (which requires float)

BUT it is really easy to add a parameter to the constructor to switch to float here: https://github.com/BabylonJS/Babylon.js/blob/master/src/Probes/babylon.reflectionProbe.ts#L19

 

I hope you guys @dbawel, @brianzinn, @Nabroski, can take a look at my code on the other thread and make any suggestions / contributions to see if we could come up to create something useful and great!

Thanks everyone again

[Guest]

HDR is used to store more than 255 values per color component. I'll add the flag to Reflectionprobe for you in next commit