KHR_materials_transmission

[MiiBond]

Here's a quick draft of a possible KHR_materials_transmission to handle thin-surface optical transparency.

There are some open questions, especially around how we define refraction. A thin surface is, of course, not physically correct and, without thickness, really shouldn't refract light. However, it seems reasonable to expect micro-facets in a rough surface to refract light and blur the transmitted light. So micro-facet refractions but not large-scale refraction?
Also, in real-time implementations, how do we propose recommended ways to simulate this blurred refraction?

[proog128]

A thin surface is, of course, not physically correct and, without thickness, really shouldn't refract light. However, it seems reasonable to expect micro-facets in a rough surface to refract light and blur the transmitted light. So micro-facet refractions but not large-scale refraction?

Maybe this extensions should be called KHR_materials_thin_transmission to make clear that it assumes a thin surface. As a follow-up, we could add a KHR_materials_volumetric_transmission extension that defines refraction for closed meshes, including parameters for absorption and scattering.

[proog128]

Do you consider the effect of the index of refraction on thin refraction? E.g., as the index of refraction approaches 1, does the blur effect disappear? Or should the glTF spec leave this up to implementations? But what is the reference to test implementations against?

[MiiBond]

Yes, I think IOR should affect the amount of blur but I wasn't sure how to define anything mathematically yet. How much does an infinitely-thin surface refract light? Well, none...

There would be a (1-F) affect from light hitting the micro-facets, I guess, but no refraction.

Unless I'm not thinking about this correctly.

[MiiBond]

Am I right in assuming that we do want a refractive blurring effect, even with an infinitely thin surface?
What do offline renderers do in this case? That should be our target reference.

For real-time, we can provide a recommended implementation but I think we need a higher-quality target so that real-time renderers are free to set their sights higher.

[proog128]

How much does an infinitely-thin surface refract light?

If I am not mistaken, it depends on the orientation of the microfacets on the two sides of the object enclosing the infinitesimal small volume. If the facets on both sides are aligned such that the light ray entering the thin "volume" exits through a facet that has exactly the same orientation, there is no refraction. On the other hand, if the orientation is different, refraction will occur (if IOR != 1). This can be approximated with a microfacet lobe that is centered around the reflection direction flipped to the lower hemisphere wrt. the macro-surface normal. Here is an example how it is implemented in MDL for offline rendering: libbsdf, line 514. Another good source regarding offline rendering is the Thin Surface BSDF from Imageworks (slide 34). They also show comparisons between actual modelled thin surfaces and their thin BTDF, and derive a roughness mapping that includes the IOR.

[MiiBond]

Thank you. I've modified the spec to reflect this.

[MiiBond]

I have a note here about modeling refractive blur in rasterizers because I'm looking for input on how to define a recommended implementation:

This is fine for raytracers or for rasterizers that just sample the existing IBL for refracted light. However, what about rasterizers that want to cheaply render transmission in screen-space (I think this is pretty common)? Typically, the refractive blurring is done by sampling mips or manually-blurred versions of the background scene. This can be eye-balled by an implementation but is there a slightly more mathematically rigorous version that we could recommend?

I currently am using log2(roughness² * scene_texture_resolution) to get the LOD to sample and it gives okay results. However, this is based on the calculation for sampling from a prefiltered IBL in Babylon. It assumes that there's a correlation between the prefiltered IBL mips and the screen-space RT mips, which isn't really the case.
https://adobe.ly/2PzbKuM

[cesss]

I'm in real need of having transparency with refraction supported in GLTF. However, GLTF has focused in PBR materials (which I consider a very good choice, BTW), so of course I feel this contradiction: what I need is thin-surface transparency, but I want to work with PBR materials definitions.

So, just wondering (and hoping it's not a dumb question), would it be possible to implement a physically valid transmission model, as an extension to PBR materials, but having some parameters that let you adjust the physically-valid model so that it behaves as thin-surface even if there are no real materials with that behaviour in nature?

My contradiction is of course that I'd like to use the refraction model from the 80s raytracers but at the same time keeping everything into a PBR material definition rather than having one definition for PBR materials and another definition for non-PBR materials.

[MiiBond]

If I understand you correctly, you’re basically just asking to have refractions supported in this extension?
I’ve considered this but I think it belongs in the KHR_materials_volume_transmission extension (or whatever we call it). I’ve been held up with other things lately that have prevented me from finishing the first draft of that extension but hope to get back to it soon.

[cesss]

Well, I was not thinking in what part belongs to each extension, but in being able to encode (in glTF files) a PBR-correct BSDF (BRDF together with BTDF --and BTDF implies refraction). In other words, to encode (in glTF) a parametric definition for physically-correct BSDFs.

However, because for some applications I'd like to render transmission using the old-fashioned transparency+refraction model as implemented in older raytracers, this creates a sort of contradiction to me: To use a different BSDF for encoding materials that shall be rendered with PBR raytracers and another non-physically-correct BSDF when I'll be rendering with older raytracers. Or (much more preferably) to use always the same BSDF encoding (a BSDF that somehow lets you define old-fashioned transparency+refraction together with physically-correct BTDF, just depending on the way you tune the parameters). I prefer this approach to having two different BSDF models.

Now, this can be implemented in one extension (i.e: KHR_PBR_BSDF), in two extensions (i.e: KHR_PBR_BRDF and KHR_PBR_BTDF), or in a large number of extensions that, when used together, let you define a BSDF. But I wasn't thinking in the extensions (I tend to believe that the lesser extensions, the better, but that's just my opinion).

[proog128]

To use a different BSDF for encoding materials that shall be rendered with PBR raytracers and another non-physically-correct BSDF when I'll be rendering with older raytracers.

Do you have an example implementation for the non-physically-correct BSDF? What makes it non-physical? glTF provides non-physical transparency as alpha coverage with different blend modes. This can be combined with both thin optical transparency (this extension) or volumetric/refractive optical transparency (a potential future extension).

I’ve considered this but I think it belongs in the KHR_materials_volume_transmission extension (or whatever we call it). I’ve been held up with other things lately that have prevented me from finishing the first draft of that extension but hope to get back to it soon.

@MiiBond I wasn't aware that you are working on a volume extension (this one?) and also started to work on a draft. In case you are interested, you can have a look here. I was planning to create a pull request in the next days, but now I am not sure how to proceed. Two separate pull requests? Or one pull request on which we can work together?

[MiiBond]

@proog128, yes, that's the one (though I hadn't pushed changes from last month yet). Yours is far more detailed so I think we should just continue editing off of that. We chose a different parameterization but I don't have strong feelings either way about that.

I do think we do need to consider a couple of issues for realtime rendering (since that's the primary market of glTF). For example, I think we should allow for representing volume on non-closed meshes with a thickness texture (with min and max values as other params). But this conversation can wait until you open a PR.

[proog128]

@MiiBond OK, I will create a pull request for the extension. I like the detailed explanations in your proposal, we should keep them somehow.

I don't have strong feelings for the parameterization either. In the Enterprise PBR material, we chose parameters that are more user-friendly (attenuation (extinction) color and distance, multi-scattering albedo), but for the extension I thought something more "basic", closer to quantities used in physics, is better for compatibility with other material models (Standard Surface, MDL, ...). In the end there is a lossless conversion between them anyway. At least if the conversion is computed on spectral color values (RGB triples are problematic). For this reason, I also thought about allowing different parameterization, so that renderers working on spectral data can do the conversion with spectrums. Let's discuss this also in the pull request.

[jackcaron]

Could it be a single channel selected for this? If it's red, we could pair it with the metallic roughness texture.
Or better, be able to select the channel to read from.

On the other hand, could go with an RGB texture and allow for transmit at different level each color to a create prism effect (assuming the ior property is part of the specs).

[MiiBond]

The red channel of the metallic-roughness texture can already be used for ambient occlusion.
However, I had thought that we could share this texture with the KHR_materials_volume (#1726) extension which could use one of the other channels for thickness information (if we go the route of allowing the thickness param).

[MiiBond]

Dispersion (the prism effect) is specifically not part of this extension to keep things simple. It could certainly be included in another extension though.

[donmccurdy]

I agree with @jackcaron that we should pick a single channel, for two reasons:

1. By requiring that three channels be filled with the same value, we are losing any packing flexibility that might (intentionally or not) have been available. The fact that the r channel in the ORM texture could be used for either occlusion or transmission is a good thing.
2. If in the future we have a general-purpose swizzle extension, it will be awkward that it's required to provide a 3-channel output for a 1-channel value here.

Either r or a seem like reasonable choices, to me.

[echadwick-artist]

I would propose this extension use the r channel of a texture separate from the ORM, as I would not want to have to omit occlusion in order to take advantage of per-pixel transmission. Pre-calculated ambient occlusion has a strong visual benefit.

Perhaps we could use a channel of another texture, and reserve the green and blue channels for other PBR Next features? Transmission (r), clearcoatTexture (g), and clearcoatRoughnessTexture (b) for example.

[donmccurdy]

We don't need to choose a specific texture to pair transmission with — as long as the channels don't conflict, users can mix and match. Any of (transmission; occlusion/rough/metal), (occlusion; transmission/rough/metal), (transmission; occlusion; ____/rough/metal) would be allowed.

I would be tempted to try choosing volume texture channels that complement transmission, but I'm not overly concerned about fitting into empty channels of orthogonal features like ORM or clearcoat.

[MiiBond]

We've been discussing using the 'G' channel for thickness over in the KHR_materials_volume thread.

[rsahlin]

I agree that the texture should be one channel (prism not being part of this extension).
As for which channel to choose I would propose either R or A.
Picking A would make it easy to have occlusion/roughness/metallic/transmission (without swizzle)
Picking R could have benefit if occlusion is less likely to be used with this extension.

Either way is fine to me, I will solve by internally swizzling the textures if needed.

[rsahlin]

I think this extension is well thought out and would probably suit the needs of Ikea very well - thanks for all your work!

I wonder why the IOR is not included in this extension?
My assumption would be that as soon as you have transmission (light entering the surface) the light would refract due to change in the medium?
With that said I would think it makes sense to either include IOR in this extension or declare dependency to #1718

[MiiBond]

Should this spec have an "exclusions" section, similar the clearcoat extension?

Yes. Done

[emackey]

Actually can I propose a bit of a rewrite? I'm concerned we've buried the lede here.

Suggested change

	## Tinting
	You may notice that the transparent materials shown above are tinted. i.e. they aren't actually transmitting 100% of the non-reflected light. This is because `KHR_materials_transmission` specifies that the `baseColor` of the material be used to model the absorption of light by the surface. This allows materials like stained glass to be easily represented with this extension.
	Absorption is usually defined as an amount of light at each frequency that is absorbed over a given distance through a medium (usually described by Beer’s Law). However, since this extension deals exclusively with infinitely thin surfaces, we can treat absorption as a constant. In fact, rather than absorbed light, we can talk about its inverse: transmitted light. The `baseColor` of the material serves this purpose as it already defines how the light that penetrates the surface is colored by the material. In this model, the transmitted light will be modulated by this color as it passes through. Note that because `baseColor` is defined as sRGB, it must be converted to linear before being used in this calculation.
	## Tinting
	The `baseColor` of the material, as defined in the [Metallic-Roughness Material](https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#metallic-roughness-material)
	section of the glTF specification, defines the amount of light at each frequency
	that is transmitted (not absorbed) by the material. Absorption is usually defined
	as an amount of light at each frequency that is absorbed over a given distance
	through a medium (usually described by Beer’s Law). However, since this
	extension deals exclusively with infinitely thin surfaces, we can treat
	absorption as a constant. Therefore, its inverse, `baseColor` transmission,
	can also be constant. This provides the color tinting effect in the materials
	shown above.

[MiiBond]

Yes, I like this. Lead with the important bits :)

[donmccurdy]

I like Ed's the rephrasing, but two nits — in the last two sentences could you replace the words "its" and "This" with the terms they refer to? I had a bit of trouble following the pronouns at the end.

[MiiBond]

I rephrased this a bit. Maybe it's clearer now?

[proog128]

Isn't this missing the G term and the normalization, like in f_specular?

f_transmission = (1-F) * T * baseColor * D_T * G_T / (4 * abs(dot(N, L)) * abs(dot(N, V)))

[MiiBond]

Yes, quite right. Thank you.

[emackey]

@MiiBond Any chance these small fixes can be done in time for Monday? 👼 Let us know if you want patch contributions for any of them, thanks!