真实光照度

[xiaosanxiao13]

我一直有个问题，那就是数据集中的图像的真实照度究竟是如何获取的，你能解答下吗

[nbanic]

The ground-truth illumination for the images in the dataset has been obtained by placing a calibration object, namely a Spyder cube inside every image as can be seen. Since the cube faces have a color that is known exactly, any deviation from it in terms of the chromaticity can be directly attributed to effects such as illumination. More detail is available in the paper: https://ieeexplore.ieee.org/document/9296220

[xiaosanxiao13]

谢谢老哥，这个我知道，但我想知道的是具体的操作，我看论文上好像写的不是很清楚，我也看过其他的数据集的论文，比如Intel-TAU,它说的是，利用MCC的第#20-#23的灰色块来进行操作，但具体，我就不知道怎么操作了，我在网上也找了很久，没什么信息。
对了，你也是研究颜色恒常性的吗，我是研一的学生，今年刚开始做这个方向，导师不是很明白，基本都要我自己来，所以，如果可以的话，能不能给个联系方式，希望有时能请教一下，多谢老哥了

[nbanic]

The specific operation is to focus on the gray face(s) of the calibration object. If the illumination were pure white, than the R, G, and B components of the gray face pixels would all be the same, since this is the very definition of gray. However, if, e.g., the R component has a somewhat higher value, this can usually be direct attributed to the illumination, which means that the scene illumination is somewhat reddish. In short: the color of the gray face is practically the color of the illumination since the gray face directly reflects the illumination evenly among all the color components. This is why it is used to extract the ground-truth.

Yes, I studied it and I would say I understand it. Was the explanation above OK or do you need some additional clarifications?

[xiaosanxiao13]

十分感谢你的回答，关于上面，我有一点困惑，In short: the color of the gray face is practically the color of the illumination since the gray face directly reflects the illumination evenly among all the color components。这句话不是十分的理解，并且，我想知道，如果，我有一张带有校准对象(24色卡或者SpyderCube)的图像，我该如何获得它的真实光照都呢，这是我最关心的问题。

[nbanic]

If you are interested in the chromaticity of the illumination, and for the ground-truth illumination this is its most important information, then it is enough to simply take the chromaticity of any surface that is known to be gray under the white illumination. In the case of a color card, you take the gray fields, i.e., you take their color and calculate the chromaticity from them. This chromaticity is then the same as the chromaticity of the ground-truth illumination that illuminates the scene.

[xiaosanxiao13]

抱歉没有及时回复，谢谢你的耐心解答，但我明白这个道理，我想知道的是，得到ground-truth illumination 的具体过程(可以以24色卡为例子)，如果你觉得表述麻烦的话，也可以给相关的论文或者网站，真的十分感谢您的帮助，朋友

[nbanic]

1. Put the calibration object (in your case, a color card) into the scene.
2. Take a photo of the scene.
3. Use the raw image of the taken photo.
4. Take the pixels of any gray color card.
5. Calculate the chromaticity of the mean color of these pixels and use it as the ground-truth.

I guess that there may be some misunderstanding with some of these steps. Which one would you like me to explain in more detail?

[xiaosanxiao13]

抱歉，昨天我看到的时候已经很晚了，所以没能及时回答。对于您的慷慨的耐心回答，我十分感激。对于上述回答，我有几个问题。首先关于第四点，可以取任意的灰色色卡吗？我看Intel-TAU用的是#20-#23,因为它认为#19太亮，#24太暗，所以排除。如果取任意的位置的灰色色卡像素计算会不会不准确。关于5，你是指将选择的灰色像素的R，G，B相加再除以他们的数量，来得到最终的真实照度吗。比如Intel-TAU用的是#20-#23，将20-23的R，G，B分别相加，再除以4吗。不知道我的理解正确吗？期待您的回复

[nbanic]

Regarding 4), you take any gray card that is suitable with respect to the equipment that you are using. The Inter-TAU makes a good choice. Namely, if the gray card is too dark, then noise has too much influence. On the other hand, if the gray card is too bright, then there may be too many clipped pixels. In that sense, it is good to avoid a too bright or too dark card as you also mentioned. Regarding 5), you choose a card, e.g., #22. Then you calculate its mean color in the RGB space. The mean color will obviously also be a color and since it has three components, it will be a vector. Now, the direction of this vector is the same as the direction of the scene illumination. The amplitude is obviously probably not the same, but that doesn't matter. What matters is the direction. Because of that, you can take the chromaticity of this mean color (regard it as a normalized color) and use it as the ground-truth illumination color. How does that look?

[xiaosanxiao13]

我想我大概明白你的意思了，我看了一些论文，它们也强调，可以将光照度分为亮度和色度，也即大小和方向，关注的是色度，也就是方向，到时候设计模型得出的最初的结果也是色度。按照您的解释，可以任选一个合适的灰色区域，比如#22，然后将包含这张色卡的图像放到软件操作，可以得到它的RGB值，从而可以得到它的色度值，从而得到一个方向，而这个色度值就是我需要的ground-truth。我的理解正确吗

[nbanic]

Yes, it is something like that. When you create the ground-truth, you usually segment the gray card by hand. The reason that the direction of the vector is the only important thing can also be seen from the fact that the main error metric is the angle between the color vector that a method gives and the color vector of the ground-truth illumination.

[xiaosanxiao13]

哦，十分感谢，对了，还有个问题就是，我对包含色卡的场景拍摄后，是不是存在专业的软件去处理这张图片，从而可以得到我想要的信息，比如，灰色区域的RGB值，因为我还没有实际操作过，只是从论文上大概了解，对此十分的模糊

[nbanic]

To the best of my knowledge, there isn't a widely known software that would do that for you. In most cases when you have a situation like this, you usually have to deal with only a single scene, or with only a limited number of them. This means that it should not be a problem to manually segment the color card.

When the Cube++ dataset was prepared, it was somewhat easier because the SpyderCube was always at the roughly same location, so it was possible to extract the face colors by knowing the approximate location of the face and using padding that was wide enough. I would say that the answer about automation mostly depends on your use-case scenario. If you have to do it only a single time for a single image, maybe Matlab with its drawpolyline command is a good choice.

[xiaosanxiao13]

好的，我大概明白你的意思了，对于色卡来说，我应该尽量保证它们位于同一位置，并对同一个灰色区域进行处理，以此来批量的得到图像的真实照度。我可以将它们输入python里，然后用opencv库来处理吗，即算出灰色区域所有像素的RGB，然后取平均，以此作为真实照度。你看如何，对了，关于这方面，有相关的论文或者书籍可以推荐的吗，不胜感激！对了，我还有个不情之请，我导师最近给了我一个任务，目前的第一步，我需要去采集人的舌头的图像，为了方便后续的对舌头做的颜色恒常性的研究，您对如何采集舌头图像，有什么见解或建议吗？我的想法是，仿照之前的几个知名的数据集，在拍摄视角内，摆放上色卡或者SpyderCube进行图片的拍摄，不用在意当时的环境光照，因为有色卡，然后，通过色卡，我可以得到所有图像的真实照度。这样就得到了一个和之前的知名的数据集相似的数据集。您看怎么样，十分感谢您的回答，十分期待您的回复！

[xiaosanxiao13]

对了，刚才看到论文，不是很理解，请问这个achromatic pixels是什么像素啊？

[nbanic]

Yes, anything that helps should be OK. OpenCV is definitely a good choice, I also used it for these tasks.

As for the tongue images, I did not have any direct experience with that. I guess that you could find some small gray card (maybe on a stick?) and use it while taking numerous such images on volunteers.

[nbanic]

Achromatic pixels are basically gray pixels, i.e., pixels without a color. Strictly speaking, you could argue that gray is also a color, but in the literature achromatic is usually used for gray. It is an achromatic color card that you are interested in here.

[xiaosanxiao13]

您是指灰色像素正是色卡上用来得出真实照度的那几种区域的颜色吗？对于舌头的问题，我的导师希望，我们能对普通的舌头图像进行颜色恒常性计算从而使它成为在标准白光下的图像，目前导师的想法是，搭建一个标准白光环境，在这个环境下拍摄图像，从而直接得到标准白光下的舌头图像，他们将作为数据集的一部分(可以看作真实值)，然后通过对这些标准白光的图像做各种变化从而得到训练集，从而得到一个完整的数据集，即数据和标签。您觉得怎么样呢，关于这个想法。

[nbanic]

I think the order of defining it should be reversed: the pixels used to determine the ground-truth illumination are supposed to be gray, i.e., achromatic.

As for the tongue images, any collected tongue images with known scene ground-truth illumination are good.