How does IR light influence camera color reproduction?
In many outdoor and global security applications it is desired to get good image performance of a camera in both daylight and at night. In such cases the camera needs to be IR sensitive to perform better at night or needs to be sensitive to a certain wavelength. However, the infrared component is not desired in daylight as it will influence color reproduction. When designing a system, you have three choices for dealing with this contradiction in the specification.
- Implement an IR-cut filter for daylight situations which mechanically slides away during night time situations.
- Implement a fixed IR-band-pass filter when mechanical movement is not desired.
- Implement no IR-cut filter when mechanical movement is not desired.
The correct IR-cut filter depends on the application and must be chosen by the system designer. Here we show the consequences on the image color when an IR-cut filter is used or when the IR-cut filter is removed.
Why IR light has influence on color perception?
For optimal performance a camera’s sensor response to light is matched with the perception of the human eye as close as possible. Figure 1 shows the spectral response of the human eye. An example of the spectral response of a color sensor can be found in Figure 2.
Figure 1: Spectral response human eye
Figure 2: Typical spectral response of a color CMOS image sensor
When comparing the two spectral responses there are clear differences. We compensate for the imperfections by post-processing the colors of the RAW sensor image with a color matrix. The color matrix thus matches the spectral responsivity of the camera with that of the human eye.
When looking at the sensor performance at >700 nm a rise of the blue and green pixels in wavelength responsiveness is seen. This increase in response for IR light makes it more difficult to render good color. The IR light that is normally invisible to the human eye will contribute to the color representation. It gets even worse when looking at organic objects as these might reflect certain IR wavelengths more than inorganic objects.
Using an IR cut filter will make the camera blind for IR light, just like our eyes. Without detecting IR light, calculating a natural color representation from the RGB signal with the color matrix will provide a much better result.
A practical example
|Image 1: Camera with IR filter||Image 2: Camera without IR filter|
When comparing Image 1 and Image 2 the benefit of an IR-cut filter becomes clear. For example, compare the colors on the color checker. Note that no camera setting alterations have been made between these two images. They both use the same white balance and color matrix. The white balance and color matrix are optimized for use with an IR-cut filter.
Because of the sensitivity to IR, the camera makes that IR light visible as color. Especially in the range between ~700 nm and ~850 nm the difference in response to each color mixes up the colors considerably. It is not possible to compensate for this in camera processing. There is no way for the camera to tell how many photons of each wavelength are converted to electrons in the pixel as the spectral response of the objects reflecting the light is unknown.
When taking a closer look at the vegetation in Image 3 and Image 4 the hue shift towards red can be seen very clearly. It is interesting to see that green is still visible in the color checker. This is a result from the different IR response of the color checker material as compared to the grass. The vegetation can absorb and emit IR much more as compared to the inorganic plastic color checker.
|Image 3: Camera with IR filter||Image 4: Camera without IR filter|
Image 1 and Image 2 show that the camera color performance degrades significantly after removing the IR filter. It is possible to tune the color processing algorithms to better perform under these conditions, but it is important to keep in mind that objects will have a different color response due to the IR light. Furthermore, tuning of the algorithms will introduce noise into the image.
Two algorithms can be tuned when changing the IR-cut filter. These are the white balance algorithm (1) and the color matrix (2).
- Image 5 and Image 6 show the performance difference between no optimization and a white balance optimization. The difference is very subtle but when looking at the white building in the back, white in Image 6 is a little whiter as opposed to in Image 5 where it is a little redder.
- Image 7 and Image 8 show the performance difference between no optimization and a white balance and color matrix optimization. For the color matrix optimization it is best to look at the color checker in the front. The optimized matrix pushes the blue and red patches more towards the color that we see with our eyes. A big downside is a significant increase in color noise as the matrix values are very extreme and a lot of color gain is used to compensate for the IR response. For a zoom-in on the color checker see Image 9 and Image 10. In this zoom-in you can observe the additional noise if you look carefully.
|Image 5: Non-optimized settings for camera without IR-cut filter||Image 6: Optimized white balance for camera without IR-cut filter|
|Image 7: Non-optimized settings for camera without IR-cut filter||Image 8: Fully optimized color processing for camera without IR-cut filter|
|Image 9: Optimized white balance for camera without IR filter||Image 10: Optimized white balance and color matrix for camera without IR filter|
Adimec has the ability to re-calculate the white balance and color matrix parameters to optimize performance for Adimec designed cameras with your specific optical path. To do this it is important to have a detailed spreadsheet of the spectral response curve of the complete optical path.
Color perception by the human eye and a typical camera sensor are different and the ways IR light is handled is also not the same. With a camera, infrared light shows up in the color matrix perverting the color representation without an IR filter. There are trade-offs for optimizing the camera settings to get a better color image and reduce the impact of infrared light on the color reproduction capability of a camera. As discussed, there are means that can minimize the effect of this IR contamination via reoptimizing the white balance and the color matrix and therefore both take advantage of the IR light and maintain most of the visual color fidelity.
Contact your Adimec representative for more information on this topic.