The Achilles heel of modern camera sensors based on the Bayer color filter array (CFA) is their loss of light. Because each photosite is filtered to only receive either red, green, or blue, over half of the light hitting the sensor is thrown away. For low-light situations, that represents the loss of at least a stop of light — meaning cutting the camera’s shutter speed in half.
Panasonic claims it has come up with a radical new way to redirect the light coming into a sensor so that nearly all of it can be used by the sensor. Instead of using an array of tiny microfilters in a traditional CFA, the new approach uses what Panasonic calls “micro color splitters” that diffract the light so that various combinations of wavelengths (colors) hit different photosites. In their paper in Nature Photonics, Panasonics researchers claim their solution allows the sensor to gather 1.85 times more light than traditional Bayer-array-based sensors.
Pioneers get the arrows
Panasonic is not the first company to attempt to throw off the yoke of the Bayer array. Foveon became famous for its unique technology that used all the light hitting its sensor by layering the three color receptors on top of one another. Each layer essentially stripped off the color of light to which it was receptive, passing along the rest. Unfortunately for Foveon, they were alone in using this technique, so all the hard work of developing the sophisticated algorithms (and hardware) for processing the resulting uniquely coded raw images was up to them. As a result, it was years before Foveon had effective noise reduction and powerful enough chips to produce JPEGs in the camera.
Similarly, photosites in the Panasonic sensors will not simply register red, green, or blue. They’ll be receiving combinations of colors: white+red, white-red, white+blue, and white-blue, that come out of the two deflectors in use. Camera firmware will need to demosaic the results — calculating color values from those composites. Unlike with Bayer-array demosaicing — an active field of research with decades of science behind it — Panasonic will need to pave the way for decoding images from its unique sensors. While it is possible to generate RGB values with a simple matrix multiplication of the four color combinations received by diffraction-based sensors, doing it without noise is much more complex.
Panasonic claims that it has been able to use the very powerful FDTD (finite differential time domain) method to compute the final image by simplifying it to be fast enough to compute in real time. This provides a more accurate image than easier to calculate algorithms like B-BPM (Babinet beam propogation method). Like Foveon, Panasonic has protected its invention with a thicket of patents. It has received or filed for 21 Japanese and 16 overseas patents on the technology so far.
Not the only attack on the Bayer CFA
In a less-radical attack on the Bayer array, Fujifilm’s high-end X-Pro 1 relies on a clever 6×6 arrangement of differently colored photosites to reduce the image artifacts found in Bayer array cameras. Unlike Panasonic, though, Fujifilm’s approach doesn’t help address the loss of light issue. It still uses microfilters over each photosite which only allow one color to be recorded at each site.
Panasonic is reticent on some of the key attributes of its new technology. It doesn’t say what resolution can be achieved in its press release — although its researchers have claimed that there is no loss of resolution compared to traditional solutions — or how much noise is introduced by the diffraction process. Those details will determine whether Panasonic’s approach will be successful in replacing Bayer arrays in cameras and smartphones or be relegated to specialized market niches like surveillance that require the latest in low-light performance.
[Image credit: Maxmax]