This process is more accurate than the interpolation used to increase the size (i.e. pixel count) of an image. In interpolation, completely new pixels are "made up" based on what the neighboring pixels look like. In Bayer data reconstruction we already have pixels, we just don't know two of the three color values. Since we do know the colors and values of neighbor pixels and since there is a color spectrum overlap, we can reconstruct the missing information very accurately.

Please note that the actual color reconstruction is more complicated than the method described here. For instance, to determine a given color value for a given pixel we use more than just the eight neighboring pixels. Furthermore, it is also possible to improve the result by incorporating certain assumptions about real world images in the algorithms (e.g. colors coincide at edges, etc.). We have simplified the process in this description to aid in understanding.

But what about resolution? First of all, we have to clearly define resolution. Resolution tells us how small the smallest structures (e.g. alternating black and white stripes) are that an optical or opto-electronic system is capable of reproducing. In digital photography, there is a tendency to describe resolution in terms of the number of pixels on the chip. Depending on the technology used however, the actual pixel count of a chip does not directly correspond to the resolution the system is capable of reproducing. The D20, for instance, is designed to accurately reproduce images at HD resolution (1920 horizontal pixels). In order to achieve this goal, a Bayer mask CMOS chip of a higher pixel count is necessary.

On the Bayer mask chip itself the full number of pixels is not available for each color. For a 2880 x 2160 chip, the red channel for instance does not have a resolution corresponding to 2880 x 2160 pixels. One could assume that since every second pixel is red in every second row, we have half the resolution for red (1440 x 1080). But that is not accurate either, since for most natural images the missing color pixel values can be reconstructed very accurately, so the resolution of the red channel is somewhere between 2880 x 2160 and 1440 x 1080.

Our goal with the D20 design is to output a very high quality HD image with a resolution corresponding to 1920 horizontal pixels. In order to achieve such an image output from a Bayer mask chip we need substantially more than 1920 horizontal pixels, which is the reason the chip's pixel count (2880 x 2160) is much higher than the desired image output resolution. The raw Bayer data at 2880 x 2160 goes through the color reconstruction process to fill in the missing color information and is downscaled to a pixel count that corresponds more closely to its actual resolution. This allows the D20 to create a high definition image that looks as good as if not better than the images produced by current high definition cameras.

One of the output options of our D20 demonstrator is to process the Bayer data in real-time inside the camera, and to output a 1920 x 1080 YUV (4:2:2) signal at 25 fps progressive through single link HD-SDI. To speed processing, color reconstruction and downscaling are carried out in a single operation. This is the live output that was shown at IBC 2003.

Another option is to output the raw Bayer data via dual link HD-SDI. In our lab we are currently recording this data on a disk array. The reconstruction and downscaling of the color image can then be done in non real-time, allowing us to apply more complicated algorithms to get a higher quality color image at 1920 x 1080 RGB (4:4:4).

It is important to remember that all of these processes are clever ways of getting the most out of the technology that has been employed. There is no “best” solution, only solutions that will work better than others under given circumstances. A major part of our ongoing research involves the evaluation of image processing techniques so that we can learn how best to apply them to achieve optimum results with all the different types of image content. This way can we ensure that our camera will not only have an impressive specifications on paper but will also be a useful creative tool for cinematographers.