Aliasing and Moire issues are heavily discussed topics for DSLR HD video, however the performance evaluations remain very subjective and are typically based on random Youtube or Vimeo videos. Adding to this, camera manufactures hardly disclose any details about the related actual sensor level pixel processing.
This is an alternative method to determine /characterize the aliasing and Moire performance of a DSLR video. Basically, the method is a scanning for critical line densities which produce interferences/resonances in the image. Knowing this will give hints on what is going on at the low level sampling process, reveling the effective vertical and horizontal resolutions. This will also provide information on the requirements for an optical antialising filter, ie. the blur diameter needed, to eliminate the aliasing artifacts.
The test is demonstrated in the videos:
The test is basically 2 fine line patterns (stacks of 600 lines), one vertical and one horizontal. To carry out the test, simply record a video of the test patterns while zooming in and out. In this way, the sensor is scanned for line densities with critical interferences/resonance. A subsequent analysis of the video will readily provide precise respective line density values. (Bear in mind that this line pattern is designed to be a worst case scenario for provoking Moire patterns)
The line test is here performed on the Nikon d5100 and a kit lens 18-55mm at f/5.6. The Nikon D5100 camera has the same CMOS image sensor as the Nikon D7000. The specifications of the image sensor are:
- Sensor size: 23.6 x 15.6 mm (Nikon DX size)
- Pixel count: 16.2 MP
- Still resolution: 4928 x 3264 px
- 1080p video sensor crop area (16:9) : 4928 x 2772 px.
With the line test charts I was able to find 3 characteristic interference points:
1. The first strong Moire interference point is found at a line density of about 462 vertical lines (see video no. 2). This is calculated from measuring the height of the line stack and comparing it to the available pixels on the 16:9 crop area. At this point there are 6 sensor pixels per line period. I believe this resolution corresponds to the Nyquist frequency of the video resolution, meaning the line pattern at this point ideally should appear as something like alternating light and dark pixels. However, vivid blue, magenta and yellow color bands are generated at this point.
2. Zooming out to double line density leads to another strong interference point, as seen in the figure below. The color bands are sligthly different, alternating with white, blue, black, and reddish colors. The line density now corresponds to 924 lines horizontal lines (see video no. 1) which corresponds to 3 sensor pixels per line period. I believe the vertical video resolution now exactly matches to the number of lines of the test pattern, giving one video pixel per line period (2 x Nyquist frequency).
So far, the vertical line pattern has been almost free of any colored Moire, however a closer look at the vertical line pattern during the zooming process shows Moiré patterns similar to the horizontal lines, only much fainter and without the coloring.
3. A third characteristic interference point is reached when zooming further out.
Here, it is the vertical lines that shows the strongest moire colors in blue, red and green. The horizotal lines show the same pattern, only a bit fainter. Measuring the width of the 600 lines block gives a resolution of 2 sensor pixels per line period. So, the line patterns at this point has a density of 1386 x 2455 lines .
It is noticeable how the horizontal lines through out the test appear very sharp and highly flickering when zoom to the primary interference point, and even further on. The vertical lines, on the other hand have much more smooth and calm appearance with minor aliasing problems.
It can be noted the exact same Moire interferences are observed in 720p mode, indicating that the sampling process causing aliasing the same for 1080p and 720p modes and the image scaling is happening on a later image processing step.
During the testing it turned out that the observed Moire coloring is sensitive to various camera parameters. E.g. iIt appeared that the distance of the line pattern and the camera seemed to affect the coloring and shooting from a longer distance suddenly didn’t produce the same amount of color. I suspect that chromatic aberrations or other color fringes of the lens could enhance coloring of the Moiré. ISO, white balance or other image post processing could also affect the coloring of the Moire.
Low-level sampling and image processing
From studying the line patterns in detail I am pretty convinced that the first and second interference point (1) and (2) do in fact correspond to the Nyquist and Nyquist x2 frequency respectively of the video sampling process. Designing an antialiasing filter must address the aliasing of the horizontal lines, and be a low pass filter that relates to this fundamental sampling rate, ideally cutting all frequencies above the Nyquist frequency. This means a blur diameter of 1-2 pixels is required when taking into account the effect of OPLF. + pixel aperture.
The line density of interference point (3) corresponds to the Nyquist frequency of the full sensor resolution, this means RGGB bayer cell per line period. Strong inteference with the bayer cell pattern can be expected. I guess this should have been prevented by the OPLF.
Trying to draw too detailed conclusions on the low level image processing process from the test is not feasible. I was far too optimistic to expect to decipher this whole process chain of Pixel binning (lineskipping?), debayer processing, image enhancement, rescaling, video encoding, etc. This is a seriously complicated reverse engineering task, and the image processing after the sampling process until the stored data on the disk is better left as a black box. The results are more complicated than what to be expected from a simple line skipping process.
However what the test do tell is that certain (horzontal) line densities and colors produce interferences. These interferences appear to be harmonics of a fundamental sampling rate of 1/3 of the full sensor resolution, giving a Nyquist frequency of 642 lines horizontal lines.
The sampling performance will finally depend on the specific pixel binning scheme. This will then correspond to 1.5 x 1.5 RGGB bayer cell per effective video pixel. The strong aliasing indicates of the horizontal lines indicate that this pixel binning process has various incomplete fill factors for the individual colors in this direction, while the horizontal direction is more complete.
The questions yet to be answered are:
- Why is the aliasing typically much more pronounced for one direction (horizontal lines)?
- Why do feature a wide range of DSLR cameras tend to have the same type aliasing limitations?
- What is the technical/ computational limitation behind that?
From the test I propose the following conclusions.
- The main Moire issue relates to horizontal lines, which are prone to heavy and colorful aliasing issues. The primary Moire patterns were observed for line densities corresponding to an effective vertical resolution of 471 px (Nyquist) and 924 px (Nyquist x 2) in 1080p video mode. This corresponds to a fundamental sampling rate of 1/3 of the full sensor resolution.
- Vertical lines are generally well sampled with much fainter Moiré and mostly grayish colors. Even though much weaker, the interference frequencies are generally the same as for the horizontal lines.
- Another colorful Moire interference was observed for vertical and horizontal lines densities of at 1/2 of the native sensor, – likely an interference originating from the Bayer color mask.
- Just a tiny amount of defocus completely removes all Moire patterns and coloring. Based on the factor of 3 pixel sampling process, it is estimated that an effective antialiasing filter should feature a blur spot with a diameter of 1-2 pixels.
All in all, it is definitely concluded that this camera will have great image detail- and color improvements by applying an optical anti aliasing filter.
I have only tried the test on my NikonD5100. So, any information about test results from other cameras would be very relevant when designing an optical anti aliasing (AA) blur filter one has to follow the fundamental parameters for the image sensor. The anti aliasing filter is designed for compensating for low fill factors. Knowing the effective sensor resolution and the number of pixel lines skipped will make it possible to determine the optima blur diameter of an anti aliasing for different image sensor types.