Posted by: recordingsofnature | February 27, 2014

Simple test to evaluate DSLR video aliasing and Moiré performance

[Updated article...]

Aliasing and Moire issues are heavily discussed topics for DSLR HD video, however the performance evaluations remain very subjective and are typically based on random Youtube or Vimeo videos. Adding to this, camera manufactures hardly disclose any details about the related actual sensor level pixel processing.

This is an alternative method to determine /characterize the aliasing and Moire performance of a DSLR video. Basically, the method is a scanning for critical line densities which produce interferences/resonances in the image. Knowing this will give hints on what is going on at the low level sampling process, reveling the effective vertical and horizontal resolutions. This will also provide information on the requirements for an optical antialising filter, ie. the blur diameter needed, to eliminate the aliasing artifacts.

The test is demonstrated in the videos:

The Test

The test is basically 2 fine line patterns (stacks of 600 lines), one vertical and one horizontal. To carry out the test, simply record a video of the test patterns while zooming in and out.  In this way, the sensor is  scanned for line densities with critical interferences/resonance.  A subsequent analysis of the video will readily provide precise respective line density values. (Bear in mind that this line pattern is designed to be a worst case scenario for provoking Moire patterns)

The line patterns can be printed from this document:  600 lines – line width 0.5.pdf , -alternative line widths are available here: 600 lines – line width 0.2.pdf and  600 lines – line width 1.0.pdf

The line test is here performed on the Nikon d5100 and a kit lens 18-55mm at f/5.6. The Nikon D5100 camera has the same CMOS image sensor as the Nikon D7000. The specifications of the image sensor are:

  • Sensor size:  23.6 x 15.6 mm (Nikon DX size)
  • Pixel count: 16.2 MP
  • Still resolution: 4928 x 3264 px
  • 1080p video sensor crop area (16:9) :  4928 x 2772 px.

Observations

With the line test charts I was able to find 3 characteristic interference points:

1. The first strong Moire interference point is found at a line density of about 462 vertical lines (see video no. 2). This is calculated from measuring the height of the line stack and comparing it to the available pixels on the 16:9 crop area. At this point there are 6 sensor pixels per line period. I believe this resolution corresponds to the Nyquist frequency of the video resolution, meaning the line pattern at this point ideally should appear as something like alternating light and dark pixels. However, vivid blue, magenta and yellow color bands are generated at this point.

Moire pattern at Nyquist frequency.

Moire pattern at 462 lines per height.

2. Zooming out to double line density leads to another strong interference point, as seen in the figure below. The color bands are sligthly different, alternating with white, blue, black, and reddish colors. The line density now corresponds to 924 lines horizontal lines (see video no. 1) which corresponds to 3 sensor pixels per line period. I believe the vertical video resolution now exactly matches to the number of lines of the test pattern, giving one video pixel per line period  (2 x Nyquist frequency).

Primary horizontal Line skipping Moire pattern

Horizontal line Moire pattern at 2 x Nyquist line pattern

So far, the vertical line pattern has been almost free of any colored Moire, however a closer look at the vertical line pattern during the zooming process shows Moiré patterns similar to the horizontal lines, only much fainter and without the coloring.

3. A third characteristic interference point is reached when zooming further out.

Higher order Moire on vertical and horizontal lines

Higher order Moire on vertical and horizontal lines

Here, it is the vertical lines that shows the strongest moire colors in blue, red and green. The horizotal lines show the same pattern, only a bit fainter. Measuring the width of the 600 lines block gives a resolution of 2 sensor pixels per line period. So, the line patterns at this point has a density of 1386 x 2455 lines .

It is noticeable how the horizontal lines through out the test appear very sharp and highly flickering when zoom to the primary interference point, and even further on. The vertical lines, on the other hand have much more smooth and calm appearance with minor aliasing problems.

It can be noted the exact same Moire interferences are observed in 720p mode, indicating that the sampling process causing aliasing the same for 1080p and 720p modes and the image scaling is happening on a later image processing step.

During the testing it turned out that the observed Moire coloring is sensitive to various camera parameters. E.g. iIt appeared that the distance of the line pattern and the camera seemed to affect the coloring and shooting from a longer distance suddenly didn’t produce the same amount of color. I suspect that chromatic aberrations or other color fringes of the lens could enhance coloring of the Moiré.  ISO, white balance or other image post processing could also affect the coloring of the Moire.

Low-level sampling and image processing

From studying the line patterns in detail I am pretty convinced that the first and second interference point (1) and (2)  do in fact correspond to the Nyquist and Nyquist x2 frequency respectively of the video sampling process. Designing an antialiasing filter must address the aliasing of the horizontal lines, and be a low pass filter that relates to this fundamental sampling rate, ideally cutting all frequencies above the Nyquist frequency. This means a blur diameter of 1-2 pixels is required when taking into account the effect of OPLF. + pixel aperture.

The line density of interference point (3) corresponds to the Nyquist frequency of the full sensor resolution, this means RGGB bayer cell per line period. Strong inteference with the bayer cell pattern can be expected. I guess this should have been prevented by the OPLF.

Trying to draw too detailed conclusions on the low level image processing process from the test is not feasible. I was far too optimistic to expect to decipher this whole process chain of  Pixel binning (lineskipping?), debayer processing, image enhancement, rescaling, video encoding, etc. This is a seriously complicated reverse engineering task, and the image processing after the sampling process until the stored data on the disk is better left as a black box. The results are more complicated than what to be expected from a simple line skipping process.

However what the test do tell is that certain (horzontal) line densities and colors produce interferences. These interferences appear to be harmonics of a  fundamental sampling rate  of 1/3 of the full sensor resolution, giving a Nyquist frequency of 642 lines horizontal lines.

The sampling performance will finally depend on the specific pixel binning scheme. This will then correspond to 1.5 x 1.5  RGGB bayer cell  per effective video pixel.  The strong aliasing indicates of the horizontal lines indicate that this pixel binning process has various incomplete fill factors for the individual colors in this direction, while the horizontal direction is more complete.

The questions yet to be answered are:

  • Why is the aliasing typically much more pronounced for one direction (horizontal lines)?
  • Why do feature  a wide range of DSLR cameras tend to have the same type aliasing limitations?
  • What is the technical/ computational limitation behind that?

 

Proposed Conclusions

From the test I propose the following conclusions.

  • The main Moire issue relates to horizontal lines, which are prone to heavy and colorful aliasing issues. The primary Moire patterns were observed for line densities corresponding to an effective vertical resolution of 471 px (Nyquist) and 924 px (Nyquist x 2)  in 1080p video mode.  This corresponds to a fundamental sampling rate of 1/3 of the full sensor resolution.
  • Vertical lines are generally well sampled with much fainter Moiré and mostly grayish colors. Even though much weaker, the interference frequencies are generally the same as for the horizontal lines.
  • Another colorful Moire interference was observed for vertical and horizontal lines densities of at 1/2 of the native sensor, – likely an interference originating from the Bayer color mask.
  • Just a tiny amount of defocus completely removes all Moire patterns and coloring.  Based on the factor of 3 pixel sampling process, it is estimated that an effective antialiasing filter should feature a blur spot with a diameter of 1-2 pixels.

All in all, it is definitely concluded that this camera will have great image detail- and color improvements by applying an optical anti aliasing filter.

 

I have only tried the test on my NikonD5100. So, any information about test results from other cameras would be very relevant when designing an optical anti aliasing (AA) blur filter one has to follow the fundamental parameters for the image sensor. The anti aliasing filter is designed for compensating for low fill factors. Knowing the effective sensor resolution and the number of pixel lines skipped will make it possible to determine the optima blur diameter of an anti aliasing for different image sensor types.

About these ads

Responses

  1. ” I do not have knowledge enough to say which method is used on the Nikon d5100″
    Non of them. You’re whole approach is wrong since the 5100 is using the toshiba MT9H004 and that sensor is using at Video Framerate 60 @ 1080 a 2×2 binning.

    http://www.promelec.ru/pdf/mt9h004_flyer.pdf

    On the other hand with any type of skipping you would not even get close to the dynamic range of the camera.

    http://en.wikipedia.org/wiki/Poisson_distribution

    I recommend to study this paper and to change you’re article.

    http://asp.eurasipjournals.com/content/pdf/1687-6180-2012-125.pdf

    best Wolfgang

    • Thanks for the feedback! Your references are highly appreciated!
      I have removed some of on my own (rather unsupported) conclusions in the article, while leaving more focus on the descriptions of the test observations.

      Best regards
      Kristian


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

Join 49 other followers

%d bloggers like this: