Flat Fielding

The subject of flats arises frequently: newbies want to know what they are and why they need them and how to make them and veterans just make them and don't think very much about them except when they have problems with them
There is a lot of "kentucky windage" and old wives tales that surround flats, and I hardly ever see any quantitative discussion related to them.
Yet the process is best understood when you start with the theory so that you understand why you do what you do
The primary function of a flat is to remove fixed pattern noise from the image. The sensors have non uniform pixel response and the optical system has non-uniform light intensity distribution including dust motes and vignetting.
The flat field, when properly applied, levelizes the gain of each pixel in the optical system to give a uniform response to a uniform stimulus: a featureless uniformly illuminated target will only result in a featureless uniformly illuminated image when the flat field was properly taken and applied.
The sensor has four regimes of operation: read noise limited, shot noise limited, fixed pattern noise limited and full well. The read noise dominates the noise response of the camera when the signal level is less than the square of the read noise. That happens for low signal levels and may be an issue when imaging faint nebulae using emission line filters that essentially block most of the background light. A example of such a scenario can be found here where the signal differs from the background by only small amount and the overall level is low enough that the read noise is significant: http://www.narrowbandimaging.com/images/faint_nebula_exposure_example.jpg
The next regime of operation is the the shot noise limited regime. This says the noise is really only a function of the signal level and that is the best you can possibly do. In fact the goal of flat fielding is to correct the response of the sensor to be shot noise limited all the way from the transition out of the read noise limited regime to full well.
The fixed pattern noise limited regime is what we get rid of by applying flats. It turns out that the FPN limited regime covers the majority of the dynamic range of a sensor and kicks in at 1/(PRNU)^2
If the PRNU value (photoresponse nonuniformity, and is measured from a Photon Transfer Plot [ http://www.narrowbandimaging.com/photon_transfer_curves_page.htm ] ) is 1% (a typical value) then the FPN limited regime begins at 10,000 electrons and runs all the way to full well. For a 100,000e- well capacity that works out to be 90% of the dynamic range of the sensor that is limited by fixed pattern noise.
A proper flat will eliminate the fixed pattern noise, so what is a proper flat?
The signal level in a proper flat needs to be high enough so that the sensor's response is FPN limited. So that says right away that you need more than 10,000 electrons if your PRNU is 1%. With a PRNU of 2% then the FPN limited regime starts at 5000 electrons and so on.
The noise of the flat will affect the noise of the calibrated image. So it is essential that the flats be low noise or they will add noise to the image and that will defeat the purpose of applying the flats.
look at the first equation on page 9 of this presentation: http://www.narrowbandimaging.com/images/Flat%20Fielding.pdf
the last term, if zero, will cause the equation to reduce to the normal noise equation that contains only read noise and shot noise.
So the goal is to minimize the last term of the equation.
You do that by maximixing the term in the denominator, the Q(e-) term.
Since Q(e-) = Signal_flat * number_of_flats and since you are limited to the total signal in any one flat field exposure to full well on the high end and non-FPN limited level on the low end, the only real knob you have to turn is the number of flats.
by increasing the number of flats, you maximize the Q(e-) parameter and that minimizes the last term in the referenced equation.
referring to page 10 of the presentation, you will see a Photon Transfer Plot of Noise versus Signal plotted on logarithmic scales. The various traces marked by the Q(e-) values beside them show the noise characteristics of the calibrated image for varying numbers of flats used for the master flat. The Q(e-) term is the number of electrons contained in the data set used for the flat and as that approaches 1 million electrons, the corrected image's noise approaches the shot noise limited ideal response.

On log log axes a sqrt(n) function will plot as a straight line with a slope of 1/2 while a linear relationship will plot with a slope of 1, so by making the Photon Transfer plot using Log Log scales, it is trivial to see if the signal is shot noise limited (ideal) or FPN limited (non desirable).
The reason the FPN is a problem is that once it is dominated by FPN, additional signal will NOT improve the S/N ratio of the image. If shot noise limited, then a quadrupling of signal will result in a doubling of S/N and that is monotonic to full well.


Hit Counter