Pixel Peeping Part II

I'm a complete dummy when it comes to the math and physics of photography. I think I kinda understand the graph. Would you be able to post examples of images where all this comes into play? Maybe a side-by-side comparison. I guess I'm just slow.

Well, if I can find a subject that actually shows aliasing problems (other than a resolution test chart!) I'll do that, but as I said, in actual photographs aliasing effects are pretty rare.

I'm guessing, but I would shoot at an aperture that gives an MTF equal to 0.5, which would be about f11 for this lens (sweet-spot?) The colour fringing is harder to deal with, since the Bayer-pattern sensors are more than 1 pixel apart (30 lp/mm?)

I guess that's why Leica didn't put any anti-aliasing filter in front of their sensor in the R digital back. Their R lenses must resolve some pretty fine details and I guess they don't want to mess it up with a filter. Not that I can afford anything more than a used Leica lens hood myself, but I guess their lenses must be quite good.

A picture with some kind of printed text in it could show real-life aliasing problems.

I think that Bob's point here was to shout something like "get to know your own equipment". What he shows here is an extreme case, with alternating black and white lines, which, for digital cameras, are the worse things to deal with.

If the lines would have been 60& gray and 40% gray, probably you wouldn't have had any image artifact at all: a change from white to black has a higher frequency response than a change from 60% to 40% gray... next time you shoot, start trying variations of aperture and speed, keeping the exposure the same, and see how the quality of the results changes.

Not only: what is good in one situation, might be bad in another. I think that it is useless that Bob provides other examples, since every camera is different, and the only thing you can do is try yourself with your own.

I goes to show that all pixel peeping should be done under the supervision of a fully trained and certified scientist whose findings must be reviewed by a panel of peers!

James: This page at dpreview: http://www.dpreview.com/reviews/kodakslrc/page19.asp shows some real-world aliasing on one of Kodak's 14MP cameras (which don't have AA filters).

Interestingly, I had always thought that diffraction at f/11 on my 10D should only create minimal diffraction effects, but failed to realize that f/22 would have such a dramatic effect, being "only" two stops away (stupid me thought too much of a logarithmic scale and didn't really visualize that diffraction at f/22 would be twice as big as f/11).

You could get almost the same effect with a focus error. If you look at the defocus MTF of a lens at f5.6 and give it just the right amount of defeocus, you can pretty closely match the MTF at f22.

So if one person claims to see really bad aliasing artifacts and another claims not to see them at all, it could just be that the second person has a slight focus problem. The difference in measured "resolution" would be negligable - though the difference in perceived "sharpness" would be significant.

I'd say it's true that stopping down past f11 will reduce sharpness with all lenses on DSLRs. The best lenses peak around f4-f5.6 and the worst lenses peak around f8-f11. After that it's all downhill!

This is also something of a lesson in "what's left out" being important. There was a recent thread here with someone complaining that his new "L" lens was really soft. After people guessing at why and suggesting maybe to send it back to Canon and after it was revealed that high ISOs gave better results than low ISOs, it turned out he was hand-holding the lens at slow shutter speeds...nothing wrong with the lens at all. It's not what you don't know that gets you, it's what you don't know that you don't know.

I guess that's why Leica didn't put any anti-aliasing filter in front of their sensor in the R digital back. Their R lenses must resolve some pretty fine details and I guess they don't want to mess it up with a filter.

The point is that the sensor can't resolve all that fine detail, and mostly it just causes aliasing effects.

Digital cameras really don't like lenses which resolve a lot of fine detail. They really like lenses with a very high MTF below the Nyquist frequency, but very low MTF above it. Unfortunately this is an impossible optical problem!

Dare I suggest that the average 28-300mm f5.6/6.3 superzoom resolves about the optimum for the digital sensors on your fancy camera ;-)

With film you could always claim that your fine grained film resolves XX lines per mm - with digital you know that it can't possibly do better than the pixel spacing (worse for Bayer...)

(OK, I know that MTF is multiplicative, but aliasing proves that sharper isn't better in this case).

I think the comment "Digital cameras really don't like lenses which resolve a lot of fine detail. " highlights the essential difference between users of high quality (and often larger format) film and digital users. One is seeing high sharpness, the other is seeing fine detail. One is related to the MTF cutoff frequency (resolution) and the other relates to the beginning of the MTF cutoff at about 50% (sharpness). High sharpness on appropriate low resolution subjects can blow you away, which is precisely the kind of image that is used for digital equipment advertisments. The much quieter, high resolution images often have lower perceived sharpness but I believe have a longer lasting appeal.

Two things:

First, it depends on the size of the image. With prints up to 8x10 (or maybe even 11x14), you really can't see finer detail than the sensor can provide unless you look at a print with a magnifying glass.

Second, though 35mm film may be able to record finer detail, it's not easy to get that detail into the print. Conventional optical englargement has problems because of the MTF of the enlarging lens. Scanning and digital printing can maybe do better but you're probably going to need better than a desktop scanner and that can get expensive.

Digital cameras really don't like lenses which resolve a lot of fine detail. They really like lenses with a very high MTF below the Nyquist frequency, but very low MTF above it. Unfortunately this is an impossible optical problem!
I was under the impression that that this describes the pre WWII Leica lenses as compared to contemporary lenses, or older lenses in general vs lenses made in the last 30 years.
IE older lenses were "optimized for contrast rather than resolution"
I'd like to find a nice fast lens with a tiny bit of under corrected spherical aberration but no chromatic aberration or linear distortion for my Canon.

Hmmmm... From what I see and understand, unless this happens to be taken with a Sigma this may not REALLY be a nyquist frequency issue. Instead it may be more of a moire issue due to the uneven sampling of the 3 channels on a bayer-mask sensor. That might still count as a nyquist issue, but I think it's more of a special case since the sampling has gaps either the same size as the sample or larger. I think you might only really be able to demonstrate aliasing due to nyquist issues on one of the old Kodak dSLR's with monochrome sensors.

>>> unless this happens to be taken with a Sigma this may not REALLY be a nyquist frequency issue. <<<

I'm no information theory expert but it would still seem to be a nyquist frequency issue - essentially antyhing above the nyquist frequency is essentially noise at the sensor level and since in a Bayer pattern the color photo sites are separated in space they're subject to individual noise which the interpolation trys to make sense of. The result is color artifacts.

It's just a guess, but I'd expect the Foveon sensor to have less color artifacts in the same situation and the artifacts might be less obectionable.

Thanks to Bob for the excellent work on this article. I've seen artifacts like this show up occassionally and it helps to have some additional knowledge of what the cause is.

The original posting did not mention two factors:

1. Was a camera used that uses a Bayer sensor (most likely). In this case the aliasing is highly related to the Bayer pattern of the sensor.

2. Which raw converter used (or the one in camera) as different raw converters have different abilities to avoid aliasing.

My 2 cents.

Uwe

Yes, it was a Bayer sensor (everything but the Sigma Foveon bodies) and it used in camera RAW processing (images were saved as highest quality JPEGs).

A comparison with a fine grained, 35mm colour negative film and a Foveon sensor based camera would have been interesting.

Thanks for the article, Bob - I'm sure that some of us got something out of it.

Bob, you are right, the whole issue is acedemic for 8x10 images. Everything but the basic consumer cameras can achieve the required effective resolution (around 5-7 lp/mm) on a print. I was thinking of larger prints. Interestingly, I did some testing and at their best settings, I resolved 3x higher spatial frequency with slow 35mm monochrome film, enlarged conventionally with a Nikkor lens than I could acheive with an 8 Mpixel Olympus 8080 or Fuji S2, at optimum aperture, lowest ISO setting, in Raw mode and optimised in PS CS. Of course the resolution on digital cameras is also directional, the S2 is better on orthogonal planes and the Oly on diagonals, whereas film of course is onmidirectional. You could have taken your two test images and just turned the camera 45 degrees to confuse everyone!

Excellent couple of articles. Thank you, Bob.
I have a question:
If we now take into account the possibility of post-processing (PS and Co), can we even reduce further the difference between A(high MTF) and B(low MTF) by boosting appropriately the local contrast ?
(I would think that you can get almost the same perceived sharpness in both cases, although A would have a small advantage in Resolution and disadvantage in Aliasing)

Side-note: DxO Optics Pro is looking right into this kind of battle...

Olivier

Olivier

Be careful now. You're coming dangerously close to saying that by using sophisticated digital processing we can make the results of inexpensive lenses look just as good as those from very expensive lenses.

While there is some truth in this, a lot of people won't be very happy to hear it and won't thank you for pointing it out.

Olivier: like Bob said, be careful about not reaching the wrong conclusions.

In theory, if you can know very precisely the system MTF, you can deconvolute the measured signal and get back the original signal.

Problems start to arise with Shannon/Nyquist. If you deconvolute a measured signal that was sampled at too low a frequency (i.e. where the original signal contained response above the Nyquist frequency), you will amplify the sampling artifacts. In Bob's example if you were to deconvolute picture "A", the response visible on the right (about 75% into the frame) would be severely amplified and would become even more damaging.

In order to avoid those Shannon/Nyquist problems, you want to make sure that the response above the Nyquist frequency is as low as possible, essentially starting with picture "B" instead. Looking at the MTF curve that Bob plotted, you can see that at about 60 lp/mm the response at f/5.6 is approximately 4 times higher than that at f/22. In order to make "B" look like "A" at 60 lp/mm, you'll have to multiply the measure signal in "B" by a factor of 4. And that's the part that hurts: you'll also have to multiply the 60 lp/mm component of the noise by a factor of 4. Which means that your image "B" shot at ISO 100 and deconvoluted will have about as much noise as image "A" shot at ISO 1600 (noise goes up approximately like the square root of the sensitivity). Suddenly you'll realize that other factors like shutter speed become significant: shooting at f/5.6 ISO 1600 1/16000s with no additional processing or at f/22 ISO 100 1/60s with deconvolution will get you about the same response (MTF & noise) at 60 lp/mm, if you can shoot on a steady tripod (with a normal lens, 60 lp/mm hand-held isn't reasonably reachable hand-held).

Furthermore, in order for such a technique to be successful you'd have to apply it to the raw data (before bayer reconstruction), and in that case you need to realize that the Nyquist frequency in the red and blue planes in very low. Blue is especially annoying as blue light is less affected by diffraction, so in order to get blue light to show little significant response at the Nyquist frequency of the blue sensors you'd have to stop down beyond f/45 given the pixel size of current DSLRs.

So, while such techniques work in the lab, they don't quite work in the real world (if you take an original signal with no response close to the Nyquist frequency and apply a Gaussian blur to it, is it possible to reserve the effect of the Gaussian blur reasonably precisely if you were careful to work with enough bits per pixel). Yet, with a very sharp lens (e.g. a 50/1.4 on a 20D) with subjects that have lots of high contrast detail, it may make sense to stop down to f/8 or f/11 to reduce the lens resolution and reduce the effect of artifacts, then apply a reasonable amount of sharpening to recover some of the lost detail below the Nyquist frequency. It's a tradeoff between artifacts and noise, essentially, once that can't be pushed too far.

I have downloaded and tried DxO Optics listed in the above post. Wow, it improved dramatically an already very sharp image from my Nikon 12-24. Appears to deal with issues raised here and improve digital performance.

In theory, if you can know very precisely the system MTF, you can deconvolute the measured signal and get back the original signal.

Assuming there is no noise. Stacking images can reduce the noise to almost any desired level, but this is unlikely to be practical for common photographic work. And no amount of stacking can restore a signal that isn't present due to a band-limiter (aka "anti-alias filter").

Regarding deconvolution in the presence of noise, aliasing, etc.

In the early 90s I worked on post-processing approaches to the Hubble mirror flaw (which essentially amounted to sophisticated deconvolutions). One of the major problems was the so-called ill-conditioned nature of the original (near-exact) equations: Small errors in the input (like noise, including quantization noise) would result in large artifacts/errors in the output.

The way out for this, is to solve a "nearby" equation whose solutions in the absence of noise if nearly identical, but which is not nearly as sensitive to errors in the input. Many such methods exist and I suspect DxO Pro uses something of that nature. (These methods are sometimes called "regularization" because the nearby operator being inverted is farther from being singular--i.e. more regular-- than the direct model.

Incidentally, boundary effects are another pain in this context: Pixels in the ideal scene that are just outside the captured scene will "leak" into the scene through lens imperfections. Similarly, some energy from the pixels in the ideal scene is lost through other leaks. The net effect can cause severe "ringing" artifacts unless a special kind of regularization is introduced. (I mention this mostly because that was the part of the Hubble problem I worked on.) I did note that DxO slightly crops its input and I think it does it more than distortion alone would justify: I wouldn't be surprised if some of it is to reduce these boundary effects.

Wrt. aliasing, I remember that in the mid-80s the big thing on then-high-end CD players was oversampling: CD players would take the 44KHz stream from a CD and insert in between null samples to obtain a new stream of higher frequency. Although the new stream doesn't have new information, it turns out that in the frequency domain the "blobs" to be separated by an analog filter are farther apart; i.e., the analog filter becomes more effective. For a camera sensor, that sort of suggests (I'm musing on the fly here; it may all be nonsense) that one might like to separate the pixels in a sensor to make the anti-alias filter's life easier.

In all, I suspect that -- as Bob Atkins suggests -- "sophisticated digital processing" may well change the future of the photographic lens industry. My guess is that it will lower the manufacturing cost of high-end lenses (which would have a truly "digital component": their inverse transfer operator), and move additional value to the compromises made in the sensor.

Ok: I was just looking for an excuse to use my average Canon 70-210mm f/3.5-4.5 instead of buying a 85mm, 135mm and 200mm (and using them on a tripod all the time). Thanks, guys... ;)

Half-true: I am also very interested in knowing more about "what's going on" and understanding the limits of my equipment (present and future). These articles and the discussion that followed are extremely interesting... keep on :)

I was actually observing a picture of mine taken eith my 10d and 135 f2 lens. The resulting picture of an old door with fasade was not sharp (taken at 1/125s f5.6 handheld), it is strangely unsharp, not like shaken i can`t explain the thing, but blame it on a to slow speed, now this test let me think! Is it actually possible that i have a "too good" lens for my sensor? What`s the point then?

David: no, f/5.6 on a 135/2 wouldn't explain not being sharp. The only real detrimental consequence of using a very good lens is the potential increase in artifacts, as seen in picture "A". Notice that "A" is a lab test, in probably optimal or near-optimal conditions, with a target designed to have high-contrast high-frequency details. I'd definitely have a look at your shutter speeds, 1/125s hand-held with a 135mm is something I would normally try to avoid, I'd feel better at 1/250s or higher.

Just to add a little more to my earlier comments, I think we're very likely to see a shift away from 19th and 20th century approaches to imaging and image quality.

In the 19th and 20th Century, the lens was king. You wanted better image quality, you designed a better lens. You didn't have a lot of choice.

My feeling is that we've only just started to glimpse 21st Century approaches to improving image quality. We're still mostly stuck with the conventional optics hardware approach. There's a huge amount that can be done with software and with non traditional optics (diffractive and holographic).

If people think current technology is in some way "cheating" compared to the old lens-film photography, I think they're in for a real shock when they see what's coming down the road in 5-10 years time. We've maybe moved from stone tools to crudely fashioned bronze tools in the last 20 years, but we have a long way to go yet!

I have had a different experience with respect to aliasing in the "real world". I've seen colored moire fringes appear on many subjects from my Canon D30 and D60 with a 50mm/1.4 used at good apertures (around f/8), and other lenses too. I've seen it appear on clothing fabric, bird feathers, sand, concrete, basically any fine high-contrast texture. This was "real world" shooting, I wasn't intentionally trying to expose the problem. I have not seen any problems from my 20D yet, but I haven't used it as much.

I agree with Bob that the current paradigm of digital camreas is most likely a very temporary one. Right now digital cameras are more or less identical to the film camera designs that have been around for decades (or centuries in the case of some view cameras) but with a digital sensor instead of film.

Last year in college, there was a group doing research on "lensless cameras" This wasn't a pinhole camera or anything, these were camreas with the sensor jut totally exposed. And they got images from their system that were perfectly recognizable color images.

Granted this wasn't something you could carry around, in that it was made of dozens of webcams all precisely aligned, attached to racks in the lab, but still, it just made me think that the future of digital imaging will probably be very interesting and unpredictable.

As a guess, I think in all this discussion, there is a systematic confusion between "digital" and... the Bayer matrix (although I don't of any digital camera with anything else... Foveon?). It seems to me that what is shown is the limitation of the Bayer matrix (and its periodicity) and not necessarily the "digital" adjectif. Maybe, as in printing, and since you wanted a comment that could be pertinent for the years to come, a random matrix (and not periodic) could solve this problem...Thanks for a comment...

Well I'll be damned, I was so off base with my guess its not even funny. Guess that shows how tricky pixel peeping really is, and how hazardous it is.

At f22, you almost certainly have unsharpness from diffraction. Thus the failure to resolute in image B, resulting in a 'softer' transition to indistinguishable line pairs - because the image itself is less sharp.

On the other hand in image A, the lines get resolved correctly - but what happens when for example a white line and two adjecant black lines get resolved onto a single row of pixels? They simply turn into a dark grey line, while one white and one black line will simply resolve to grey. This leads to the artifacts visible in both pictures, they're just more apparent in A because the image has higher overall sharpness.
B looks 'better' here, but I'm quite sure on real life applications, the f22 will result in worse images.

Best regards,

Matthias

P.S.: Diffraction: This is the reason for images from pinhole cameras being 'unsharp' - the circle of confusion can't be greater than the pinhole itself and should lead to an optimal sharp result - if only the light wouldn't have the habit to bend around the edges of the pinhole and create this typical softness. Ever wondered why your digital SLR won't stop down below f22 while the average LF lens has f64 as a minimum aperture? This is (part of) the reason...

Besides, with photochop everywhere, the difference between a mediocre and a good camera won't be really obvious, of course depending on the size of the result - but if that is what you are after, you shouldn't discuss digital but rather LF and these Images

The Digital Module R and M8 of Leica employ Kodak sensors without anti-aliasing filters. .

Wow! after wasting 5 minutes with that garbage, I think I'll go out and find something to photograph!

"My feeling is that we've only just started to glimpse 21st Century approaches to improving image quality. We're still mostly stuck with the conventional optics hardware approach. There's a huge amount that can be done with software and with non traditional optics (diffractive and holographic)."

This is most probably correct and explains why we can care less about our current equipment. The whole software approach to better imaging on the other side seems to be a way of further technological complication and dependency. I am wondering when science is going to make imaging better without creating new impossibilities, read: better AND simpler.

I seem to run into this a lot when using the flash and shooting cats with my D70 / kit lens.

And MTF stands for?

Thanks, Jay

MTF=Modulation Transfer Function

Now what I would like to see is a geek-friendly camera allowing recalibrations with black and whitefields to correct for stuck pixels, fitted with a liquid N2 tank for cooling the sensor and with a software package for customizable deconvolution that can take data from the camera about lens attached, focal distance, aperture etc... *drool* Unfortunately, I fear that after the IT crash there aren't enough rich geeks out there to create a market for this product. :-(