DxOMark Lab

How about Shun Cheung<g>?

How about googling the lens in question?

Ted, there is no "reliable rating system" as there are too many variables to measure and most variables are difficult to measure properly.

For example, DxO is ok for sensor performance, but there tests are really, really simplistic and they miss a lot of the nuance and aren't that relevant to the real-world. Also, keep in mind that numbers between brands are not comparable. If you want a real in-depth discussion of sensor performance (and why quantifying this sort of thing is so hard) look at http://www.clarkvision.com/articles/digital.sensor.performance.summary/
My apologies that the link is Canon-centric, but such is life.

As for lens tests, there is no convenient way to scientifically test a lens as lens performance varies greatly as a function of focus distance. Thus, the data at the photozone.de or the-digital-picture .com is only relevant if you focus your lens at the exact same distance they focus theirs. Generally, the only lenses they test at something approaching a real-world focus distance are the really long telephotos. The wide and normal lenses are tested at a couple of feet, where these lenses typically perform the worst. Real world performance will be better. And to make your life even more difficult, even if lens A tests better at 2 feet than lens B, there is no way that you can infer that lens A will still be better than lens B at 8-10 feet or at infinity.

So basically, read a bunch of reviews to get a feel for what people think about the lens or camera in the real world. Then buy from a reputable retailer with a solid return policy.

I always use several sources when evaluating a lens or body prior to purchase. There is a lot of useful information on Photo.net.

I find this site very useful for comparing bodies, especially for high ISO comparisons:

http://www.imaging-resource.com/IMCOMP/COMPS01.HTM

Ultimately the best source for rating gear you will be using is you.

I don't agree with Craig's comments about DXO:

"tests are really, really simplistic:
"numbers between brands are not comparable"

Neither of these statements are factual. You can accurately compare numbers from brand to brand and get accurate performance analysis.

There is a lot of information about the site under their 'ABOUT' tab:

http://www.dxomark.com/index.php/About/What-is-DxOMark2

Elliot, both of those statements are factual.

The sensor tests are indeed very simplistic as there is a difference between what a computer can measure and what a human eye can measure. If the noise is imperceptible to me or my audience, it doesn't matter what the computer thinks. Finally, there is a lot of other stuff to deal with... for example is a FF sensor better than a APS-C if you have to crop the FF image to APS-C size (No). Non-random noise (banding and such) is more easily noticeable by humans than random noise... does DxO mark make the distinction... No. So there is a whole lot more going on than one might first think.

As for comparing sensors between brands, as we never have access to the truly raw sensor data, we don't know what each manufacturer is doing under the hood. It would be really easy to game DxO by applying some noise reduction to the raw sensor data before it is stored... is anyone doing this... who knows.

As for the lens tests, they only test at a single focus distance, rendering those less useful than one might think.

"we never have access to the truly raw sensor data"

per DXO "All measurements are performed on straight-from-camera RAW files, the only reliable way to evaluate intrinsic hardware performance"

"it doesn't matter what the computer thinks" Their computer's don't 'think' - they record data. And their site simply shares the data. It the data important? It depends. It is very important to read the information on the site carefully so you clearly understand what the data means.

Personally I use it as just one of many resources. I don't think anyone should rely on just 1 source. And as individual results vary, one person may find a lens or body good where others and sometimes many others have rated it poorly.

Right... but the straight-from-camera raw data is not straight-from-sensor raw data. We have no idea what happened to the raw data during the initial processing. Maybe nothing, but I doubt it. And how does calculating signal-to-noise ratios from a 18% gray card extrapolate to real world performance... I don't know, DxO mark doesn't know, and you don't know

And again, there is no point in testing things below the limits of what the human eye can perceive. So yes, DxO can find a mathematical difference between two cameras... but can you or I or our viewers? In most cases, the answer is "no". And this all assumes that we use the image straight out of the camera, which is also not the most likely scenario.

So yes DxO mark provides data. Is it good data... perhaps. Is it useful data... not so much

I'm with Elliot on this, DxO provides good, reliable comparative data.

How that relates to the Real World is much more subjective and relates to your specific type of shooting. I don't think DxO has ever intended it's work to answer that question.

I just use DxO as part of the info gathering process before deciding anything.

Before this thread goes completely down a rabbit hole, I would like to emphasize the point the DxO does not review cameras; they review characteristics of camera sensors. How useful the number they post is up for debate, which is the rabbit hole this thread is descending into.

Regarding their lens testing, and lens testing in general, people are always most interested in how "sharp" a lens is, and a well conducted test will tell you that, but only under lab conditions. In the real world, focus and camera stability dominated resolution performance. I suggest reading comments made by photographers who know what they're doing, have actually used the lens and know what they're looking at.