A Site for Photographers by Photographers

photo.net camera review standards

mostly by Philip Greenspun
These are the things that distinguish photo.net from other Internet and print sources of product reviews:
  • we are interested in cameras as tools rather than gadgets
  • we have a comment server and 140,000+ registered users, which means that our review need not be comprehensive; we don't have to have the last word; our review is the start of a discussion in the comment area underneath, not the end of a discussion
  • we are not obsessed with price; if something is 20 percent cheaper that's only worth mentioning in passing, especially if taking advantage of the cheaper product would require switching systems
The tool focus means that we're interested in how a camera performs when asked to do a task. If a camera seems reasonably well suited to covering a family event, take the camera out to cover the event then write about how it went. It would be ideal if you owned the camera and applied it to dozens of tasks. But if you can only borrow the camera for a few weeks, just try one or two small projects. People who own the camera will be sure to flesh out your limited perspective in comments underneath. (They'll also complain that you were lazy and did a bad job and wrote a sketchy review -- of course it never occurs to them that if they've owned the camera for some time and haven't reviewed it, they are probably the lazy ones!)

The tool/task focus means that we don't need to test every feature. If a digital camera can make movies, that's great. But we don't need to mention it in a photo.net review. If someone wants comprehensive specs they can visit the manufacturer's site or a site like www.dpreview.com. photo.net is for people who are serious about our craft. If we want to make movies we'll use a 3-CCD camcorder or maybe rent a 16mm film camera.

The tool/task focus means that we don't waste a lot of time comparing equipment from different systems. Is the Canon 70-200 a better lens than the Nikon equivalent? Suppose it is. If a reader owns three Nikon bodies and 10 lenses, do we expect him to chuck it all and buy a Canon EOS system?

We concentrate on areas where products are likely to be weak. In a mature industry such as 35mm film cameras we can assume that a $1000 Nikon body will be well made, reliable, accurate, etc. Nikon has been making cameras for the better part of a century and they have lots of good engineers. On the other hand their products tend to be (1) heavy, (2) unusable (custom functions useful only if you remember what "16-3" standds for; bizarre user interfaces on cheaper bodies), and (3) cumbersome for simultaneous auto/manual focus. So these are the things worth exploring right in the first few paragraphs.

In all reviews, if you're not one of our editors include an H3 headlined "About the Author" section at the very bottom of the article. This should be one paragraph saying (a) when you began pursuing photography seriously, (b) where you live, (c) what your day job is (if you're a professional photographer, that's the best). This should close with hyperlinks to your personal Web site and your community member history page on photo.net.

Note: the results of a community discussion of these standards is available from http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=0021A6

A Digital Camera Review

An ideal digital camera review will contain the following information in the first paragraph:
  • price
  • lens focal length range
  • format (SLR, ZLR, point and shoot)
  • resolution and bit depth (8, 10, or 12 bits per pixel?)
The next few paragraphs should cover at least the following topics:
  • anything significant and unusual about the camera, e.g., a Minolta that has an electronic viewfinder or an Olympus ZLR with a real mirror and high quality optical viewfinder
  • an indication of the kind of photographer for whom the product is appropriate, e.g., is this for a studio catalog photographer, a photojournalist, a kid, a traveler?
  • if the pictures illustrating the article are all from one project, some background on what kind of project it was and what settings were used (8-bit high quality JPEG or 12-bit RAW)

Speed of operation has been horrible with digital cameras, to the point that the vast majority made through 2002 are useless for taking pictures of anything that moves (e.g., people). Consequently this speed of operation deserves a separate H3 headlined section, covering:

  • Power on to first image capture: *** seconds.
  • Sleep to first image capture: *** seconds (how many presses of the shutter release before you get a picture?)
  • Autofocus on human face, outdoors: *** seconds.
  • Inter-exposure lockout: *** seconds

After putting out those statistics on operation speed, get into the basics of using the camera. The things that we take for granted on film cameras are often painful or impossible with digital cameras: Does the viewfinder show you what the picture is going to be? Is the viewfinder big and bright? Is it easy to evaluate focus? Is it easy to change or bracket exposure?

One thing that digital cameras do much better than film cameras is allow for the instant review of pictures taken. However, some digital cameras are much better than others. The Canon line has a nice feature whereby if you keep holding the shutter release down after an exposure the picture just taken will remain in the rear LCD display. Note if this feature exists on the camera you're reviewing and complain if it doesn't (most digital cameras have you select from a menu how many seconds you want each photo taken to linger; so photos that you don't care about clutter the display and drain the battery for 10 seconds while photos that you want to study disappear all too soon). One of the worst ideas in digital cameras is the "Playback mode" in which the camera cannot be used to take pictures. This is a real user interface nightmare: a cryptically labelled with icons switch that, if thrown, will render the device useless for its primary purpose. Better cameras have a "review" button that gives you all the power of a playback mode but whose effects evaporate the instant that you touch the shutter release.

Digital cameras that seem competitive in bright sunlight may perform very differently indoors. Try some exposures indoors, flash off, at the higher ISO speeds. Put these in a separate section and discuss the level of image noise, possibly compared to other digital cameras that you've tried.

Write a paragraph on using flash. Dispense quickly with the capabilities of the built-in flash (if any). Talk about compatibility with accessory and studio strobes.

Right now (2002), assume the person reading the review owns a Microsoft Windows machine but has never owned a digital camera before. Thus your reader is going to have the following problems:

  • storing and organizing digital photos on the computer's hard drive (image library management)
  • rotating images to correct orientation if the camera lacks an internal sensor (it is worth mentioning this lack in the review -- all digital cameras will eventually be smart enough to remember how they were oriented when held)
  • editing images (inclusion of a lightweight version of PhotoShop or similar program is a big plus)
  • creating thumbnails hyperlinked to larger images for Web publishing (good if this can be done in a batch)
Does the software de-JPEG and re-JPEG unnecessarily, e.g., when rotating or renaming an image? This will result in a loss of image quality and you can tell if it is happening by comparing the file sizes on the flash card and on the computer's hard drive. If the file sizes differ, the picture has been decompressed and recompressed.

Include a section of nagging gripes, maybe H3 headlined "Warts", small things that the engineers could have done better. photo.net does not get any money from camera manufacturers so there is no reason for us to suck up to them. If there are some surprisingly clever or good features that don't fit logically anywhere else, put in a section headlined "Features".

Try to fit a discussion of power source in somewhere. How many pictures can the camera capture on a charge (your experience, not the manufacturer's spec)? Do you need to carry a separate charging cradle or can the camera itself act as the charger? Can you use standard disposable batteries?

Close with a section H3 headlined "Competition". Stick to things that are within 0.5X to 2X the price of the camera reviewed. Break up the paragraph or paragraphs by photographic objective. E.g., "For macro work, the Nikon Coolpix 99999 is a better choice", "For sports photography you'll need the longer telephoto range of the Olympus E8000", "For available light indoor photography, consider the Sony Mavica X77 with its low noise at ISO 800 and fast f/1.4 lens."

At the end of the article is a "Where to Buy" section with links to camera shops that pay photo.net a referral fee. Try to copy the form of the links from other similar reviews on photo.net.

The last required section of the article is a "More" section with links to reviews of the same camera on dpreview.com and Steve's Digicams and other popular sites where digital cameras are reviewed feature-by-feature. By linking to these dreary tech-oriented reviews we eliminate the need to emulate them. People can come to photo.net first to find out whether a particular camera is good for a particular photographic challenge. Then they can click through to a "gearhead" site and make sure that the dimensions and weight are right for their coat pocket.

If you have a bunch more images that you'd like to display but that don't illustrate any of the points that you're making in the article, stick them all at the very end in a "Gallery" section. This way a reader who is pressed for time can read the whole review without scrolling too much but a reader who wants to see more camera output can indulge. The photos should be presented with a CENTER tag.

Illustrations for a digital camera review: it is important that users be able to pull maximum resolution JPEGs (and sometimes RAW images) from our server in order to make comparisons for themselves. Rather than adding images to the photo.net image sharing service, collect them up in a ZIP file and arrange with our editors to transfer them directly into our file system where we can produce thumbnails and scripts that will make other sizes available to readers.

35mm point and shoot review

For a point and shoot, concentrate on the following issues:
  • maximum aperture (at both ends of the zoom range, if a zoom)
  • picture quality
  • viewfinder quality
  • manual controllability
  • size and weight (light and small is good; if we didn't mind a bit of size and weight we could take a Canon Rebel or similar plastic SLR)
  • weatherproofing
  • construction quality and ruggedness

35mm SLR Review

In reviewing a 35mm SLR your primary objective is to provide advice to people already committed to the underlying system. It is much more useful to compare the Canon EOS-3 to the Canon EOS-1 than to a Nikon body. photo.net is primarily targeted to people who own 5 lenses per body not for yuppies with a big SLR and one mid-range zoom. An exception to this rule is when reviewing "starter bodies", the cheapest and second cheapest bodies in a popular line.

The first paragraph should summarize the camera: price, year introduced (remember that we're going to keep this review on our server for the next 50+ years), autofocus system (or note lack), built-in flash or no, any special features, e.g., weatherproof.

What separates one 35mm SLR body from another?

  • viewfinder (100 percent coverage? big? bright? eye relief? easy to evaluate focus?)
  • one-wheel versus two-wheel control (for easy simultaneous control of aperture/shutter speed)
  • ease of doing simultaneous manual/auto focus
  • weather sealing
  • ability of autofocus system to find and stick with moving off-center subjects, especially important for sports photography
  • ease of getting a depth of field preview
  • maximum flash sync speed
  • ability to imprint exposure settings and date-time data between frames or store it up for subsequent download to a computer
Modern plastic cameras often come in slightly different variants in different countries. We are primarily a North American community so title the article with the North American name but we have enough readers in Europe and Asia that a parenthetical note right in the title with the Eurasian name is helpful. If there are important features available only in a Eurasian model, note that fact since North Americans will be able to buy these grey market from retailers in New York or possibly mail order from Hong Kong, Germany, etc.

Medium Format Camera Review

Our target reader for a medium format camera review is an experienced photographer who does not own any medium format camera currently. The assumptions of photographic experience and non-ownership imply some changes from a 35mm SLR review:
  1. inter-brand comparisons become more useful
  2. you won't need to explain terms as much; the reader is assumed to have used every exposure and AF mode on a modern 35mm SLR
Unless you're talking about a sub-$1000 camera, it isn't worth spending a whole lot of time talking about image quality. Except for lens flare, which is sometimes a problem with big lenses, any medium format camera should provide excellent image quality. There is a big market for 35mm equipment that sucks, e.g., $100 mid-range zoom lenses. But there really isn't any consumer market for medium format equipment. It is all professional photographers and amateurs with serious goals.

Start off your review with a summary paragraph. This should include the image size (6x6 or 6x17 or whatever), an indication of the camera's weight and size, a characterization of the automation level ("classic all manual" or "modern electronic"), and the price including a normal lens and 120 film back.

You can cover roughly the same issues as with a 35mm SLR (above). Give special attention to the areas where medium format cameras have traditionally been weak: weight/size, dependence on idiosyncratic rechargeable batteries, clunky attempts at automation, limited choice of lenses.

Generally with medium format camera systems we won't have separate reviews for each body and lens but rather one article covering the entire system. So include a separate H3 headlined section for each lens that you tried with an example photo gallery at the end (use CENTER tags to set off the photo gallery).

Large Format Camera Review

Cover the following issues:
  • price
  • size and weight
  • ease of packing/unpacking
  • availability of accessories (compare to the Sinar system)
  • precision of movements
  • guides to help set up tilts and swings
  • integration with digital backs and digital camera control
philg@mit.edu