Photographer and author, LaNola Stone, shares with us thoughts on being behind the lens while photographing children with this celebration of mothers of all kinds.
This article gives a brief introduction to the Jpeg (pronounced jay-peg)
lossy compression image storage and transmission format, and describes
some important aspects of its use. You do not need to understand all these issues
to use Jpeg, but some knowledge may help you create better images.
The default
quality setting on most
IJG
packages (including ImageMagick used on photo.net) is 75, which gives a good
tradeoff between file size and
perceived quality.
The default
chroma sub-sampling of 2:1 works well for the
majority of images (photo.net uses 1:1). There are known
errors and tradeoffs in the
compression
process.
Several previous attempts have tried to compare the Jpeg's produced by
Photoshop against those produced by IJG, but these are unsuccessful for various
reasons. First the
quantisation tables used by each package
are not the same, and each has different emphasis on visual perception. In
addition the
chroma sub-sampling used by Photoshop changes
from 1:1 to 2:1 half-way across the quality scale, without direct control by the
user.
The use of the
Jpegdumputility allows for
examination of the output of images from many different sources, without needing
to own the package that produced them.
Few of the available
sources and articles seem to answer
all the questions that come up often in
Q&A Forums, hence this attempt to
clarify as much as possible. I am not an expert, nor did I take part in the
development of Jpeg, so my discussion and conclusions are
empirical and subjective. Comments and feedback are welcome.
The most common questions are covered in the
Jpeg FAQ. Jpeg images
were first mentioned on this site in
Adding Images to Your Site by
Philip Greenspun in his 1998 book. There are also written sources (I have no
references for these).
More discussions can be found in the rec.photo.digital newsgroup, or by
searching
Google Groups (Usenet). My own
learning curve has spanned this same time-frame, but it seems the sources of
information are not clear enough to avoid repeated questions.
The aim of Jpeg compression is to take full-color (and gray-scale)
"real-world" scenes and reduce the file size of images for storage
and transmission. While capacity and bandwidth have improved dramatically over
the last decade, the increased size of images make Jpeg still relevant for
digital cameras users and websites.
The
Jpeg organisation is an umbrella body
that coordinates formats for encoding and compression of images. What is commonly
known as Jpeg is the JFIF (Jpeg File Interchange Format) implementation
(described by Eric Hamilton in 1992). This makes up the vast majority of images
viewed today by browsers on the world-wide-web.
This standard doesn't define exactly how to implement this process, but is
sufficiently wide that images from any program can be viewed. The most common
version in use is that produced by the
Independent Jpeg Group
or IJG (see below). Other implementations are the Pegasus Jpeg Library, and the
version used in Photoshop. These differ only on the encoding side of the
process.
Human visual perception comprises the eye and retina, plus the
processing done in the cortex of the brain. There are many shortcuts and tricks
used to create the feeling of accurate perception, but "optical illusions" dispel
this myth. The human eye is more sensitive to moving object and edges rather than
gradual transitions, and to red and green rather than than blue light. It is also
more sensitive to changes in brightness than colour.
In the diagram on the left various grey-scales A to E represent gradual
changes in brightness. These appear smooth and continuous to the eye, although
each is made up of less than 256 steps (B and D are not as smooth as C). This can
be seen after several Jpeg compressions are applied to create the right hand
image.
This perceptual bias is exploited in Jpeg (see below). Similar perceptual
biases are taken advantage of in television images, and MP3 audio coding.
Conventional methods of lossless compression such as Zip
reversibly reduce file sizes while preserving information by compacting
regularities in the data. Jpeg compression goes one step further, by organising
regularities in the
visual perception of an image
and using lossy compression to reduce the file size of the
image. This process involves a small but irreversible loss of
quality as discussed in the
errors below.
Edges in a typical image - zoomed in to see the pixels
After compression most of the edges are still present, with some artifacts
The actual method of Jpeg/JFIF compression is well described in this
tutorial by
imaging.org.
The main steps are as follows (some require heavy maths)
Standard colorspace is 256 levels of Red, Green, Blue (16.7 million RGB
colors)
Colorspace separation (YCbCr) from RGB
e.g. Y (luminance) = 0.299 * R + 0.587 * G + 0.114 *B
Spatial separation into 8X8 pixels blocks
Sub-sampling (if required) of chroma Cb and Cr (colors) in 16X16 pixel
blocks
Discrete Cosine Function (DCF) of the spatial frequencies in each 8X8
block
Quantisation of the spatial frequency matrix
Lossless compression of the resulting matrix
For illustrative purposes large images are not needed, since all of the
Jpeg compression takes place inside 8X8 (or 16X16) pixel blocks. Note that
a Jpeg cannot be compressed further using Zip or any other process of
lossless compression, since this is already done as the last step of the
Jpeg encoding.
Edges in a typical Jpeg image - split by red, green and blue channels
Note the predominance of green and blue pixels, with few red pixels
The green channel is closest to what the eye sees, with blue having next most
artifacts
Decoding an image from a Jpeg is the reverse of this process, and does not
need elaboration here.
Typically the only thing that the user can control in Jpeg compression is the
quality setting (and rarely the
chroma sub-sampling). The
value chosen is used in
quantisation stage above, where
less common values are discarded by using tables tuned to visual perception. This
reduces the amount of information while preserving the
perceived quality.
Chroma sub-sampling
settings are dealt with separately (below).
Ranges of quality settings differ in each implementation, but the
IJG values range from 99 (best) to 1 (worst). Please note that these
are not percentages, nor is there a direct correlation with the final
file size. The example at the
top of the page uses an
IJG quality setting of 50, and has a file size ratio of roughly 20:1. Anytime you
read that an image has been compressed with 10:1 Jpeg quality, you should know
that this is slightly misleading (see
digital cameras
below).
Jpeg is a discrete algorithm, and for a given quality setting different input
images may give widely differing file sizes. An image with lots of texture
and fine detail will produce a large Jpeg file, while one consisting only of blue
sky will be very small. Chosing an appropriate Jpeg quality setting is a
subjective decision, with no hard rules. I personally use
IJG
quality settings of 75 to 50 depending on the subject.
IJG compression settings comparison - zoomed in
There is a tradeoff between quality and file size, and smeared edges called
artifacts
The Photoshop scale from 12 to 1 covers the same range, but in differing
steps. The optimum PS quality settings for web images range from 6 to 3 depending
on subject. See
Jpegdump below for equivalent values and
the variation in
chroma sub-sampling.
Photoshop compression settings comparison - zoomed in
There is a slightly different tradeoff between quality and file size.
Picture Window 2.0 used the Pegasus Jpeg Library and has a scale from 100 to
1. This is totally different from the IJG settings, and typical PW quality
settings range from 95 to 85 (below that is not useful).
Picture Window compression settings comparison - zoomed in
This scale has a different set of tradeoffs between quality and file size
Chroma sub-sampling
Chroma sub-sampling works because human vision is relatively insensitive to
small areas of colour. It gives a significant reduction in file sizes, with
little loss of
perceived quality. Some important
exceptions are noted below. The default
IJG
chroma sub-sampling is 2:1, which can cause
perceptibleerrors in some images. Photo.net uses ImageMagick 1:1
chroma sub-sampling in most cases. The default for Photoshop varies by version,
and also by
quality setting, as indicated by
Jpegdump below.
For historical reasons the standard 1:1 chroma sampling is sometimes called
4:4:4 and the lower quality 2:1 sub-sampling is called 4:1:1. The 4:2:2 option is
an old standard from the television industry which is seldom used. One exception
seems to be
consumer digicams which use the 4:2:2
sub-sampling.
Note how the color edges smear out, these are also artifacts
Digital Cameras
Some characteristics of consumer digital cameras are particularly relevant to
this article. The current generation have a single CCD (or CMOS) chip with R, G
& B colour filters arranged in what is called the Bayer pattern. This allows
interpolation to give an approximation of full-colour for each pixel, from an
array of 8-bit grey-scale values. (This is quite different from scanners and
high-end digital cameras which have 3 separate RGB sensors). These interpolations
create artefacts called fringing on the edges of colour areas (analagous to
chroma sub-sampling).
A test of
Jpegdump on the output from a 1.3 mega-pixel
digicam gives interesting results. The Jpeg file sizes vary with the image
content, but for this camera the specifications are as follows:
SHQ mode File size 440KB (max) and HQ mode File size 220KB (max)
For an 1280x960 pixel image these are equivalent to compression ratios of 8:1 and
16:1 respectively. The "super-high-quality" mode gives an
IJG
equivalent
quality setting of about 93, and the
"medium-quality" mode gives an equivalent of 75-80 for typical scenes. These are
consistent with suggested settings for use on a website (and give good detail in
4X6" prints from the Jpeg).
The Jpeg chroma sub-sampling is the 4:2:2 setting (not the default used in IJG
or any other package). This may be because the Bayer pattern interpolation
already introduces a 2:1 horizontal sub-sampling (similar to that in a television
picture). This gives a small space saving, and using a 1:1 chroma sub-sampling
would probably be of little benefit. Re-saving these images from a digicam as IJG
2:1 chroma sub-sampling introduces a small but detectable loss.
The decoding stage is well implemented, and there are no issues that warrant
discussion here. Older computers with 16-bit color display are an exception. IE4
was the only browser that catered to them, by dithering the output image.
An existing bug in many browers caused crashes on Jpeg images created with the
File Save-As option in Photoshop 7.0 (embedding excessive amounts of
information). The fix was to upgrade to 7.01, or use the Save for Web option
instead.
By definition every Jpeg contains imaging errors no matter how high the
quality setting, and should not be used for master-archiving. The main concerns
relate to subjective
perceived quality, and the file
size tradeoffs in the original compression. There is no hard-line where
errors become perceptible, it is a matter of judgement.
The Jpeg compression process introduces more artifacts at lower
quality settings. These become particularly visible near
edges, and in areas of solid colour where boundaries of 8X8
pixel blocks can show irregularities. Use of Jpeg images as input
for editing can be problematic, particularly since resizing/resampling may move
the boundaries of the 8X8 pixel blocks.
Repeated cycles of compression and de-compresison don't normally affect the
quality, unless the
quality setting is changed between
cycles. Once information has been lost it cannot be recovered from a Jpeg, except
by starting the process from the original bitmap. Trying to improve the
appearance of a Jpeg image by re-compressing at a higher quality setting achieves
very little (except an increase in file-size).
Repeated compression and decompression cycles in steps
Chroma sub-sampling errors
Note the edges and artifacts that were never in the original scene Significant
errors can occur when there is chroma (colour) detail at the pixel level,
if the
IJG 2:1 default
chroma
sub-sampling is used. This blurs 2X2 blocks of saturated pixels into pale
colours (red becomes pink). Fortunately this is rare (a tiny fraction of photos
are affected), but happens even at high quality IJG
quality
settings. (PhotoShop
chroma is different).
Note how some red and orange pixels become blurred to form pink ones
The
Independent Jpeg Group developed and
licenced a free set of utilities for Jpeg compression and de-compression.
You don't need these directly since they are included in almost every image
editor and web browser. You may see credits such as: "Portions of this software
are based in part on the work of the Independent JPEG Group".
Note that Paint Shop Pro uses the IJG utilites, but reverses the scale (in PSP
1 is best quality and 99 is worst).
Jpegdump
A utility written by Allan Hessenflow (with modifications by Tom Lane)
allows examination of actual Jpeg data. It has outputs that estimate the
IJGquality settings for any given Jpeg. When used
together with an
IJGcompression utility
this allows
almost exact recreation of a Jpeg. The only
versions of this utility that I know of are source code and command line
executables, so they are unlikely to be of direct use to the general public. It
would be most useful for Jpeg writing software to list the prior quality level so
you could rewrite (if necessary) at the same level.
JpegDump 96X64_95.JPG
Approximate quality factor for qtable 0: 95 (scale 9.93, var 1.25)
Approximate quality factor for qtable 1: 95 (scale 10.18, var 0.26)
Baseline JPEG width 96, height 64 components 3
id 1 horizontal sampling 1, vertical sampling 2, quantization table 0
id 2 horizontal sampling 1, vertical sampling 1, quantization table 1
id 3 horizontal sampling 1, vertical sampling 1, quantization table 1
Photoshop Jpeg's do not use the IJG tables, so their equivalent quality can
only be estimated.
Thanks to Bill Tuthill
for helpful input and contributions.
Glossary:
Bitmap - the uncompressed image, with file size = H x W x
3 (for color) Cumulative - opening and saving the same Jpeg with different quality
settings Diff ^2 - the arithmetic difference between two bitmaps (squared
for emphasis) Dimensions - the height and width of an image File-Size - either the dimensions of an image, or the number of
bytes stored or transmitted Full-colour - 24 bit (RGB) images with 16.7 million colours Image - an array of pixels making up a digital picture Lossless Compression - a reversible way of reducing the size of data while
preserving it Lossy Compression - an irreversible way of reducing the size of data by
approximating it Multiple - opening and closing the same Jpeg several times with the same
setting Pixel - picture element or smallest component of a digital
image Utility - a set of source code or complied program module Zip - a commonly used lossless compression format
Thank you so much for this wonderfully informative article. My scanning skills are not the best and your article addressed some issues that I've been unclear about for almost three years now : )
I intentionally omitted any discussion of Jpeg 2000 in this article, because in reality the advantages do not match up to the marketing hype, and the number of images and systems using it have not reached the critical mass necessary to take it seriously.
it's barely supported by the big guys; it has too many features, IMO (which makes decoding and even encoding tricky for developers and a hassle for users - too many options makes it likely that the options you chosose won't be supported in all readers); the results aren't that much better than plain JPG, which is supported everywhere, without trouble. J2K is 10 years too late, IMO. the last thing the world needs right now is yet another format.
that said, some of our applications (www.smalleranimals.com) do support J2K, but not because any customer has ever asked for it. we only support it because it has buzzword value.
At the compression ratios typically used by digital photography, JPEG 2000 offers almost no advantages over JPEG. It's only when compressing more than about 20:1 that the differences become significant. Here is an interesting discussion of some the the visual differences: http://ai.fri.uni-lj.si/~aleks/jpeg/artifacts.htm
JPEG2000 does support 16-bit per channel images. I don't know whether the old JPEG format supports that theoretically, but nobody supports it in practice. In contrast, all the JPEG2000 implementations that I have seen support 16-bit channels.
JPEG2000 also supports lossless compression in a standard way. If JPEG2000 leads to digital cameras that support 16bit, lossless compressed images, then that should be a good thing.
The article on Jpeg2000 by Aleks Jakulin uses the notation bits-per-pixel, which is useful when comparing different images. For a full-colour (24 bit) image a compression ratio of 20:1 equals 24/20 or 1.2 bits-per-pixel.
The Jpeg values given for the digital camera listed above correspond to 1.5 and 3 bits-per-pixel respectively (24/16 and 24/8). The Exif headers from the digicam Jpeg's give approximate bits-per-pixel of 1 and 2 for HQ and SHQ modes respectively.
The number of bits-per-pixel depends on the image content, so this rule should not be generalised. For very small images the Jpeg header size makes this is not an exact measure.
For the photo at the top of this article the bitmap size is 200*134*3 or 80400 bytes, and the Jpeg file-size of 3672 bytes represents 3672*8/26800 or 1.1 bits-per-pixel. This is near the lower end of the acceptable quality range (intentional on my part).
IJG compression produces the following values:
Q 25 2,594 bytes = 0.8 bits-per-pixel (30:1 compression ratio)
Q 35 3,278 bytes = 1.0 bits-per-pixel (25:1)
Q 50 4,086 bytes = 1.2 bits-per-pixel (20:1)
Q 65 5,234 bytes = 1.5 bits-per-pixel (15:1)
Q 75 6,393 bytes = 1.9 bits-per-pixel (12:1)
Q 85 8,810 bytes = 2.6 bits-per-pixel (9:1)
Q 95 15,162 bytes = 4.5 bits-per-pixel (5:1)
The IJG Q 95 Jpeg is nearly double the size of the IJG Q 85, with little benfit in perceptible quality.
The color count of an image is another measure of image content, though not an exact one (and not useful for B&W which has only 256 shades of grey). Editing functions such as resampling and sharpening can have significant effects on the pixels in an image, and increase or decrease the final Jpeg size.
Yes, JPEG 2000 supports 16 bpp. It's debateable how useful this is for a lossy compression scheme that does color subsampling. (If your application is critical enough to depend on the difference between 8 and 16 bpp, are you really going to use a lossy compression scheme at all?)
Lossless JPEG exists is even used by the Canon 1D & 1Ds for their RAW format compression. Even better is JPEG-LS. It's standard, runs at about the same speed as LZW, and is fairly symmetric (compression and decompression are about the same speed.) It offers slightly better compression than PNG (9.54 bit per pixel as opposed to 10.38 bpp on one test set).
Yet, despite the existance of "better" formats, lossy JPEG and lossless TIFF (uncompressed or LZW) still dominate for photographic applications. Why? Well, they're good enough, fast, and widely supported. Given all of that, merely "better" is not enough, partcularly when the alternative is neither fast nor widely supported. (Of these, "widely supported" is the most critical factor. Who wants to store their photographs in a format that may be difficult to read in 20 years. FlashPix, anyone?) About the only format type that has become significantly more popular in recent years is the digital camera RAW format. Alas, it's not one format but dozens and many worry that these formats are not documented and may become obsolete.
Erik Magnuson's observation "...many worry that these formats are not documented and may become obsolete..." strikes me as an obvious and solvable problem but which requires a significant commitment of time, money and resources to solve. Would Adobe or filext.com be the appropriate long-term holder for this information? What file type does NASA use? Does IEEE or some other organization already have this for action? Does anyone out there have the inside scoop? If so, please post.
AFAIK, there none of these formats has been proposed for standardization. The impetus for this would have to come from the camera vendors. However all is not lost: Dave Coffin has reverse engineered many of these formats and made the source code freely available. Thousands of people have a copy of this code, so it should find a home on the internet for a long time to come. Or you can just download a copy of the source for yourself and archive it with your photos. Even if you are not enough of a computer geek to compile it yourself, I'm sure that finding someone else who is geeky enough to compile or translate 'C' code will still be possible in 20 years or more.
At the compression ratios typically used by digital photography,
JPEG 2000 offers almost no advantages over JPEG.
As CCDs and CMOS censors ramp up in resolution, JPEG2000 will become
increasingly important. Digital imaging software and cameras also will increasingly
support it. Its day will come.
Note that a Jpeg cannot be compressed further using Zip or any other
process of lossless compression, since this is already done as the
last step of the Jpeg encoding.
This is generally the case and correct for practical purposes,
but I thought I would point out that from a theoretical standpoint,
it is not really true. Unless the lossless compression done in the
last stage of the algorithm produced encodings with zero stochastic
redundancy between the bits, then it still would allow for further
compression, although Zip or other compressors one is likely to have
available likely would not have the ability to squeeze the last bit
of redundancy out to compress further.
The Jpeg compression in Paint Shop Pro version 8 is based on different tables from the IJG ones which were used in all previous versions. The usable settings for web purposes range from roughly from 25 to 45, with settings of 50 and beyond being of very low
quality (lesser range than IJG, and no longer 100-Q scale). The tables are much the same as IJG up to setting 25, but diverge beyond that point.
PSP 8 has a large variety of chroma sub-sampling options. Many of these are obscure, so avoid using any setting you do not understand. A typical user is likely to use 1X1 (no sub-sampling or higher quality), or 2X2 (the standard quality default in previous
versions). The marginal file-size saving from 4X2 sub-sampling is not justified by the loss in quality.
8 bits per channel is not enough if you plan to do some some fine touch up on the image. If you use a software that uses 8 bits per channel it will round the results of all your actions (modifycations) to the picture and give you an increasingly in acurate result compared to using 16 bits per channel.
The movie industry has allready noticed this and uses software like film gimp and others that use 16 bit per channel for this very reason.
I for one would very much like to get as much information from image as I can so postprocessing will be more acurate.
Although repeated compression and decompression of Jpeg's is NOT recommended, the IJG library produces mininmal degradation if the same quality setting is used.
I came across a thread on Jasc User Forums (http://forums.jasc.com/)
which indicates that PSP 8 has a bug when repeatedly saving and re-opening the same Jpeg. Paint Shop Pro 8 uses a new Jpeg library with different settings from IJG. After 10 saves the artefacts are quite noticeable (in areas of sky).
Just thought I'd add this diagram to illustrate how DCTs are organized for one JPEG implementation. Note for these purposes, YUV is same or similar to YCrCb.
The combination of editing and Jpeg recompression produces some additional artefacts, and a useful test (in a reversible way) was suggested by John P Sheehy. Shifting an image 1 pixel down and to the right disrupts the 8X8 blocks in Jpeg quantisation, and forces recompression, with cumulative artefacts and errors.
In this example the colour saturation is significantly reduced with only a few iterations, and the result is a seriously degraded image. I assume that this situation is not common, but does indicate that editing combined with Jpeg compression is undesirable.
The question of Jpeg recompression comes up frequently on photo.net. This example simulates what occurs when an image compressed with Photoshop (V5.5) is submitted to photo.net, which then uses IJG 75 (chroma 1X1) to recompress the image (if certain size limits are exceeded).
The combination of medium-low quality Photoshop compression (PS Q6) with photo.net recompression produces slightly worse results, since Jpeg artefacts are cumulative. If the uploaded image uses reasonably high quality compression (PS Q8-12), slightly better results are produced. (This is contrary to previous comments with suggested
the opposite).
Chroma subsampling is also important, but has become less of an issue since photo.net changed to 1X1 subsampling (more than 2 years ago).
In my website, about clouds classification, I inserted photos with resolution 125 dpi and quality factor 15 in a scale of 2 to 255. It is manipulated in the Corel PhotoPaint program, version 5.0. I utilise the option "4:4:4". Which means sub-sampling chroma?
Within the professional cinematography, video, and imaging industries, chroma subsampling is expressed using the Y:Cr:Cb notation. This notation was originally devised to convey the horizontal (not vertical) resolution ratios of the Cr (red) and Cb (blue) channels to the horizontal resolution of Y (luminance). 4:4:4, therefore, indicates the absence of horizontal subsampling (1:1), while the 4:2:2 and 4:1:1 formats indicate that the Cr and Cb channels each horizontally boast 50% (4:2:2), and 25% (4:1:1), of Y's horizontal resolution.
While less common, a 4:2:1 format also exists, yielding 2:1 horizontal Cr resolution vs. Y, and 4:1 horizontal Cb resolution vs. Y. This exploits the lower sensitivity to blue than red definition in human vision.
Unfortunately, the Y:Cr:Cb notation often causes confusion because it was "hacked" after its introduction to express vertical subsampling levels in addition to horizontal ones. For example, while 4:2:0 seems to suggest 2:1 Cr vs. Y horizontally and the total lack of Cb, it actually indicates that both Cr and Cb are present, and that each is subsampled 2:1 vs. Y horizontally *and* vertically. The reason this "makes sense" is because, in vertical subsampling, only one of the two chroma subchannels is sampled per horizontal line (co-siting). Therefore, scanline 1 contains only Cr at 50% of Y's horizontal resolution (4:2:0), while scanline 2 contains only Cb at 50% of Y's horizontal resolution (4:0:2), etc. Upon decoding, Cr and Cb are vertically interpolated to form a vertical average. But in encoded form, it's literally "4:2:0".
The 4:1:0 format uses the same technique, whereby Cr and Cb are alternately present (2:1 averaged vertical chroma resolution), and each boasts 25% (4:1) of the horizontal resolution of Y.
4:4:4 - High-end imaging/cinematography processing, high quality JPEG (used by Photoshop's higher quality levels)
4:2:2 - D-1, D-5, D-9 (a.k.a. Digital-S), Ampex DCT, Digital Betacam,
Betacam SX, DVCPRO-50, studio/high profile MPEG-2, good quality JPEG
4:2:0 - PAL DV, PAL DVCAM, main profile MPEG-2 (e.g. DVD, American SDTV and HDTV, SD and HD DVB), average quality JPEG (used by Photoshop's lower quality levels)
As I haven't found any Windows-based utilities (besides old command-line variants) that do the full JPEG decoding and reporting of the JFIF markers (quantization tables, compression settings/quality, chroma subsampling, etc.), I wrote my own simple tool:
On rec.photo.digital, Martin Brown pointed out that Bayer-pattern
digicams use 4:2:2 (2x1) chroma subsampling because there isn't more
information to encode from their alternating GB/GR pixels, anyway.
The popular free program IrfanView recently added a plugin that allows you to optimize Jpegs for a given size. (Radical Image Optimisation Tool or RIOT can also be downloaded as a standalone Exe.) This is done by iterating to change the compression factor until a given file size is achived. There is no shortcut, so for large files it takes a while.