Intro Image: To create this image, I first shot the transparent flower on a white background and then used the techniques explained in this article to mask the photo and place it on a black background.
Imagine a Photoshop world without selecting. See if you really can. No ability to select would mean no way to work on parts of an image. You’d be forced to make modifications to an entire image—or not at all.
Obviously, the point of Photoshop is flexibility, and you do need to be able work on portions of an image without impacting the whole thing. You are probably aware that in our Photoshop world there are numerous tools available to help you select. These include a veritable battery of lasso and marquee selection tools, the Pen Tool, as well as the ability to select by color and color range.
It’s important to understand that selecting is comparable to creating a mask for a layer. Both selections and masks are stored as alpha channels. With a selection you can create a mask, and vice versa.
This equivalence being the case, experienced denizens of the Photoshop world tend to prefer to work with layer masks over selections. There’s a great deal more flexibility in how you can work with a mask than with a selection. For example, the Brush and Gradient Tools give you granular control over the boundary areas of a mask; whereas, with a selection, you can use feathering for edge control, but the options clearly lack finesse. When you are using a layer mask and have “gone too far” it is totally trivial to back off and revise your work—not always the case with a selection.
Whether you are working with layer masks or selections, the key is to work “smarter not harder.” Sometimes there is a simple and obvious way to isolate and area of an image, but other times there is not—and complex edge drawing to create a selection or a layer mask takes a great deal of time and may not produce satisfactory results when all is said and done.
The challenge is to see if you can isolate a channel of your image that can be used to get you most of the way to your desired mask. Between the RGB and LAB color spaces, most likely you will be able to isolate a channel with most of the desired information for your layer mask. (See my column LAB Color Adjustments for more information about the LAB color space.) This channel may need a little tweaking, but you can then use it as a (mostly) automated layer mask via the Image > Apply Image command.
Image > Apply Image is the simplest of the cross-channel operations, sometimes referred to by the acronym CHOPS. Being able to apply a channel as a layer mask is tremendously powerful, and will take you a long way towards mastering the Photoshop beast.
It’s Better to Receive
Before I get started with the nitty-gritty of how to use the Apply Image dialog, here’s the one concept that you need to understand. If you don’t get this—as I didn’t when I first tried to use the Apply Image dialog—it will trip you up when you try to use this advanced Photoshop feature. Think about your Source and your Target. The Source is the image that will be used to create a layer or layer mask. This needs to be an image that is open in Photoshop.
The Target is the destination where the applied image goes. This must be the image in the active window or tab in Photoshop. The confusing thing is that the Target—the image that will receive the applied image—is never named, it is merely implied and assumed to be the active file.
If you bear in mind the distinction between Source and Target—and understand that the Target is always the active image—then you will find Image > Apply Image a much less confusing command than it would be otherwise.
Note that both the Source and the Target images must be the same size to use Image > Apply Image. They also need to be the same bit-depth.
As an example, I’ll show you how to create a layer mask for the photo of an Allium blossom shown in Figure 2. This flower is a relative of the onion. I photographed it on a white background for transparency. Note that it would be difficult to the point of impossibility to select this flower—while leaving the white background within the flower unselected—by “drawing” manually around the tiny blossoms that make up the flower. However, an alternative selection strategy that might be successful would be to select white as a color, and then invert the selection.
Figure 2: Complex shapes like this large Allium blossom photographed on white are hard to select or mask manually, making this a good candidate for Image > Apply Image.
My idea is to place a masked version of the Allium on the background shown in Figure 3. Actually, from the viewpoint of understanding the technique the background doesn’t really matter—it could be a solid color, or anything else for that matter. We are primarily concerned with the layer that needs masking.
The visual concept is to add richness and depth to the foreground flower image by masking it so the dark, but multi-colored, flower background comes through the semi-transparent flower on top.
Figure 3: I created this background for the Allium; however, the eventual background layer in an applied image situation is not all that important to understanding how the operation works.
Preparing the Image
To get started, I first duplicate the Allium image. I’ll use this version to create the actual layer mask that is to be applied, so logically enough I name it Duplicate for Mask.
Next, I’ll copy the original Allium layer over my desired background to create the stack shown in the Layers palette in Figure 4. This layered image will be my Target.
As I’ve noted, to use this technique it doesn’t really matter that much what the background is—but obviously you should plan an effective visual combination, or what is the point of masking?
Figure 4: The layer I want to mask is on top of the background.
The next step is to add a layer mask to the Allium layer. In Figure 5, I’ve added a Reveal All layer mask by choosing Layer > Layer Mask > Reveal All. Before opening the Apply Image dialog, this layer mask needs to be selected because, as I’ve mentioned, the source has to be active for Image Apply Image to work as you expect.
Figure 5: A Reveal All (white) layer mask has been added to the Allium layer.
By the way, the Apply Image dialog is very useful for moving layers from one image to another and not just for creating layer masks—however, these non-masking operations are beyond the scope of this article. But, by all means experiment! You’ll be amazed at what this seemingly modest dialog can accomplish.
Preparing a Mask
Stepping back for a minute, it is time to prepare the image that will be used to create the mask (the Source). You’ll remember that I duplicated the original image, naming the copy Duplicate for Mask. There are many ways to go about this, and a great deal depends upon the situation. The general concept is to inspect the channels associated with the image, and to start with the channel that is closest in shape to the mask you’d like to create.
The task is made easier because you have a number of color spaces to choose from, and therefore a number of channels. In addition to the three RGB channels, and the four CMYK channels, there are the three LAB channels. The L channel in the LAB color space turns out to be particularly useful for creating layer masks.
To create a mask using the L channel, first convert the image to LAB by choosing Image > Mode > LAB Color. Next, in the Channels palette, click the eyeball icons to the left of the channels until only the Lightness channel is visible (Figure 6). At this point, the single channel should begin to look like the mask you are trying to create.
Figure 6: Only the Lightness channel is visible.
The Apply Image dialog gives you the flexibility to choose a specific channel as your Source, if you choose, rather than the entire set of channels. However, it’s sometimes easier to convert the source image back to RGB so you can use normal adjustments on the Source before applying. To achieve this conversion, you must first drop the channels you don’t want, by dragging the A and B channels to the trashcan in the Channels palette. The image now consists of a single Alpha channel, as you can see in Figure 7.
Figure 7: By dropping the A and B channels in LAB, the Lightness channel becomes an alpha channel in a multichannel image (rather than in the LAB color space).
If you look at the Image > Mode menu, you’ll find that Multichannel is now checked. Converting the image back to RGB is a two-step operation. To convert this image back to RGB first you need to convert it to Grayscale by choosing Image > Mode > Grayscale. Next, choose Image > Mode > RGB. It’s now a regular, happy three-channel RGB file—although it will look pretty much the same no matter what the mode.
Using the Threshold Command
Most masks generated using a channel need to be tweaked. In some cases this means a bit of hand painting or cloning. Whether or not your mask needs this kind of hand work depends upon the situation—but it is quite likely that you will want to make whites lighter or blacks darker. Making the whites whiter lets the image on the layer associated with the mask come through completely like a white Reveal All layer mask does. On the other hand, making the blacks darker hides the pixels on the associated layer like a black Hide All layer mask.
The Threshold adjustment is an easy way to accomplish increasing the contrast range in the mask. In Figure 8, I’ve altered the white area so the mask is more completely transparent by choosing Image > Adjust > Threshold, and then moving the Threshold Level to the left.
Figure 8: The Threshold adjustment can be used to increase contrast so that the image is usable as a mask.
By the way, there are alternative adjustments besides Threshold that can be used to achieve your goals with the layer mask. Levels and Curves are both effective adjustments to achieve the same kind of results as Threshold, although applying either takes a bit of finesse while Threshold is a completely straightforward adjustment.
Combining the Image with the Mask
Now it’s time to rock and roll! Minimize the Source image to keep it available but out of the way. Click the Target image to make it active, and select the Reveal All layer mask in the Layers palette. Choose Image > Apply Image. The Apply Image dialog, shown in Figure 9, will be open.
Figure 9: When the white Layer mask in the Target image is selected, the Apply Image dialog lets you blend the Target image with the layer mask.
The first step in the Apply Image dialog is to select the Source from the drop-down list. Even if only two images are open in Photoshop, you need to check that the correct Source image is selected because the Target will also show up on this list. (Of course, sometimes you want Source and Target to be the same!)
Next, choose the Layer to use. In the case of the Duplicate for Mask image there is only the one Background layer, so no choices here. Finally, you can choose the entire image or a specific channel.
If you have Preview checked, you should be able to see the impact of your choices. In the relatively simple example I’ve shown, I’m finished with the Image Apply Image operation, so I click OK and the new layer mask is shown in thumbnail in the Channels palette.
Note that there are quite a few options to play with in the full Apply Image dialog, shown in Figure 10.
Figure 10: You can use the Apply Image dialog to apply a mask (rather than a layer).
In addition to choosing Source, Layer, and Channel, you can choose to invert the source image, change the blending mode with which it is combined with the Target, and change the Opacity with which it is combined. If you check the Mask box, you can also apply a mask (an alpha channel) rather than a normal color channel. My reaction to all these is like being confronted with a box of goodies: go forth and play, and see what all these wonderful options do in specific situations! I’m sure you’ll find some interesting choices.
It’s worth taking a look at the actual mask created using the Apply Image dialog. You can see this in the Channels palette as shown in Figure 11 by making the alpha channel the only layer visible.
Figure 11: It’s easy to take a look at the new mask to make sure it is what you want.
As you can see, the mask is pretty complex and does the trick in creating a composite image on a textured background (Figure 12). This would be an extremely difficult layer mask to create using manual techniques due to the complexity of the patterns, the fine lines of some of the little flower buds, and the relative lack of contrast of parts of the flower and white background in the original image.
Figure 12: The Allium flower is shown masked on the new background.
Figure 13: I used a layer mask creating using the Image > Apply Image command to composite this figure study with a forest scene.
Using layer masks—the more flexible version of selecting—lies at the heart of the venerable old Kingdom of Photoshop. With a proper strategy, cross-channel operations can be an easy way to create a complex layer mask without much—if any—hand work. In particular, using a LAB color duplicate of an image to generate a layer mask based on the luminance information in the Lightness channel presents a great opportunity.
The image that is created using the L-channel can be massaged using a variety of adjustments, including Threshold and Levels. Once the mask is fully ready, it is easy to use the Apply Image dialog to add it to an active layer mask.
While cross-channel operations are sometimes feared as arcane and inscrutable, if you follow the ideas presented in this column and use them as a starting place for your own experimentation, you’ll find you don’t have much to fear—and you’ll gain a great deal of power over the Photoshop beast!
This article explained:
The conceptual equivalence of selecting and using a layer mask
Introduced the concept of CHOPS—Cross-channel operations
Understanding Source and Target in an Image Apply Image operation
Harold Davis is a photographer and author. His photographs have been widely published, exhibited, and collected. Many of his fine art photography posters are well known. Harold’s images have won a Silver Award in the International Aperture Awards 2008 competition, and inclusion in the 2009 North American Nature Photography Association Expressions Showcase.
Harold is the author of The Photoshop Darkroom: Creative Digital Post-Processing (Focal), Creative Composition: Digital Photography Tips & Techniques (Wiley), Creative Night: Digital Photography Tips & Techniques (Wiley), Creative Close-Ups: Digital Photography Tips & Techniques (Wiley), Practical Artistry: Light & Exposure for Digital Photographers (O’Reilly Digital Media) and other books. Harold gives frequent digital photography workshops, many under the auspices of the Point Reyes National Seashore Association.