There are many ways to create a mask in Photoshop, and even more ways to improve that mask. We've covered some of these before, but here we'll look at how to move seamlessly between selections and masks, using one to improve the other.
The content will define the strategy you need to use to get your basic selection. Sometimes the Quick Selection Tool will be the best option, sometimes Select > Color Range. Even the tools I love to hate, the Magic Wand Tool and the Magnetic Lasso Tool, can have a place. Mostly, it comes down to a few things:
If the object has crisp edges and is a different color to the background, use the Quick Selection Tool, or the Magic Wand Tool if that fails.
If the object has soft edges and is a different color to the background, use the Color Range Tool, or start by Command-clicking one of the Channels (R/G/B) in your image.
If the object is at least partly the same color as the background, you'll have to use the Pen Tool to define a clipping path, or paint a selection in Quick Mask or a Layer Mask. That's where it can get messy.
Cutting out this awesome birthday cake is a bit of a challenge '" the Quick Selection Tool makes a mess of it, there are too many colors for Color Range, and the edges are soft '" a mask would need to be manually painted.
If your mask is nearly there, one of the easiest ways to clean up a selection is to use Refine Edge. You can:
For a smoother look, you could use settings like these:
Alternatively, leave all those settings at 0, then:
A layer with a layer mask.
Now that you have a Layer Mask, you can paint the selection to make it good, or you can use the Dodge Tool (set to Highlights) and Burn Tool (set to Shadows) to make your job easier. This technique was discussed in an earlier article here so check it out now.
Dodge the Highlights, Burn the Shadows.
Even the best mask isn't perfect. Maybe part of it could be sharper, or maybe you need to punch a hole in it '" or reveal the transparent part of a glass. To tweak part of your mask, the easiest way is often to select part of your image with Channels. If you Command-click the Red, Green or Blue channel in your Channels panel, you'll load it as a selection. Select > Color Range... can also work well here if the areas you want to select are more complex.
Clicking just once on the back of the glass in the main image was enough to select it '" use Fuzziness to get it right.
Press Q to enter Quick Mask, converting your selection back into a temporary mask. In a Mask, white and black map to selected and not selected, with a grey range connecting the two smoothly. However, it's unlikely that the areas you want to change on your mask are all white.
Here's the selection viewed as a Quick Mask, with RGB hidden (and note, it's possible to invert this display if desired).
Press the eye next to the RGB channel to hide it, and you'll see your Quick Mask alone, as a greyscale image. Press Command-L to access old-school destructive Levels, then tweak to make the part of the image you want to adjust white and the rest black. Use Command-I to invert the mask if it's all wrong.
Here's the old-school Levels command '" only good for masks! '" in action.
Press Q again to exit Quick Mask, and you'll be able to use your new selection to limit your painting. Click on your Layer Mask, then paint with black or white through your selection. Black hides and white reveals your image, and you can press Command-H to hide the marching ants around the selection if they prove distracting.
With the glass interior selected, paint black on the existing Layer Mask to hide it.
Using the image data like this is ideal for fixing up any problems with your mask. Also, don't be afraid to invert your selection with Command-Shift-I if it's the wrong way round.
Masks are wonderful things, yet few masks are perfect out of the box. The more tools you have at your disposal, the better your chances of making a mask as good as you can.