Selection to Mask Workflow

From Photoshop
Revision as of 13:34, 2 July 2016 by WikiSysopPS (talk | contribs) (Overview)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Selecting is a key aspect of the compositing workflow and one that is not without a good deal of complexity. There are many selection tools, many reasons to select and many things you should know about it. In the compositing skill-set selection is most vital when it is used in order to separate a figure or object from its background and it is in this context that I shall adress it. If you get good at this then all other selecting tasks shall seem trivial.


The selection to mask workflow is, in principle, simple enough. Here is the general workflow:

Stage Task
Review Review the image to determine what region needs to be masked, and if this region is naturally defined by a single color.
Arrow down.png
Select Select the region.
Arrow down.png
Make the mask Press the mask button. This will automatically make a mask based on the selection.
Arrow down.png
Edit non-bordering regions The mask that you have just created will almost certainly require editing: refining and correcting.

There are separate process for removing junk, editing soft and hard masks and painting.

Edit hard edges
Edit soft edges
Edit by painting


At this stage, you should review the image to determine exactly where the mask is to be located. Is the object, or its background, defined by a single flat color? If yes, then you are in luck. This will make the whole process easier.


At this stage, you simply make the selection. This you do using one of the selection tools. For a hard edge object the Quick Selection tool is sufficient. For regular objects like boxes, buildings etc the Polygonal Lasso Tool is a good choice. For things like hair the Select / Color Range is the only option. Using the Color Range is simple, but also critical.

Here the sampled color is the greenish background. The selected region is visualized as white in the preview. The 'Fuzziness' value is too low for this image, and the selected region is very 'noisy'.
Here the 'Fuzziness' value is very high. See how the edge of the face is now selected. This over-selection would create problems later on.
Here the 'Fuzziness' value is as good as it can be at this stage. The face is not overly selected. However, the region behind the boy's shoulder is dark, indicating the it is under-selected .
Here the 'Add to Sample' button has been pressed, and clicked on the area behind the shoulder. This adds to the selection, improving the mask. The 'OK' button may now be pressed.

Make mask

With the selection made, press the Add Layer Mask button at the bottom of the Layers Palette. This will make a Layer Mask from the selection. Its possible that the mask will now require inverting. Simply 'Command i' whilst in the layer mask.

The 'Add Layer Mask' at the bottom on the Layers Pallet.

Its possible that you now have the perfect mask. However, most likely, the mask will require editing. To get a better idea of how good your mask is, 'Alt click' on the mask in the Layers Pallet. This will show the mask in the window. In this case, the face needs some editing.

'Alt click' on the mask in the layers Pallet to reveal the mask in the window.

To 'stress test' the mask, press 'Command L' to summon the Levels adjustment. Move the middle slider (the 'gamma') to either side. This reveals any flaws in the mask. This temporary adjustment is only for testing purposes. Do not press the 'OK' button. This test is known as a 'Gamma Slam'. Moving the gamma slider to the left will test the black of the mask, and to the right the white of the mask. In this case, the background looks a bit noisy... however, the background is probably good enough to require no more work.

A gamma slam showing up the noise in the mask.

Edit the mask

Having made the preliminary mask you now need to get rid of the obvious noise and imperfections. The white 'keep' area of the mask is called the holdout and the black 'remove' section is called the junk. Both might require attention.

Edit non-bordering regions

Editing regions of the mask that do not include any border is simple. Just painting them out with a brush. In the case below, the girls face does not include any border.

The imperfections in the mask of the girl's face do not include any border and can easily be painted out.

Edit hard edges

For refining a hard edge object, Clamping may be employed.

Here the edge of the boy's shoulder is showing a white fringe when placed against a black background (n.b. it might look ok against a white background).
To make the clamp work more effectively, a slight blur has been applied to the edge using the Blur Tool.
Here a Levels adjustment has been applied. By moving the sliders around, the edge can be softened, hardened and/or moved.

Edit soft edges

In the masking of a figure, the hair will require a very soft mask. The problem is that any severe adjustment to a mask is likely to damage these soft regions. Therefore, in the example case of the image of the boy, any edit to the hard-edged cloth will damage the hair.

There is a way of combining a junk and holdout adjustment in such a way as to leave the bordering edges untouched. However, it is for advanced users. Details are here: Dilate Erode Combine.

For newbies, the simplest approach is to first make a 'pre-selection' using a Lasso or Marquee. This should encompass only the hard-edged regions. By doing so, any edit made to the mask will not include its soft-edged regions.

Edit by painting

For painting into tight regions of the mask, the cutting-in method may be used:

This hand is to be masked out manually. The sharp edge folds in the cloth and the small negative shapes are very hard to paint inside of.
Here is a poorly painted mask (junk mask shown as green). Its edges are very 'rounded'. The temptation is to refine the mask using a tiny brush (laborious and inaccurate).
The solution is to employ the 'cutting in'® method. Stage one: paint-out with a black, ignoring the fact that you are overpainting.
Stage two: cut-in with white:.
Stage three: bake until cooked.

Layer Clipping and Masking

In the example below, the hair of the masked girl has a white fringe. This cannot be fixed using a clamp as it would damage the softness of the hair. Instead it is fixed after the mask process is complete. This is done by first clipping a layer to the masked layer. Then, using a Clone Tool, the hair is patched. Remember to set the 'clone sample mode' to 'Current & Below'.

Here the edge of the girl's hair is showing a white fringe when placed against a black background (n.b. it might look perfect ok against a white background).
On a clipped layer, some hair is patched onto the white fringe.

Miscellaneous Observations

The following are some miscellaneous observation reading the art and craft of masking.

Masking as drawing

Masking is a drawing activity. Organic shapes in particular can be subject to a lot of creative refinement in the process of masking. In the before (left) and after (right) below the shape of the girl has been clarified and the unfortunate wiggly pattern on the outline of her right sleeve has been removed.

Selecting 05.png

Where are you?

Whilst masking, be aware of where you are. After having made the mask from a selection you will be taken automatically to the mask component of the layer (B). Any painting or filling will be done to the mask. If you accidentally navigate to the layer contents component then you may well accidentally painting into the layer contents component of the layer (A). This is a BAD thing. Option Click on the mask to get into and out of it.

Selecting 01.png

See also


Clamp, Dilate and Erode

Taking a Photo

Quick Mask Mode