There have been a few people who have stated that they don't really know or understand layer masks and thus never use them. I have to say, layer masks are an incredible technique to add to your mapping arsenal.
I hope with this short tutorial to provide at least one person with a new respect for layer masks. If you get something useful from this tutorial, please remember to rate the thread. Likewise, if anyone has any additional tips or tricks I did not include, please feel free to include them in this thread for others to learn from.
Now, on with the tutorial.
The first question is "what is a layer mask": Layer masks are adjustments which are attached to a layer that affect the visibility or the layer.
Second question is: "why would I use a layer mask": To maintain flexibility!
Third question is "how do layer masks work": A layer mask is a gray scale image in which white indicates 0% opacity and black indicates 100% opacity. Any shade of gray in between black and white is the amount of the image that will show though.
So, let's start with a few exercises. First, we will begin with the typical background image and use a layer mask to show a section of a different color.
(yea, not to terribly exciting).
The typical way someone who does not use masks would do this would be to make a selection with the circle section tool and fill with the secondary color. Now, this works, but, if you need to edit either color, you have a lot of extra work to do and unneeded extra work is bad.
So, what about using layers:
This is better as you have made a second layer transparent and then used the circle selection tool and filled the selection with the second color. This gets the job done and maintains more flexibility than the previous technique, so is much better. But, since this is a tutorial about layer masks, we will be going further AND providing ourselves with more flexibility if we need to change the second color in the future.
So, with the base layer, add a new layer. Now create your selection. In order to create our layer mask, we will first save your selection to a channel.
A channel is a special type of layer which is in grayscale that can be used to create a layer mask. Creating channels is a way to apply the exact same layer mask to multiple layers quickly. Once you have created the channel remove the selection, fill the new layer entirely with your secondary color. Now, in order to get our image to look like the original reference, we will add the layer mask by right clicking the top layer and selecting add layer mask from the menu.
From the popup window, select the Channel radio button and select the channel you just created.
So why are we going to all this work? Because now, we can change either color simply by selecting the layer and using the paint bucket tool. Likewise, you could fill with a texture also. With the first technique(direct painting) you have TONS of work to do a simple color change. With the second technique(two layers) you would have to select the painted image on the second (top) layer and then fill with the new color. Not hard by any means, but not as flexible as if using masks, which you will see shortly.
I have reached the posting limit of 5 images, so I will briefly explain why using masks is better, before I explain how in the next post. Imagine using the above two layers but with some type of irregular selection, AND in addition to wanting to change the color, you want to expand or contract the size. Not hard if you want to maintain the exact same shape/diminsions, but what if you want to change the shape. Then you have some annoying tedious work to do. This is where layer masks will save the day, as we will see in the next post.