Face SwapJul 25, 2016
Are you curious how the face swap filter works? Do you want to see what my final project for Computational Photography was? Keep reading and I’ll show both!
Step 1: Find the faces
First I used
dlib.get_frontal_face_detector to find all the faces in the image.
Step 2: Find each face’s landmarks
Next I used
dlib.shape_predictor to identify 68 important points in each of the faces from the previous step. This resulted in features such as the nose, eyes, eyebrows, lips, and jaw line.
Step 3: Turn the landmarks into a mesh
Once I had the landmarks, I used a technique called Delauney triangulation to create a series of triangles representing the mesh. I averaged the two meshes so that the triangles from each face would be made up of the same landmark points. I found that there were two ways to generate the mesh that worked with varying results. The first image is creating a mesh using all of the landmark points, and the second image shows using just the outer landmarks.
Step 4: Swap the triangles
Next, I used math to warp the source triangles to match their destination and I swapped each set of triangles. You can see the difference between using all or just the outer landmarks here.
Step 5: Blend
Then, I blended the faces so that the edges looked smoother. I created a mask by generating a convex hull from the points (
And finally, I used
cv2.seamlessClone to apply the masks and create these masterpieces.
Hope you learned something! Feel free to comment with any questions you have!