If you'll recall, just a few months ago we were doing this with cats. Draw something in a little box and an algorithm will try to interpret it as a cat and then fill in the colors and textures according to a machine learning model training on thousands of cat images. This was the first iteration of the pix2pix project. Now, we have human faces and things have become fully uncanny.
The pix2pix project demonstrates something pretty profound about machine learning circa 2017: It's awful at generating new images, or at least meaningful new images. Machine learning is better at classifying existing images, but, even then, things drop off dramatically as we move beyond a handful of really robust object-recognition models. To get a really robust model, we need millions of images to serve as training data and loads of time to actually do the training.
Pix2pix is based on a promising technique called generative adversarial networks (GANs). GANs work by training generative models that seek to minimize a particular "loss function" according to a prediction that the generated image is fake or real. Basically, it learns to draw stuff by iteratively drawing a bunch of decreasingly random images while a second component of the algorithm judges them and screams "fake!" or "real!"
With a huge dataset, we can even take GANs and draw stuff from scratch. For a smaller dataset, we can do, well, this. Image-to-image translation. Rather than learn how to produce images from scratch, the model here learns to map the abstract image representation contained within a machine learning model to a trackpad doodle. It's able to pick out features within the doodle and pair them up with features within the model, with the observed results.
This may seem crude or trivial now, but expect GANs to play increasingly important roles in generative machine learning in the future.