Out in the Open: Free Software That Teaches Your Smartphone How to See
- 6:22 pm |
Pete Warden has been trying to teach computers to see since the 1990s. Now, thanks to the branch of artificial intelligence called deep learning, he’s finally making some progress.
Deep learning attempts to model the structure and behavior of the human brain to solve complex computer science problems. The field has been around since the 1980s, but there’s been an explosion of interest in its techniques in the past few years as the cost of powerful computers has fallen. Google now uses deep learning inside several of its online services, and last year, it hired Geoffrey Hinton, the central figure in the movement. Not long after Facebook hired his colleague Yann LeCun.
Warden, an ex-Apple engineer, studied machine vision as an undergraduate, and recently, he adopted deep learning at his startup Jetpac, a company that analyzes Instagram photos in order to make travel recommendations. It has worked so well, he’s now sharing his methods with the world at large, offering a free developer kit called DeepBelief. Able to run on smartphones, tablets, even the tiny Raspberry Pi computer, the DeepBelief framework aims to provide all programmers with a simple way of building mobile apps that can recognize simple images.
Jetpac offers the developer kit under an open source license, so anyone can use it to build applications for any purpose. “We’ve benefited from the hard work of Geoff Hinton and other pioneers,” he says. “So I wanted to give something back in return for all of the awesome research that they’ve contributed.”
The Computer Teacher
Warden is careful to stress that deep learning isn’t magic. “There’s lots of ways you can trip it up,” he says. It misses and it misclassifies some images. If it was a human doing the sorting job, you’d fire them.” But he also says it has revolutionized machine vision. “The thing is, computer vision never used to work at all,” he says. “Having something that kind of works in a massive leap forward.”
The big advantage, he explains, is that instead of writing and hand tuning algorithms to recognize specific details of a photograph–such as the shape of eyes, noses, and mouths if you want to identify photos of faces–you can just give a deep learning system a series of examples and it will learn to spot the patterns that all the photos have in common. That usually means feeding the system hundreds or thousands of relevant photos.
Warden believes DeepBelief could eventually be used to create a whole host of novel applications, such as pet doors that open for cats but not racoons or skunks, or video monitoring systems that can count how many people, cars, or dogs pass through a public space. The important thing, he says, is to figure out where DeepBelief is useful and where it isn’t. Jetpac uses it to analyze millions of Instagram photos to try to learn more about the places they were taken. For example, it tries to figure out if a place has lots of hipsters by counting how many photos have people with mustaches in them. Perhaps more practically, the team is also trying to detect which restaurants have patios based on which photos include blue sky, and which places are dog friendly based on which ones include photos of dogs.
“I feel like I’m becoming a computer teacher rather than a computer programmer,” Warden says. “Like with kids, you have to make sure i have the right lesson plan so that they’ll learn the right thing. For example, in photos of dogs, I have to make sure that they don’t all have grass so I’m training it to see dogs not grass.”
The Secret Sauce
Though the DeepBelief development kit is free, there’s a caveat. The “secret sauce” behind the kit is the way it compresses its neural networks to fit onto relatively low-powered smartphones and Raspberry Pis. And the source code for that isn’t available. That prevents developers from really being able to study and modify those parts of the code.
But the framework can be used by developers for any type of application, including commercial ones. There are other deep-learning-based machine vision frameworks, such as OverFeat and Caffe, but a core part of these projects–the “model”–are only available for research purposes, not commercial applications. “The model is the set of neural network parameters, and is the actual wiring of the ‘brain,'” Warden explains. “All of the smarts are in the model, and they take a lot of time and experimentation to create, so having a freely-usable one is important.”
Eventually, though, he’d like to open source the entirety of the DeepBelief kit. “On a personal level I’m itching to get that out there,” he says. “But we’re a VC backed company and we have IP and I have shareholders to answer to.”