Microsoft researchers have developed a technique for converting
first-person videos, such as those captured with a helmet camera
while cycling, into wonderfully-smooth "hyperlapse" videos.
Hyperlapse videos are timelapse videos that appear to be shot
with a smoothly moving video camera. An algorithm eliminates the
erratic camera shake that tends to be present in casually captured
The technique is particularly useful in an age when we can shoot
hours and hours of footage as we go about a particular activity --
whether it's skiing, walking, surfing or climbing. Always-on
cameras such as those made by GoPro and Hero are very simple to use
but can suffer from camera shake (given that they are generally
positioned on helmets or similar) and changing lighting. The videos
can also be long and monotonous, so they are boring to watch and
difficult to navigate.
While it's easy to create hyperlapse videos from very stable
images, such as those generated by a camera attached to a car -- as documented in this story about stitching Google Street View
journeys together -- it's much harder to do the same with cameras
attached to wobbly bipeds.
Traditional stabilisation methods and simple frame sub-sampling
techniques don't work with first-person videos as the shakiness
gets exacerbated as the footage is sped up. The Microsoft Research
team worked on a system that reconstructs the journey and develops
a new, virtual camera path for the output video that is rendered
from the input footage.
There are three key parts to the process. The first is scene
reconstruction, which involves developing a 3D model of an
environment based on the captured frames using
"structure-from-motion" algorithms. Once the model has been built,
the system will plan an optimised path for the camera that is
smooth in location and orientation and makes the most of the
supplied input footage. Finally, the image is rendered at ten times
the original speed using stitching and blending of carefully
selected frames from the original footage. The result is a pretty
cool fly-through of the journey with a very fluid motion.
The authors of the paper say: "As the prevalence of first-person
video grows, we expect to see a greater demand for creating
informative summaries from the typically long video captures. Our
hyperlapse work is just one step forward. As better semantic
understanding of the scene becomes available, either through
improved recognition algorithms or through user input, we hope to
incorporate such information, both to adjust the speed along the
smoothed path, the camera orientation, or perhaps to simply jump
over uninformative sections of the input."
You can read more about the technique in this research paper.