Tag: Hyperlapse

Hyperlapse from Instagram

Product designer Chris Connolly, and software engineers Thomas Dimson and Alex Karpenko.

Photo: Ariel Zambelich/WIRED

From Wired.

Today at 10am PST, Instagram is lifting the veil on Hyperlapse, one of the company’s first apps outside of Instagram itself. Using clever algorithm processing, the app makes it easy to use your phone to create tracking shots and fast, time-lapse videos that look as if they’re shot by Scorsese or Michael Mann. What was once only possible with a Steadicam or a $15,000 tracking rig is now possible on your iPhone, for free. (Instagram hopes to develop an Android version soon, but that will require changes to the camera and gyroscope APIs on Android phones.) And that’s all thanks to some clever engineering and an elegantly pared-down interaction design. The product team shared their story with WIRED.

The Inspirations
By day, Thomas Dimson quietly works on Instagram’s data, trying to understand how people connect and spread content using the service. Like a lot of people working at the company, he’s also a photo and movie geek—and one of his longest-held affections has been for Baraka, an art-house ode to humanity that features epic tracking shots of peoples all across the world. “It was my senior year, and my friend who was an architect said, ‘You have to see it, it will blow you away,’” says Dimson. He wasn’t entirely convinced. The movie, after all, was famous for lacking any narration or plot. But watching the film in his basement, Dimson was awestruck. “Ever since, it’s always been the back of my mind,” he says.

A sample shot from Baraka

By 2013, Dimson was at Instagram. That put him back in touch with Alex Karpenko, a friend from Stanford who had sold his start-up to Instagram in 2013. Karpenko and his firm, Luma, had created the first-ever image-stabilization technology for smartphone videos. That was obviously useful to Instagram, and the company quickly deployed it to improve video capture within the app. But Dimson realized that it had far greater creative potential. Karpenko’s technology could be used to shoot videos akin to all those shots in Baraka. “It would have hurt me not to work on this,” says Dimson.

Once you start using the app, you quickly see that replay speed itself becomes a novel, alluring tool: For pets and people, replaying at about 1x gives you the sense that you’re creating a tracking shot like that Copacabana scene in Goodfellas. The higher replay speeds work better for shooting the sky out your airplane window, the scenery scrolling past during a train ride, or anything else that’s moving slowly or at a distance. Where Instagram’s filters are all about changing color and light, Hyperlapse uses a simple speed slider as its main creative decision.

All of those choices must be built-in up front with traditional camera rigs. Usually, capturing even a brief tracking shot requires intricate choreography between where you’ll move with the camera and what your subjects will be doing when you film them. Time-lapse set-ups are even more intense, requiring a camera be set up on a track and programmed to move at a steady speed. Both of those art forms are hardly spontaneous, and spontaneity is supposed to be Instagram’s calling card.

Hyperlapse, by contrast, let’s you create a tracking shot in less than a minute. “This is an app that let’s you be in the moment in a different way,” says Krieger. “We did that by taking a pretty complicated image processing idea, and reducing it to a single slider. That’s super Instagram-y.”


The Technology behind Hyperlapse from Instagram

INSTAGRAM, HYPERLAPSE, INSTAGRAM ENGINEERING, VIDEO STABILIZATION
Yesterday we released Hyperlapse from Instagram—a new app that lets you capture and share moving time lapse videos. Time lapse photography is a technique in which frames are played back at a much faster rate than that at which they’re captured. This allows you to experience a sunset in 15 seconds or see fog roll over hills like a stream of water flowing over rocks. Time lapses are mesmerizing to watch because they reveal patterns and motions in our daily lives that are otherwise invisible.

Hyperlapses are a special kind of time lapse where the camera is also moving. Capturing hyperlapses has traditionally been a laborious process that involves meticulous planning, a variety of camera mounts and professional video editing software. With Hyperlapse, our goal was to simplify this process. We landed on a single record button and a post-capture screen where you select the playback rate. To achieve fluid camera motion we incorporated a video stabilization algorithm called Cinema (which is already used in Video on Instagram) into Hyperlapse.

In this post, we’ll describe our stabilization algorithm and the engineering challenges that we encountered while trying to distill the complex process of moving time lapse photography into a simple and interactive user interface.

Cinema Stabilization

Video stabilization is instrumental in capturing beautiful fluid videos. In the movie industry, this is achieved by having the camera operator wear a harness that separates the motion of the camera from the motion of the operator’s body. Since we can’t expect Instagrammers to wear a body harness to capture the world’s moments, we instead developed Cinema, which uses the phone’s built-in gyroscope to measure and remove unwanted hand shake.

Go here for more in depth info and technical videos http://instagram-engineering.tumblr.com/post/95922900787/hyperlapse

Microsoft Research First-person Hyperlapse Videos

Microsoft Researcher Johannes Kopf ascends Mount Shuksan in the North Cascades with a GoPro.

Standard video stabilization crops out the pixels on the periphery to create consistent frame-to-frame smoothness. But when applied to greatly sped up video, it fails to compensate for the wildly shaking motion.

Hyperlapse reconstructs how a camera moves throughout a video, as well as its distance and angle in relation to what’s happening in each frame. Then it plots out a smoother camera path and stitches pixels from multiple video frames to rebuild the scene and expand the field of view.

As you might imagine, working with raw video involves crunching a fair amount of data, which required a compute cluster to crunch data for several hours to complete for each video. Microsoft developed a series of new algorithms that lead to a more efficient process without compromising the image quality. The result is that Hyperlapse can now render a high-speed video in a fraction of the time, using a single PC.

The Interactive Visual Media Group focuses on the areas of computer vision, image processing, and statistical signal processing, specifically as they relate to things like enhancing images and video, 3D reconstruction, image-based modeling and rendering, and highly-accurate correspondence algorithms that are commonly used to “stitch” together images.

From Microsoft Research.

We present a method for converting first-person videos, for example, captured with a helmet camera during activities such as rock climbing or bicycling, into hyper-lapse videos, i.e., time-lapse videos with a smoothly moving camera. At high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up.


Scene Reconstruction
Our algorithm first reconstructs the 3D input camera path as well as dense, per-frame proxy geometries. We then optimize a novel camera path for the output video (shown in red) that is smooth and passes near the input cameras while ensuring that the virtual camera looks in directions that can be rendered well from the input.
Next, we compute geometric proxies for each input frame. These allow us to render the frames from the novel viewpoints on the optimized path.

Proxy Geometry

Stitched & Blended
Finally, we generate the novel smoothed, time-lapse video by rendering, stitching, and blending appropriately selected source frames for each output frame. We present a number of results for challenging videos that cannot be processed using traditional techniques.