Tag: GoPro

Microsoft Research First-person Hyperlapse Videos

Microsoft Researcher Johannes Kopf ascends Mount Shuksan in the North Cascades with a GoPro.

Standard video stabilization crops out the pixels on the periphery to create consistent frame-to-frame smoothness. But when applied to greatly sped up video, it fails to compensate for the wildly shaking motion.

Hyperlapse reconstructs how a camera moves throughout a video, as well as its distance and angle in relation to what’s happening in each frame. Then it plots out a smoother camera path and stitches pixels from multiple video frames to rebuild the scene and expand the field of view.

As you might imagine, working with raw video involves crunching a fair amount of data, which required a compute cluster to crunch data for several hours to complete for each video. Microsoft developed a series of new algorithms that lead to a more efficient process without compromising the image quality. The result is that Hyperlapse can now render a high-speed video in a fraction of the time, using a single PC.

The Interactive Visual Media Group focuses on the areas of computer vision, image processing, and statistical signal processing, specifically as they relate to things like enhancing images and video, 3D reconstruction, image-based modeling and rendering, and highly-accurate correspondence algorithms that are commonly used to “stitch” together images.

From Microsoft Research.

We present a method for converting first-person videos, for example, captured with a helmet camera during activities such as rock climbing or bicycling, into hyper-lapse videos, i.e., time-lapse videos with a smoothly moving camera. At high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up.


Scene Reconstruction
Our algorithm first reconstructs the 3D input camera path as well as dense, per-frame proxy geometries. We then optimize a novel camera path for the output video (shown in red) that is smooth and passes near the input cameras while ensuring that the virtual camera looks in directions that can be rendered well from the input.
Next, we compute geometric proxies for each input frame. These allow us to render the frames from the novel viewpoints on the optimized path.

Proxy Geometry

Stitched & Blended
Finally, we generate the novel smoothed, time-lapse video by rendering, stitching, and blending appropriately selected source frames for each output frame. We present a number of results for challenging videos that cannot be processed using traditional techniques.

Game of Drones. John Bailey ASC on drone safety.

Above: a photo I took of a drone at Cine Gear Expo 2012.

Recently, I had a post that included spectacular footage of a fireworks show shot from a drone. The increase use of drones with GoPros has become both a safety and personal privacy headache.

The Seattle Police Department recently spoke with an Amazon employee who flew a drone too close to the Space Needle.

DJI has a new system called Dropsafe, drones that deploy parachutes.

From the IEEE Spectrum

According to the Hollywood Reporter there is now a new organization called the Society of Aerial Cinematographers, prompted by the increase of drone use in movie and television production.

John Bailey ASC, on his blog post called Drones, Drones, Drones, explains the many safety issues and FAA regulations about using camera drones.

…Thornier yet are certain looming questions of privacy rights, sexual “Peeping Tom” infringements, aural and visual harassments by drones buzzing around in public spaces, and the very real danger posed by malfunctioning, mis-piloted or mis-programmed drones — including higher-altitude (but still amateur) drones that already have nearly caused midair collisions with commercial jets

….I found videos of out-of-the-box amateur drones taking to the air (even as their new owners were still reading instructions) and crashing into high rises in midtown Manhattan, then falling onto the street below, nearly injuring a pedestrian.

…Just like the Steadicam before it, these small 4-rotor and 8-rotor drone helicopters mounted with HD cameras, from GoPros to  Canon 5Ds, are quickly changing the scale of imagery that can be photographed for feature films. Many productions that have been unable to afford traditional piloted helicopters with sophisticated camera-stabilizing systems can now engage a two-person ground-based crew of pilot and operator to shoot sweeping images that “open up” a film. But that is only a small part of drones’ potential as a new camera system.

Last winter, watching director Nabil Ayouch’s Horses of God, the Moroccan entry for the Academy’s foreign-film Oscar, I saw a shot that took my breath away. A group of boys are playing on a dirt soccer pitch in the Casablanca slum of Sidi Moumen. Everything is photographed at ground level, with long-lens panning shots intercut with wider-angle close coverage on the Steadicam to build up the action sequence. A very low-angle shot then follows several boys chasing the ball — and suddenly sweeps past them, rising above their heads to reveal the intricate warren of passageways in the slum beyond. The camera continues up higher for an overview of the slum and of downtown Casablanca. It is a stunning moment because it comes at the end of an eye-level sequence. It also sets up the disjunction between these still innocent, poor children playing soccer in a trash-ridden, dusty lot—- with the indifferent modern city nearby. The film climaxes with a sequence set years later, in May 2003, when these same boys, now trained suicide bombers, simultaneously blow up several buildings in downtown Casablanca, killing themselves and 33 people. This single camera move, made with a small HD camera on a drone, set up the visual and narrative flow for the rest of the film.

Here is the trailer. There are several brief cuts of the boys on the pitch early on, and a very brief overhead drone shot tracking though the slum at 0.44:

Fireworks filmed with a drone.

UPDATE: It is illegal to do this.