Research Project P1:
Structure from motion in panoramic video

Research Project Goal

We have a large corpus of panoramic video that is geo-referenced. The video has been collected by a car instrumented with high-recision GPS and inertial sensors. The goal of this project is to recover the 3-D structure of the environment from this video. This is simpler than the general structure from motion problem in that the camera motion is known. However, difficulties arise due to occlusion, and the presence of moving objects.

Research Project Scope

Given a stream of geo-referenced panoramic images, construct a 3-D model. In an initial version, we seek to determine the 3-d position of feature points. Later, we want a full volumetric 3-D model for all visible surfaces.

Data

Sample image of a geo-referenced panorama, here overlayed with a map from the USGS. Click on the image above to see a video demonstrating the panoramic data acquisition process.

Tasks

• Extract feature points
• Implement least squares solution to SFM for known correspondence. iIntegrate camera model of panoramic camera with calibratino parameters provided by instructors.
• Implement feature matching method. Here we can use RANSAC or a particle filter.

Tricky: identify surfaces, determing 3-D locatino of surfaces. Might use a local Hough transform, the EM algorithm, and/or PCA applied to local point clouds.

Research Project Status

Emre Oto
Qixing Huang
Karin Linnersund

Point of Contact

Sebastian Thrun

Midterm Report

not yet submitted

Final Report

not yet submitted