Project Goal

The project goal is to establish a mapping between local scenes as observed by a robot ( helicopter) to aerial imagery acquired from aircrafts or satellites. An example of such an aerial image is shown in Figure 1.

---

Figure 1: Aerial Image of campus

---

The goal is to align locally acquired images in different lighting conditions and different scales to an aerial map.

Project Scope

This project involves the development of Scale Invariant Feature Transform (SIFT *) feature detectors modifying code from Intel Corporation, adding SIFT gradient based histogram identification keys and applying them to (1) the aerial image and (2) to the imagery acquired by the helicopter. The next step involved matching those features to establish global correspondence. The match may be based on a single robot image, or a sequence of images. For the latter, the relative position of the robotic vehicle might only be available in approximation. The project succeeds if both aerial map and local image types can be aligned reliably.

Tasks

The project will be accomplished through the following tasks. Basic project is completed at Task 7, bonus for going on.

• Task 1: Download the SIFT papers zip file (7M) and read all of the papers except SIFT06I*.

• Task 2: Download run and understand Matlab SIFT code (0.3M) (it doesn't include recognition keys).
• Task 3: Debug this code for small and large images.
• Task 4: Add SIFT recognition keys via 3x3 gradient histogram with overlapping bins.
• Task 5: Implement a recognition scheme such as Lowe's affine warped Hough transform.
• Task 6: Apply SIFT to the aerial map to create a location database of SIFT keys.
• Task 7: Search for correspondences between a single helicopter image and the aerial image, using geometry and appearance to reject outliers etc.
• Task 8: Integrate location over multiple helicopter images to add robustness to place recognition.
• Task 9: Lower the SIFT detection threshold and divide the Difference of Gaussian features by Sum of Gaussians for greater lighting independence.
• Task 10: Apply Random Forests and MART classifiers from Intel's Machine Learning Library (MLL, out soon) to the same data to see if better recognition results can be obtained.

You will be working closely with the Robot Learning Lab, which will provide you with all data.

Pre-requisites:

Helps to be proficient in Matlab

Project Status

Daniel Gindikin

Point of Contact

 and Sebastian Thrun

Midterm Report

not yet submitted

* David G. Lowe, "Object Recognition from Local Scale-Invariant Features", ICCV'99