Project Goal
The goal of this project is design a fast and accurate 3D scanner that can be mounted on a mobile robotic platform. An early prototype of such a scanner had been built at Intel, as shown in the following figure:
The current prototype scanner is accurate (~0.1mm accuracy on a range of a meter), but lacks speed (at least 30 seconds per scan), and portability (as it can be seen on Figure 1). Figure 2 shows an example of 3D scanning, where the left image is one of the scanning images and the left image show a view of the resulting 3D scan.
The goal if this project is to design a entirely new scanner inspired from the Intel's prototype that is smaller and lighter (to allow portability), and much faster (to enable feedback response). The camera-projector pair act similarly to a stereo system where the depth of the scene is computed by ray triangulation (the blue rays on Figure 1). The projector is used to the optical rays coming from both device. The projector is used as an active
Project Scope
This project involves acquiring a compact camera and DLP projector, and obtain drivers that lets you control both device at maximum speeds. In its first generation, the objective could be to achieve one scan per second, driving the camera at approximately 30 frames per second. Good knowledge in hardware and device drivers is recommended for the project. The projector is used to project a set of pre-determined stripe patterns in the scene (left image of Figure 2). The stripe boundaries are then accurately detected in the camera images. The subsequent step consists of computing scene geometry via optical ray triangulation (intersection of the blue rays in Figure 1). In order to achieve optimal accuracies, the prototype scanner built at Intel uses a set of 48 vertical stripe patterns at different resolutions, and a mixed temporal and spatial image processing for optimal accuracy and scan density. A reduction in the number of projected patterns may be required to achieve the initial desired speed of one scan per second, to the cost of slightly lower scanning accuracy. It may necessary to limit the number of projected pattern to 15 or 20, depending on the performance limitations of the hardware. The second generation scanner should be aimed at operating at speeds close to 30 scans per seconds. In order to achieve this goal, a new type of projected pattern sequence should be considered to allow for continuous update of the scene geometry at every captured frame. This step will naturally require further compromise in scanning accuracy.
Tasks
The project will be accomplished through the following tasks.
- Task 1: Build the new scanner with the provided camera and projector.
- Task 2: Learn about scanner calibration, by -1- download the Camera Calibration Toolbox for Matlab and go through Examples 1 and 5. Then, download the 3D scanner calibration example dataset scanner_calibration_example.zip (58MB zipped), unzip the entire content of the file in a unique folder (named scanner_calibration_example) and run and study the matlab script scanner_calibration_script.m located in the calibration toolbox TOOLBOX_calib.
- Task 3: Learn about 3D scanning, by downloading the 3D scanning example data set scanning_example.zip (26MB zipped), unzip the content of the file in a DIFFERENT folder (named scanning_example), and run and study the matlab script scanning_script.m located in the calibration toolbox TOOLBOX_calib. At this point, you should have a complete understanding of the overall calibration and scanning process (spatial and temporal processing).
- Task 4: Design a new set of patterns that are suitable for the new designed scanner, given the speed requirement. It may be necessary to drop the temporal processing part of stripe detection, and keep spatial processing only, to the cost of slightly higher scanning errors. A formal error analysis may be required to make speed/accuracy tradeoffs.
- Task 5: Build and text the scanner with the new set of projected patterns. Characterize the scanner by measuring reconstruction errors on known surfaces (planes, structure with right angles,... ).
- Task 6: (Extension) Move to the second generation scanner by pushing the speed requirement to 30 scans per second. A new set of projected patterns will need to be created to enable continuous update of the scene geometry at every capture frame.
Pre-requisites
Good knowledge of C, C++, Matlab. Knowledge in hardware (cameras, projectors) and device drivers. Background in Computer Vision is beneficial, yet not required (all the necessary theoretical background material will be provided)
Project Contact
Jean-Yves Bouguet - Intel Corporation -
Project Status
David Camarillo, Surya P.N. Singh Khian Hao Lim
Point of Contact
Gary Bradski and Sebastian Thrun
Midterm Report
not yet submitted
