Research Project P7:
Robot Object Recognition

Research Project Goal

STAIR (Stanford AI Robot) is an ongoing project funded by NSF, DARPA and industry to build a robot that can navigate, interact, work with and for humans in office and home settings. Over the coming years STAIR has milestones of cleaning up after an office party, clearing a table, loading a dishwasher, fetching an item from anywhere in Gates, and assembling an Ikea bookshelf.

In order to achieve these goals, STAIR must build up a competent vision system able to recognize 100s of objects. It must also segment those objects from the background scene to enable the robot to eventually pick up and handle the objects. This project addresses the visual recognition and segmentation of objects for STAIR. We have built up many tools and techniques and we hope to put them together to give STAIR a robust way of learning and segmenting objects.

Research Project Scope

Over time, we have built up several of the pieces for the vision system, they now need to be put together. We have fast image segmentation as shown below for a hard scene.



We have several methods of object recognition, we are trying a curve matching technique from Pedro Felzenszwalb and Joshua Schwartz . Both the above are implemented here by TongKe Xue. In order to help the curve matching robustly find the correct curves in the visual scene above, we have a stereo camera from TYZX Corporation that works well with indoor objects. Only curves with sharp depth changes would be used in shape matching. We will get output from the stereo camera, segment it and only retain curves with significant depth change (marking object boundaries). These curves will then be fed into the recognition machinery. We hope to reliably recognize and segment 15 objects by end of term.

Tasks

• With help from the STAIR robot team, be able to get image and depth data from the stereo camera.
  • Possibly we may have to calibrate another camera together with the stereo camera to get images and depth. We will try to avoid this.
• Collect data from 15 objects types (several examples of scissors, several cell phones etc) in "natural settings" around Gates using the robot..
• Run the segmentation code on the data and extract only occluding boundary curves. This may involve some curve simplification to eliminate things like power cords sticking out from objects.
• Run the curve matching on the curves to get object recognition.
• Other object recognition techniques may be tried.
• Demonstrate robust performance on real runs using STAIR (driven by the STAIR team).
• Develop techniques to scale the system. STAIR must eventually handle many 100s of objects. We need to develop techniques to rapidly prune the search space down. For example, hammers will be long and skinny and so need not be considered when viewing a something like a boxy cell phone.

Research Project Status

student names here

Point of Contact

Gary Bradski: garybradski@gmail.com

Midterm Report

not yet submited

Final Report

not yet submitted