Research Project P7:
Robot Object Recognition
Research Project Goal
AI Robot) is an ongoing project funded by NSF, DARPA and industry
to build a robot that can navigate, interact, work with and for humans
in office and home settings. Over the coming years STAIR has
milestones of cleaning up after an office party, clearing a table,
loading a dishwasher, fetching an item from anywhere in Gates, and
assembling an Ikea bookshelf.
In order to achieve these goals, STAIR must build up a competent
vision system able to recognize 100s of objects. It must also segment
those objects from the background scene to enable the robot to
eventually pick up and handle the objects. This project addresses the
visual recognition and segmentation of objects for STAIR. We have
built up many tools and techniques and we hope to put them together to
give STAIR a robust way of learning and segmenting objects.
Research Project Scope
Over time, we have built
up several of the pieces for the vision system, they now need to be
put together. We have fast image segmentation as shown below for a
We have several methods of object recognition, we are trying a curve
matching technique from Pedro Felzenszwalb and Joshua Schwartz . Both the above are implemented here by
TongKe Xue. In order to help the curve matching robustly find the
correct curves in the visual scene above, we have a stereo camera from
that works well with indoor objects. Only curves with sharp depth
changes would be used in shape matching. We will get output from the
stereo camera, segment it and only retain curves with significant
depth change (marking object boundaries). These curves will then be
fed into the recognition machinery. We hope to reliably recognize and
segment 15 objects by end of term.
With help from the STAIR robot team, be able to get image and depth
data from the stereo camera.
Possibly we may have to calibrate another camera together with the
stereo camera to get images and depth. We will try to avoid this.
Collect data from 15 objects types (several examples of scissors,
several cell phones etc) in "natural settings" around Gates using the
Run the segmentation code on the data and extract only occluding
boundary curves. This may involve some curve simplification to
eliminate things like power cords sticking out from objects.
Run the curve matching on the curves to get object recognition.
Other object recognition techniques may be tried.
Demonstrate robust performance on real runs using STAIR (driven by the
Develop techniques to scale the system. STAIR must eventually handle
many 100s of objects. We need to develop techniques to rapidly
prune the search space down. For example, hammers will be long and
skinny and so need not be considered when viewing a something like a
boxy cell phone.
Research Project Status
student names here
Point of Contact
Gary Bradski: firstname.lastname@example.org
not yet submited
not yet submitted