Project P18:
Gesture recognition for HCI (Human-Car-Interaction :-).

Project Goal

The Stanford Mechanical Engineering department offers the course ME310: Design Projects with Corporate Partners. In of the course projects there, students work with an automobile manufacturer to explore gesture recognition technology to control secondary features such as radio, CD player, windows, ventilation, etc. within a car.

As mechanical engineers and designers, their main work is in integrating the system mechanically and electrically. What they don't have is significant expertise in computer vision to build a gesture recognition system, hence our course is cooperating with them on this project. The hand gestures we want to recognize include both posture and space-time gestures.

Project Scope

Tracking gestures is a hot problem in computer and lots of recent papers deal with it. Even under laboratory conditions, it has not been solved in all generality and in order to operate in a car, it has to address additional challenges such as changing lightning conditions while driving.

On the positive side, we are able to constrain the system to easy-to-recognize actions such as moving your whole hand in a circle or from left-right across the field of view of a camera. Furthermore, we have great engineers at ME310 to build the hardware and we can work with them to make vision conditions as easy as possible.

Tasks

• Meet with the ME310 guys to specify the problem and discuss the hardware side of your solution. You can use both single and multi-camera approaches and have a budget to purchase appropriate hardware (1 week).
• Do a literature review about common algorithms in the field (1 week).
  You can use these starting points:
  • For monocular vision given stable lightning conditions, the CAMSHIFT algorithm implemented in OpenCV does a fine job finding tracking skin color in an image. The depth information could be recovered either from the hand size in the image or via triangulation from more than one camera.
  • The Iterative Closest Point Algorithm aligns 3D point clouds, for example those obtained from a stereo system with those from a hand model.
  • The Lukas-Kanade Optical Flow as shown in the CS223b OpenCV Demo by David Stavens will tell you about movement in the image, which could be exploited as well.
• Implement a tracking approach and test it on recorded data (2 weeks).
• Recognize gestures from tracking trajectories (2 weeks).
• If time allows: Discuss and implement improvements, either in software or in hardware in collaboration with ME310.

Project Status

Keith Rauenbuehler (keithr at stanford),
Dinkar Gupta (dinkarg at stanford),
Sorav Bansal (sbansal at stanford),
Emil Gilliam (egilliam at stanford)

Point of Contact

Kyle Doerksen, Hendrik Dahlkamp.

Midterm Report

not yet submited

Final Report

not yet submitted