Project Goal
Motion is one of the best understood aspects of visual processing in the brain. We know a lot about how motion is perceived by biological visual systems. We know a surprising amount about how motion information is represented in the brain. We even have some good ideas about how biological systems might extract motion information from the visual scene. But we know shamefully little about how motion information is combined across space and time to construct a “motion scene.”To interpret the visual world, any system must parse the continuous stream of sensory input into distinct salient elements. This is true in the simplest case of foreground background segregation all the way through the difficult task of object recognition. It is no different in the realm of motion vision. To estimate the motion of objects well, information must be combined across space and time - but not across sources. Motion elements must be segmented from each other and from the motion of the background to allow a system to make intelligent judgments about its environment. But how? Can we use what we know about biological vision to construct an algorithm?
Figure 1: Ambiguity of transparent motion compares to that of the Necker cube.
A particularly compelling example of the need for this algorithm is the case of motion transparency. Motion transparency occurs whenever two motions appear in the same region of space. This might happen in a reflection on a pane of glass, or in the translating specularities of a moving reflective object, or in the jungle as the shadows of leave shimmer across the form of an approaching predator. In these cases it is the motion itself that allows us to separate one object from another. Because this is such a crisp example the challenges of motion segmentation, we will use a synthetic example of motion transparency to explore our algorithm. In addition we will use human perception of this example as inspiration for how to construct our algorithm.
Project Approach
Broadly, our approach will be to construct an interconnected network of local motion processors. The characteristics of each processing unit will be taken very directly from known properties of neurons in are MT of the primate visual cortex – the apparent seat of motion processing in the human brain. We will interpret the output of these unit as a Bayesian estimate of the probability of local motion given the images, and use belief propagation to reason about the motion scene.
The goal of this approach is really two fold. Not only might we construct a algorithm which successfully segments a motion scene, we might generate testable hypothesis for future neurophysiological studies of motion processing.
Tasks
The project will be accomplished through the following tasks.
- Task 1: Construct transparent motion stimuli
- Task 2: Implement local Bayesian motion estimation
- Task 3: Embed local motion estimators in a Markov Random Field designed to force contrary motion information into separtate disparity planes
Project Status
Greg Corrado
Point of Contact
Sebastian Thrun
Midterm Report
submitted
Final Report
submitted
