Learning Low Dimensional Predictive Representations

M. Rosencrantz, G. Gordon, and S. Thrun.

Predictive state representations (PSRs) have recently been proposed as an alternative to partially observable Markov decision processes (POMDPs) for representing the state of a dynamical system (Littman; Nips 2001). We present a learning algorithm that learns a PSR from observational data. Our algorithm produces a variant of PSRs called transformed predictive state representations (TPSRs). We provide an efficient principal-components-based algorithm for learning a TPSR, and show that TPSRs can perform well in comparison to Hidden Markov Models learned with Baum-Welch in a real world robot tracking task for low dimensional representations and long prediction horizons.

The full paper is available in PDF and gzipped Postscript

@INPROCEEDINGS{Rosencrantz04a,
  AUTHOR        = {Rosencrantz, M. and Gordon, G. and Thrun, S.},
  TITLE         = {Learning Low Dimensional Predictive Representations},
  BOOKTITLE     = {Proceedings of the Twenty-First International Conference on Machine Learning},
  YEAR          = {2004},
  ADDRESS       = {Banff, Alberta, Canada}
}