Research Article Open Access

INCORPORATING PRIOR KNOWLEDGE INTO TEMPORAL DIFFERENCE NETWORKS

Britton Wolfe1 and James Harpe1
  • 1 Indiana University-Purdue University Fort Wayne (IPFW), United States

Abstract

Developing general purpose algorithms for learning an accurate model of dynamical systems from example traces of the system is still a challenging research problem. Predictive State Representation (PSR) models represent the state of a dynamical system as a set of predictions about future events. Our work focuses on improving Temporal Difference Networks (TD Nets), a general class of predictive state models. We adapt the internal structure of the TD Net and we present an improved algorithm for learning a TD Net model from experience in the environment. The new algorithm accepts a set of known facts about the environment and uses those facts to accelerate the learning. These facts can come from another learning algorithm (as in this study) or from a designer's prior knowledge about the environment. Experiments demonstrate that using the new structure and learning algorithm improves the accuracy of the TD Net models. When tested in an in finite environment, our new algorithm outperforms all of the standard PSR learning algorithms.

Journal of Computer Science
Volume 10 No. 11, 2014, 2211-2219

DOI: https://doi.org/10.3844/jcssp.2014.2211.2219

Submitted On: 6 June 2014 Published On: 18 December 2014

How to Cite: Wolfe, B. & Harpe, J. (2014). INCORPORATING PRIOR KNOWLEDGE INTO TEMPORAL DIFFERENCE NETWORKS. Journal of Computer Science, 10(11), 2211-2219. https://doi.org/10.3844/jcssp.2014.2211.2219

  • 3,103 Views
  • 2,118 Downloads
  • 0 Citations

Download

Keywords

  • Predictive State
  • Temporal Difference
  • Modeling
  • Dynamical Systems