Header logo is am

Path integral-based stochastic optimal control for rigid body dynamics

2009

Conference Paper

am


Recent advances on path integral stochastic optimal control [1],[2] provide new insights in the optimal control of nonlinear stochastic systems which are linear in the controls, with state independent and time invariant control transition matrix. Under these assumptions, the Hamilton-Jacobi-Bellman (HJB) equation is formulated and linearized with the use of the logarithmic transformation of the optimal value function. The resulting HJB is a linear second order partial differential equation which is solved by an approximation based on the Feynman-Kac formula [3]. In this work we review the theory of path integral control and derive the linearized HJB equation for systems with state dependent control transition matrix. In addition we derive the path integral formulation for the general class of systems with state dimensionality that is higher than the dimensionality of the controls. Furthermore, by means of a modified inverse dynamics controller, we apply path integral stochastic optimal control over the new control space. Simulations illustrate the theoretical results. Future developments and extensions are discussed.

Author(s): Theodorou, E. A. and Buchli, J. and Schaal, S.
Book Title: Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL ’09. IEEE Symposium on
Pages: 219-225
Year: 2009

Department(s): Autonomous Motion
Bibtex Type: Conference Paper (inproceedings)

Cross Ref: p10328
Note: clmc
URL: http://www-clmc.usc.edu/publications/t/theodorou-ADPRL2009.pdf

BibTex

@inproceedings{Theodorou_ADPRLAIS_2009,
  title = {Path integral-based stochastic optimal control for rigid body dynamics},
  author = {Theodorou, E. A. and Buchli, J. and Schaal, S.},
  booktitle = {Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on},
  pages = {219-225},
  year = {2009},
  note = {clmc},
  crossref = {p10328},
  url = {http://www-clmc.usc.edu/publications/t/theodorou-ADPRL2009.pdf}
}