Header logo is am

Bayesian backfitting


Conference Paper


We present an algorithm aimed at addressing both computational and analytical intractability of Bayesian regression models which operate in very high-dimensional, usually underconstrained spaces. Several domains of research frequently provide such datasets, including chemometrics [2], and human movement analysis [1]. The literature in nonparametric statistics provides interesting solutions such as Backfitting [3] and Partial Least Squares [4], which are extremely robust and efficient, yet lack a probabilistic interpretation that could place them in the context of current research in statistical learning algorithms that emphasize the estimation of confidence, posterior distributions, and model complexity. In order to achieve numerical robustness and low computational cost, we first derive a novel Bayesian interpretation of Backfitting (BB) as a computationally efficient regression algorithm. BBÕs learning complexity scales linearly with the input dimensionality by decoupling inference among individual input dimensions. We embed BB in an efficient, locally variational model selection mechanism that automatically grows the number of backfitting experts in a mixture-of-experts regression model. We demonstrate the effectiveness of the algorithm in performing principled regularization of model complexity when fitting nonlinear manifolds while avoiding the numerical hazards associated with highly underconstrained problems. We also note that this algorithm appears applicable in various areas of neural computation, e.g., in abstract models of computational neuroscience, or implementations of statistical learning on artificial systems.

Author(s): D’Souza, A. and Vijayakumar, S. and Schaal, S.
Book Title: Proceedings of the 10th Joint Symposium on Neural Computation (JSNC 2003)
Year: 2003

Department(s): Autonomous Motion
Bibtex Type: Conference Paper (inproceedings)

Address: Irvine, CA, May 2003
Cross Ref: p1837
Note: clmc
URL: http://www-clmc.usc.edu/publications/D/dsouza-JSNC2003.pdf


  title = {Bayesian backfitting},
  author = {D'Souza, A. and Vijayakumar, S. and Schaal, S.},
  booktitle = {Proceedings of the 10th Joint Symposium on Neural Computation (JSNC 2003)},
  address = {Irvine, CA, May 2003},
  year = {2003},
  note = {clmc},
  crossref = {p1837},
  url = {http://www-clmc.usc.edu/publications/D/dsouza-JSNC2003.pdf}