Header logo is am

From isolation to cooperation: An alternative of a system of experts


Book Chapter


We introduce a constructive, incremental learning system for regression problems that models data by means of locally linear experts. In contrast to other approaches, the experts are trained independently and do not compete for data during learning. Only when a prediction for a query is required do the experts cooperate by blending their individual predictions. Each expert is trained by minimizing a penalized local cross validation error using second order methods. In this way, an expert is able to adjust the size and shape of the receptive field in which its predictions are valid, and also to adjust its bias on the importance of individual input dimensions. The size and shape adjustment corresponds to finding a local distance metric, while the bias adjustment accomplishes local dimensionality reduction. We derive asymptotic results for our method. In a variety of simulations we demonstrate the properties of the algorithm with respect to interference, learning speed, prediction accuracy, feature detection, and task oriented incremental learning. 

Author(s): Schaal, S. and Atkeson, C. G.
Book Title: Advances in Neural Information Processing Systems 8
Pages: 605-611
Year: 1996
Editors: Touretzky, D. S.;Mozer, M. C.;Hasselmo, M. E.
Publisher: MIT Press

Department(s): Autonomous Motion
Bibtex Type: Book Chapter (inbook)

Address: Cambridge, MA
Cross Ref: p872
Note: clmc
URL: http://www-clmc.usc.edu/publications/S/schaal-NIPS1996.pdf


  title = {From isolation to cooperation: An alternative of a system of experts},
  author = {Schaal, S. and Atkeson, C. G.},
  booktitle = {Advances in Neural Information Processing Systems 8},
  pages = {605-611},
  editors = {Touretzky, D. S.;Mozer, M. C.;Hasselmo, M. E.},
  publisher = {MIT Press},
  address = {Cambridge, MA},
  year = {1996},
  note = {clmc},
  crossref = {p872},
  url = {http://www-clmc.usc.edu/publications/S/schaal-NIPS1996.pdf}