Header logo is am

Enhanced visual scene understanding through human-robot dialog

2011

Conference Paper

am


We propose a novel human-robot-interaction framework for robust visual scene understanding. Without any a-priori knowledge about the objects, the task of the robot is to correctly enumerate how many of them are in the scene and segment them from the background. Our approach builds on top of state-of-the-art computer vision methods, generating object hypotheses through segmentation. This process is combined with a natural dialog system, thus including a `human in the loop' where, by exploiting the natural conversation of an advanced dialog system, the robot gains knowledge about ambiguous situations. We present an entropy-based system allowing the robot to detect the poorest object hypotheses and query the user for arbitration. Based on the information obtained from the human-robot dialog, the scene segmentation can be re-seeded and thereby improved. We present experimental results on real data that show an improved segmentation performance compared to segmentation without interaction.

Author(s): Johnson-Roberson, M. and Bohg, J. and Skantze, G. and Gustafson, J. and Carlson, R. and Rasolzadeh, B. and Kragic, D.
Book Title: Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on
Pages: 3342-3348
Year: 2011

Department(s): Autonomous Motion
Research Project(s): Interactive Perception
Bibtex Type: Conference Paper (inproceedings)

DOI: 10.1109/IROS.2011.6094596

Links: pdf
video
Video:

BibTex

@inproceedings{6094596,
  title = {Enhanced visual scene understanding through human-robot dialog},
  author = {Johnson-Roberson, M. and Bohg, J. and Skantze, G. and Gustafson, J. and Carlson, R. and Rasolzadeh, B. and Kragic, D.},
  booktitle = {Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on},
  pages = {3342-3348},
  year = {2011}
}