TY - JOUR
T1 - Optimality and limitations of audio-visual integration for cognitive systems
AU - Boyce, William
AU - Lindsay, Anthony
AU - Zgonnikov, Arkady
AU - Rano, Inaki
AU - Wong-Lin, KongFatt
N1 - This article is part of the Research Topic "Closing the Loop: From Human Behavior to Multisensory Robots".
https://www.frontiersin.org/research-topics/9321/closing-the-loop-from-human-behavior-to-multisensory-robots#articles
PY - 2020/7/17
Y1 - 2020/7/17
N2 - Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artifacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artifacts. Finally, we suggest avenues of research toward solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.
AB - Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artifacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artifacts. Finally, we suggest avenues of research toward solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.
KW - Bayesian integration
KW - audio-visual illusions
KW - cognitive systems
KW - multi-modal processing
KW - multisensory integration
KW - optimality
UR - https://www.frontiersin.org/articles/10.3389/frobt.2020.00094/full
UR - http://www.scopus.com/inward/record.url?scp=85088972047&partnerID=8YFLogxK
U2 - 10.3389/frobt.2020.00094
DO - 10.3389/frobt.2020.00094
M3 - Review article
VL - 7
JO - Frontiers in Robotics and AI
JF - Frontiers in Robotics and AI
SN - 2296-9144
M1 - 94
ER -