Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers

James Gillespie, Ignacio Rano, Nazmul Siddique, J.A. Santos, Mehdi Khamassi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Braitenberg vehicles are bio-inspired controllers for sensor-based local navigation of wheeled robots that have been used in multiple real world robotic implementations. The common approach to implement such non-linear control mechanisms is through neural networks connecting sensing to motor action, yet tuning the weights to obtain appropriate closed-loop navigation behaviours can be very challenging. Standard approaches used hand tuned spiking or recurrent neural networks, or learnt the weights of feedforward networks using evolutionary approaches. Recently, Reinforcement Learning has been used to learn neural controllers for simulated Braitenberg vehicle 3a – a bio-inspired model of target seeking for wheeled robots – under the assumption of noiseless sensors. Real sensors, however, are subject to different levels of noise, and multiple works have shown that Braitenberg vehicles work even on outdoor robots, demonstrating that these control mechanisms work in harsh and dynamic environments. This paper shows that a robust neural controller for Braitenberg vehicle 3a can be learnt using policy gradient reinforcement learning in scenarios where sensor noise plays a non negligible role. The learnt controller is robust and tries to attenuate the effects of noise in the closed- loop navigation behaviour of the simulated stochastic vehicle. We compare the neural controller learnt using Reinforcement Learning with a simple hand tuned controller and show how the neural control mechanism outperforms a naïve controller. Results are illustrated through computer simulations of the closed-loop stochastic system.
Original languageEnglish
Title of host publicationProceedings of 2019 IEEE Symposium Series on Computational Intelligence (SSCI)
Place of PublicationXiamen, China
PublisherIEEE
Pages704
Number of pages712
ISBN (Electronic)978-1-7281-2485-8
ISBN (Print)978-1-7281-2486-5
DOIs
Publication statusPublished - Dec 2019

Keywords

  • Braitenberg vehicles
  • stochastic environment
  • Reinforcement learning

Fingerprint Dive into the research topics of 'Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers'. Together they form a unique fingerprint.

  • Cite this

    Gillespie, J., Rano, I., Siddique, N., Santos, J. A., & Khamassi, M. (2019). Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers. In Proceedings of 2019 IEEE Symposium Series on Computational Intelligence (SSCI) (pp. 704). IEEE. https://doi.org/10.1109/SSCI44817.2019.9002834