Reinforcement Learning for Bio-Inspired Target Seeking

James Gillespie; Ignacio Rano; Nazmul Siddique; Jose Santos; Mehdi Khamassi

doi:10.1007/978-3-319-64107-2_52

Reinforcement Learning for Bio-Inspired Target Seeking

James Gillespie, Ignacio Rano, Nazmul Siddique, Jose Santos, Mehdi Khamassi

Research output: Chapter in Book/Report/Conference proceeding › Chapter

1 Citation (Scopus)

Abstract

Because animals are extremely effective at moving in their natural environments they represent an excellent model to implement robust robotic movement and navigation. Braitenberg vehicles are bio- inspired models of animal navigation widely used in robotics. Tuning the parameters of these vehicles to generate appropriate behaviour can be challenging and time consuming. In this paper we present a Reinforce- ment Learning methodology to learn the sensori-motor connection of Braitenberg vehicle 3a, a biological model of source seeking. We present simulations of different stimuli and reward functions to illustrate the feasibility of this approach.

Original language	English
Title of host publication	Towards Autonomous Robotic Systems: 18th Annual Conference, TAROS 2017, Guildford, UK, July 19--21, 2017, Proceedings
Pages	637-650
DOIs	https://doi.org/10.1007/978-3-319-64107-2_52
Publication status	E-pub ahead of print - 20 Jul 2017

Fingerprint

Keywords

Braitenberg vehicles
Reinforcement learning
Source seeking

Cite this

Gillespie, J., Rano, I., Siddique, N., Santos, J., & Khamassi, M. (2017). Reinforcement Learning for Bio-Inspired Target Seeking. In Towards Autonomous Robotic Systems: 18th Annual Conference, TAROS 2017, Guildford, UK, July 19--21, 2017, Proceedings (pp. 637-650) https://doi.org/10.1007/978-3-319-64107-2_52

@inbook{d06539c2d7ca4203ad14dc435c0a4792,

title = "Reinforcement Learning for Bio-Inspired Target Seeking",

abstract = "Because animals are extremely effective at moving in their natural environments they represent an excellent model to implement robust robotic movement and navigation. Braitenberg vehicles are bio- inspired models of animal navigation widely used in robotics. Tuning the parameters of these vehicles to generate appropriate behaviour can be challenging and time consuming. In this paper we present a Reinforce- ment Learning methodology to learn the sensori-motor connection of Braitenberg vehicle 3a, a biological model of source seeking. We present simulations of different stimuli and reward functions to illustrate the feasibility of this approach.",

keywords = "Braitenberg vehicles, Reinforcement learning, Source seeking",

author = "James Gillespie and Ignacio Rano and Nazmul Siddique and Jose Santos and Mehdi Khamassi",

note = "Reference text: 1. Arechavaleta, G., Laumond, J.P., Hicheur, H., Berthoz, A.: An optimality principle governing human walking. IEEE Trans. Robot. 24(1), 5–14 (2008) 2. Bicho, E., Sch ̈oner, G.: The dynamic approach to autonomous robotics demon- strated on a low-level vehicle platform. Robot. Auton. Syst. 21, 23–35 (1997) 3. Braitenberg, V.: Vehicles. Experiments in Synthetic Psychology. The MIT Press, Cambridge (1984) 4. Floreano, D., Ijspeert, A.J., Schaal, S.: Robotics and neuroscience. Curr. Biol. 24, R910–R920 (2014) 5. Fraenkel, G.S., Gunn, D.L.: The Orientation of Animals: Kineses, Taxes and Com- pass Reactions. Dover Publications, New York (1961) 6. French, R., Can ̃amero, L.: Introducing neuromodulation to a braitenberg vehi- cle. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 4188–4193 (2005) 650 J. Gillespie et al. 7. Gallistel, C.: Navigation: whence our sense of direction? Curr. Biol. 27(3), 108–110 (2017) 8. Ijspeert,A.,Crespi,A.,Ryczko,D.,Cabelguen,J.:Fromswimmingtowalkingwith a salamander robot driven by a spinal cord model. Science 315(5817), 1416–1420 (2007) 9. Lebastard, V., Boyer, F., Chevallereau, C., Servagent, N.: Underwater electro- navigation in the dark. In: Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 1155–1160 (2012) 10. Lilienthal, A.J., Duckett, T.: Experimental analysis of smelling braitenberg vehi- cles. In: Proceedings of the IEEE International Conference on Advanced Robotics (ICAR 2003). IEEE (2003) 11. Menciassi, A., Dario, P.: Bio-inspired solutions for locomotion in the gastrointesti- nal tract: background and perspectives. Philos. Trans. A Math. Phys. Eng. Sci. 361, 2287–2298 (2003). The Royal Society – Biologically Inspired Robots 12. Mondada, F., Floreano, D.: Evolution of neural control structures: some experi- ments on mobile robots. Robot. Auton. Syst. 16, 183–195 (1995) 13. Ran ̃o ́, I.: A steering taxis model and the qualitative analysis of its trajectories. Adapt. Behav. 17(3), 197–211 (2009) 14. Ran ̃o ́, I.: A model and formal analysis of braitenberg vehicles 2 and 3. In: IEEE International Conference on Robotics and Automation (2012) 15. Ran ̃o ́, I.: Results on the convergence of braitenberg vehicle 3a. Artif. Life 20(2), 223–235 (2014) 16. Ran ̃ ́o, I., Wong-Lin, K., Khamassi, M.: A drift diffusion model of biological source seeking for mobile robots. In: Proceedings of the IEEE International Conference on Robotics and Automation (2017) 17. Salomon, R.: Evolving and optimising braitenberg vehicles by means of evolution strategies. Int. J. Smart Eng. Syst. Des. 2, 69–77 (1999) 18. Salum ̈ae, T., Ran ̃o ́, I., Akanyeti, O., Kruusmaa, M.: Against the flow: a braitenberg controller for a fish robot. In: IEEE International Conference on Robotics and Automation (2012) 19. Shaikh, D., Hallam, J., Christensen-Dalsgaard, J.: From ear to there: a review of biorobotic models of auditory processing in lizards. Biol. Cybern. 110(4), 303–317 (2016) 20. Webb, B.: A Spiking Neuron Controller for Robot Phonotaxis, pp. 3–20. The MIT/AAAI Press, Cambridge (2001) 21. Yang, X., Patel, R., Moallem, M.: A fuzzy-braitenberg navigation strategy for differential drive mobile robots. J. Intell. Robot. Syst. 47, 101–124 (2006)",

year = "2017",

month = "7",

day = "20",

doi = "10.1007/978-3-319-64107-2_52",

language = "English",

isbn = "978-3-319-64107-2",

pages = "637--650",

booktitle = "Towards Autonomous Robotic Systems: 18th Annual Conference, TAROS 2017, Guildford, UK, July 19--21, 2017, Proceedings",

}

TY - CHAP

T1 - Reinforcement Learning for Bio-Inspired Target Seeking

AU - Gillespie, James

AU - Rano, Ignacio

AU - Siddique, Nazmul

AU - Santos, Jose

AU - Khamassi, Mehdi

N1 - Reference text: 1. Arechavaleta, G., Laumond, J.P., Hicheur, H., Berthoz, A.: An optimality principle governing human walking. IEEE Trans. Robot. 24(1), 5–14 (2008) 2. Bicho, E., Sch ̈oner, G.: The dynamic approach to autonomous robotics demon- strated on a low-level vehicle platform. Robot. Auton. Syst. 21, 23–35 (1997) 3. Braitenberg, V.: Vehicles. Experiments in Synthetic Psychology. The MIT Press, Cambridge (1984) 4. Floreano, D., Ijspeert, A.J., Schaal, S.: Robotics and neuroscience. Curr. Biol. 24, R910–R920 (2014) 5. Fraenkel, G.S., Gunn, D.L.: The Orientation of Animals: Kineses, Taxes and Com- pass Reactions. Dover Publications, New York (1961) 6. French, R., Can ̃amero, L.: Introducing neuromodulation to a braitenberg vehi- cle. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 4188–4193 (2005) 650 J. Gillespie et al. 7. Gallistel, C.: Navigation: whence our sense of direction? Curr. Biol. 27(3), 108–110 (2017) 8. Ijspeert,A.,Crespi,A.,Ryczko,D.,Cabelguen,J.:Fromswimmingtowalkingwith a salamander robot driven by a spinal cord model. Science 315(5817), 1416–1420 (2007) 9. Lebastard, V., Boyer, F., Chevallereau, C., Servagent, N.: Underwater electro- navigation in the dark. In: Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 1155–1160 (2012) 10. Lilienthal, A.J., Duckett, T.: Experimental analysis of smelling braitenberg vehi- cles. In: Proceedings of the IEEE International Conference on Advanced Robotics (ICAR 2003). IEEE (2003) 11. Menciassi, A., Dario, P.: Bio-inspired solutions for locomotion in the gastrointesti- nal tract: background and perspectives. Philos. Trans. A Math. Phys. Eng. Sci. 361, 2287–2298 (2003). The Royal Society – Biologically Inspired Robots 12. Mondada, F., Floreano, D.: Evolution of neural control structures: some experi- ments on mobile robots. Robot. Auton. Syst. 16, 183–195 (1995) 13. Ran ̃o ́, I.: A steering taxis model and the qualitative analysis of its trajectories. Adapt. Behav. 17(3), 197–211 (2009) 14. Ran ̃o ́, I.: A model and formal analysis of braitenberg vehicles 2 and 3. In: IEEE International Conference on Robotics and Automation (2012) 15. Ran ̃o ́, I.: Results on the convergence of braitenberg vehicle 3a. Artif. Life 20(2), 223–235 (2014) 16. Ran ̃ ́o, I., Wong-Lin, K., Khamassi, M.: A drift diffusion model of biological source seeking for mobile robots. In: Proceedings of the IEEE International Conference on Robotics and Automation (2017) 17. Salomon, R.: Evolving and optimising braitenberg vehicles by means of evolution strategies. Int. J. Smart Eng. Syst. Des. 2, 69–77 (1999) 18. Salum ̈ae, T., Ran ̃o ́, I., Akanyeti, O., Kruusmaa, M.: Against the flow: a braitenberg controller for a fish robot. In: IEEE International Conference on Robotics and Automation (2012) 19. Shaikh, D., Hallam, J., Christensen-Dalsgaard, J.: From ear to there: a review of biorobotic models of auditory processing in lizards. Biol. Cybern. 110(4), 303–317 (2016) 20. Webb, B.: A Spiking Neuron Controller for Robot Phonotaxis, pp. 3–20. The MIT/AAAI Press, Cambridge (2001) 21. Yang, X., Patel, R., Moallem, M.: A fuzzy-braitenberg navigation strategy for differential drive mobile robots. J. Intell. Robot. Syst. 47, 101–124 (2006)

PY - 2017/7/20

Y1 - 2017/7/20

N2 - Because animals are extremely effective at moving in their natural environments they represent an excellent model to implement robust robotic movement and navigation. Braitenberg vehicles are bio- inspired models of animal navigation widely used in robotics. Tuning the parameters of these vehicles to generate appropriate behaviour can be challenging and time consuming. In this paper we present a Reinforce- ment Learning methodology to learn the sensori-motor connection of Braitenberg vehicle 3a, a biological model of source seeking. We present simulations of different stimuli and reward functions to illustrate the feasibility of this approach.

AB - Because animals are extremely effective at moving in their natural environments they represent an excellent model to implement robust robotic movement and navigation. Braitenberg vehicles are bio- inspired models of animal navigation widely used in robotics. Tuning the parameters of these vehicles to generate appropriate behaviour can be challenging and time consuming. In this paper we present a Reinforce- ment Learning methodology to learn the sensori-motor connection of Braitenberg vehicle 3a, a biological model of source seeking. We present simulations of different stimuli and reward functions to illustrate the feasibility of this approach.

KW - Braitenberg vehicles

KW - Reinforcement learning

KW - Source seeking

U2 - 10.1007/978-3-319-64107-2_52

DO - 10.1007/978-3-319-64107-2_52

M3 - Chapter

SN - 978-3-319-64107-2

SP - 637

EP - 650

BT - Towards Autonomous Robotic Systems: 18th Annual Conference, TAROS 2017, Guildford, UK, July 19--21, 2017, Proceedings

ER -

Access to Document

10.1007/978-3-319-64107-2_52