Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories

MJ Callaghan, Gildas Bengloan, Julien Ferrer, Léo Cherel, Mohamed Ali El Mostadi, Augusto Gómez Eguíluz, Niall McShane

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The first generation of affordable consumer virtual reality headsets and related peripherals are now available. Question-Answering (QA) systems and speech recognition/synthesis has improved dramatically over the last decade. Virtual assistants, based on speech-based services are growing in popularity and can be used in a range of diverse application areas. This paper explores the practical use of virtual reality, IOT and voice driven virtual assistants in remote laboratories to facilitate visualization of electrical phenomena and to tutor students; guiding them through each stage of an experiment; presenting supplementary teaching resources when requested; accessing, controlling and configuring instrumentation and hardware and providing feedback with summative and formative assessment. The re-purposing of existing teaching material for use in an immersive environment with a virtual assistant is shown and the limitations and opportunities offered by the approach taken and the technologies used are discussed. The process of integrating test instrumentation, the board under test, a switching matrix and additional teaching resources into virtual reality using IOT is discussed and the inclusion of virtual assistants described. Two case studies of practical working examples of remote laboratories in virtual reality with a virtual assistant tutor are demonstrated and the viability and long-term opportunities for the use of virtual reality and virtual assistants in the context discussed
Original languageEnglish
Title of host publicationUnknown Host Publication
PublisherSpringer
Number of pages13
Publication statusAccepted/In press - 23 Dec 2017
Event15th International Conference on Remote Engineering and Virtual Instrumentation (REV) - Duesseldorf, Germany
Duration: 23 Dec 2017 → …

Conference

Conference15th International Conference on Remote Engineering and Virtual Instrumentation (REV)
Period23/12/17 → …

Bibliographical note

Reference text: 1. L. De Floriani and D. Schmalstieg, "IEEE Virtual Reality 2017 Special Issue," in IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 4, pp. v-v, April 2017
2. A. Bouziane, D. Bouchiha, N. Doumi, M. Malki, Question Answering Systems: Survey and Trends, Procedia Computer Science, Volume 73, 2015, Pages 366-375, ISSN 1877-0509.
3. Khillare S A Ajitkumar M. Pundge and Namrata C Mahender. Question Answering System, Approaches and Techniques. Int. Jour. of Computer Applications 141(3):34-39, May 2016
4. Microsoft. 2017 Cortana Intelligence services. [ONLINE] Available at https://azure.microsoft.com/en-gb/overview/cortana-intelligence. [Accessed 03 December 2017].
5. Feisel, L. D. and Rosa, A. J. (2005), The Role of the Laboratory in Undergraduate Engineering Education. Journal of Engineering Education, 94: 121–130.
6. Black, A.W., & Taylor, P. (1997). Automatically clustering similar units for unit selection in speech synthesis. Proceedings of Eurospeech 1997 (pp. 601–604). Athens, Greece.
7. Harman Kardon Invoke review. [ONLINE] Available at: https://www.theverge.com/2017/10/20/16505468/harman-kardon-invoke-cortana-microsoft-smart-speaker-review. [Accessed 03 December 2017].
8. Cortana Intelligence. [ONLINE] Available at: https://www.microsoft.com/en-gb/cloud-platform/what-is-cortana-intelligence. [Accessed 03 December 2017].
9. E. Lindsay, D. Liu, S. Murray, and D. Lowe, “Remote Laboratories in Engineering Ed-ucation: Students’ Perceptions,” Proc. 18th Ann. Conf. Assoc. for Eng. Edu. (AaeE ’07), 2007
10. D. Lowe, S. Murray, E. Lindsay and D. Liu, "Evolving Remote Laboratory Architectures to Leverage Emerging Internet Technologies," in IEEE Transactions on Learning Technologies, vol. 2, no. 4, pp. 289-294, Oct.-Dec. 2009. doi: 10.1109/TLT.2009.33
11. Connect Raspberry Pi to Azure IoT Hub [ONLINE] https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-raspberry-pi-kit-node-get-started [Accessed 03 December 2017].
12. M.J. Callaghan, K. McCusker, J.L. Losada, J. Harkin, S. Wilson. "Using Game Based Learning in Virtual Worlds to Teach Electronic and Electrical Engineering, IEEE Trans. on Industrial Informatics, vol. 9, no. 1, pp. 575 - 584, Feb 2013.
13. Cognigy AI [ONLINE] https://www.cognigy.com/ [Accessed 03 December 2017].

Keywords

  • Virtual Reality
  • IoT
  • Automatic Question-Answering (QA) systems
  • speech recognition and synthesis
  • virtual assistants
  • voice user interface practical laboratories.

Fingerprint

Dive into the research topics of 'Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories'. Together they form a unique fingerprint.

Cite this