Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories

MJ Callaghan, Gildas Bengloan, Julien Ferrer, Léo Cherel, Mohamed Ali El Mostadi, Augusto Gómez Eguíluz, Niall McShane

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The first generation of affordable consumer virtual reality headsets and related peripherals are now available. Question-Answering (QA) systems and speech recognition/synthesis has improved dramatically over the last decade. Virtual assistants, based on speech-based services are growing in popularity and can be used in a range of diverse application areas. This paper explores the practical use of virtual reality, IOT and voice driven virtual assistants in remote laboratories to facilitate visualization of electrical phenomena and to tutor students; guiding them through each stage of an experiment; presenting supplementary teaching resources when requested; accessing, controlling and configuring instrumentation and hardware and providing feedback with summative and formative assessment. The re-purposing of existing teaching material for use in an immersive environment with a virtual assistant is shown and the limitations and opportunities offered by the approach taken and the technologies used are discussed. The process of integrating test instrumentation, the board under test, a switching matrix and additional teaching resources into virtual reality using IOT is discussed and the inclusion of virtual assistants described. Two case studies of practical working examples of remote laboratories in virtual reality with a virtual assistant tutor are demonstrated and the viability and long-term opportunities for the use of virtual reality and virtual assistants in the context discussed
LanguageEnglish
Title of host publicationUnknown Host Publication
Number of pages13
Publication statusAccepted/In press - 23 Dec 2017
Event15th International Conference on Remote Engineering and Virtual Instrumentation (REV) - Duesseldorf, Germany
Duration: 23 Dec 2017 → …

Conference

Conference15th International Conference on Remote Engineering and Virtual Instrumentation (REV)
Period23/12/17 → …

Fingerprint

Electronics engineering
Electrical engineering
Virtual reality
Teaching
Speech recognition
Visualization
Students
Feedback
Hardware
Experiments

Keywords

  • Virtual Reality
  • IoT
  • Automatic Question-Answering (QA) systems
  • speech recognition and synthesis
  • virtual assistants
  • voice user interface practical laboratories.

Cite this

Callaghan, MJ., Bengloan, G., Ferrer, J., Cherel, L., El Mostadi, M. A., Gómez Eguíluz, A., & McShane, N. (Accepted/In press). Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories. In Unknown Host Publication
Callaghan, MJ ; Bengloan, Gildas ; Ferrer, Julien ; Cherel, Léo ; El Mostadi, Mohamed Ali ; Gómez Eguíluz, Augusto ; McShane, Niall. / Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories. Unknown Host Publication. 2017.
@inproceedings{5c6f65741256400b93b429d2888b6b14,
title = "Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories",
abstract = "The first generation of affordable consumer virtual reality headsets and related peripherals are now available. Question-Answering (QA) systems and speech recognition/synthesis has improved dramatically over the last decade. Virtual assistants, based on speech-based services are growing in popularity and can be used in a range of diverse application areas. This paper explores the practical use of virtual reality, IOT and voice driven virtual assistants in remote laboratories to facilitate visualization of electrical phenomena and to tutor students; guiding them through each stage of an experiment; presenting supplementary teaching resources when requested; accessing, controlling and configuring instrumentation and hardware and providing feedback with summative and formative assessment. The re-purposing of existing teaching material for use in an immersive environment with a virtual assistant is shown and the limitations and opportunities offered by the approach taken and the technologies used are discussed. The process of integrating test instrumentation, the board under test, a switching matrix and additional teaching resources into virtual reality using IOT is discussed and the inclusion of virtual assistants described. Two case studies of practical working examples of remote laboratories in virtual reality with a virtual assistant tutor are demonstrated and the viability and long-term opportunities for the use of virtual reality and virtual assistants in the context discussed",
keywords = "Virtual Reality, IoT, Automatic Question-Answering (QA) systems, speech recognition and synthesis, virtual assistants, voice user interface practical laboratories.",
author = "MJ Callaghan and Gildas Bengloan and Julien Ferrer and L{\'e}o Cherel and {El Mostadi}, {Mohamed Ali} and {G{\'o}mez Egu{\'i}luz}, Augusto and Niall McShane",
note = "Reference text: 1. L. De Floriani and D. Schmalstieg, {"}IEEE Virtual Reality 2017 Special Issue,{"} in IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 4, pp. v-v, April 2017 2. A. Bouziane, D. Bouchiha, N. Doumi, M. Malki, Question Answering Systems: Survey and Trends, Procedia Computer Science, Volume 73, 2015, Pages 366-375, ISSN 1877-0509. 3. Khillare S A Ajitkumar M. Pundge and Namrata C Mahender. Question Answering System, Approaches and Techniques. Int. Jour. of Computer Applications 141(3):34-39, May 2016 4. Microsoft. 2017 Cortana Intelligence services. [ONLINE] Available at https://azure.microsoft.com/en-gb/overview/cortana-intelligence. [Accessed 03 December 2017]. 5. Feisel, L. D. and Rosa, A. J. (2005), The Role of the Laboratory in Undergraduate Engineering Education. Journal of Engineering Education, 94: 121–130. 6. Black, A.W., & Taylor, P. (1997). Automatically clustering similar units for unit selection in speech synthesis. Proceedings of Eurospeech 1997 (pp. 601–604). Athens, Greece. 7. Harman Kardon Invoke review. [ONLINE] Available at: https://www.theverge.com/2017/10/20/16505468/harman-kardon-invoke-cortana-microsoft-smart-speaker-review. [Accessed 03 December 2017]. 8. Cortana Intelligence. [ONLINE] Available at: https://www.microsoft.com/en-gb/cloud-platform/what-is-cortana-intelligence. [Accessed 03 December 2017]. 9. E. Lindsay, D. Liu, S. Murray, and D. Lowe, “Remote Laboratories in Engineering Ed-ucation: Students’ Perceptions,” Proc. 18th Ann. Conf. Assoc. for Eng. Edu. (AaeE ’07), 2007 10. D. Lowe, S. Murray, E. Lindsay and D. Liu, {"}Evolving Remote Laboratory Architectures to Leverage Emerging Internet Technologies,{"} in IEEE Transactions on Learning Technologies, vol. 2, no. 4, pp. 289-294, Oct.-Dec. 2009. doi: 10.1109/TLT.2009.33 11. Connect Raspberry Pi to Azure IoT Hub [ONLINE] https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-raspberry-pi-kit-node-get-started [Accessed 03 December 2017]. 12. M.J. Callaghan, K. McCusker, J.L. Losada, J. Harkin, S. Wilson. {"}Using Game Based Learning in Virtual Worlds to Teach Electronic and Electrical Engineering, IEEE Trans. on Industrial Informatics, vol. 9, no. 1, pp. 575 - 584, Feb 2013. 13. Cognigy AI [ONLINE] https://www.cognigy.com/ [Accessed 03 December 2017].",
year = "2017",
month = "12",
day = "23",
language = "English",
booktitle = "Unknown Host Publication",

}

Callaghan, MJ, Bengloan, G, Ferrer, J, Cherel, L, El Mostadi, MA, Gómez Eguíluz, A & McShane, N 2017, Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories. in Unknown Host Publication. 15th International Conference on Remote Engineering and Virtual Instrumentation (REV), 23/12/17.

Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories. / Callaghan, MJ; Bengloan, Gildas; Ferrer, Julien; Cherel, Léo; El Mostadi, Mohamed Ali; Gómez Eguíluz, Augusto; McShane, Niall.

Unknown Host Publication. 2017.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories

AU - Callaghan, MJ

AU - Bengloan, Gildas

AU - Ferrer, Julien

AU - Cherel, Léo

AU - El Mostadi, Mohamed Ali

AU - Gómez Eguíluz, Augusto

AU - McShane, Niall

N1 - Reference text: 1. L. De Floriani and D. Schmalstieg, "IEEE Virtual Reality 2017 Special Issue," in IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 4, pp. v-v, April 2017 2. A. Bouziane, D. Bouchiha, N. Doumi, M. Malki, Question Answering Systems: Survey and Trends, Procedia Computer Science, Volume 73, 2015, Pages 366-375, ISSN 1877-0509. 3. Khillare S A Ajitkumar M. Pundge and Namrata C Mahender. Question Answering System, Approaches and Techniques. Int. Jour. of Computer Applications 141(3):34-39, May 2016 4. Microsoft. 2017 Cortana Intelligence services. [ONLINE] Available at https://azure.microsoft.com/en-gb/overview/cortana-intelligence. [Accessed 03 December 2017]. 5. Feisel, L. D. and Rosa, A. J. (2005), The Role of the Laboratory in Undergraduate Engineering Education. Journal of Engineering Education, 94: 121–130. 6. Black, A.W., & Taylor, P. (1997). Automatically clustering similar units for unit selection in speech synthesis. Proceedings of Eurospeech 1997 (pp. 601–604). Athens, Greece. 7. Harman Kardon Invoke review. [ONLINE] Available at: https://www.theverge.com/2017/10/20/16505468/harman-kardon-invoke-cortana-microsoft-smart-speaker-review. [Accessed 03 December 2017]. 8. Cortana Intelligence. [ONLINE] Available at: https://www.microsoft.com/en-gb/cloud-platform/what-is-cortana-intelligence. [Accessed 03 December 2017]. 9. E. Lindsay, D. Liu, S. Murray, and D. Lowe, “Remote Laboratories in Engineering Ed-ucation: Students’ Perceptions,” Proc. 18th Ann. Conf. Assoc. for Eng. Edu. (AaeE ’07), 2007 10. D. Lowe, S. Murray, E. Lindsay and D. Liu, "Evolving Remote Laboratory Architectures to Leverage Emerging Internet Technologies," in IEEE Transactions on Learning Technologies, vol. 2, no. 4, pp. 289-294, Oct.-Dec. 2009. doi: 10.1109/TLT.2009.33 11. Connect Raspberry Pi to Azure IoT Hub [ONLINE] https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-raspberry-pi-kit-node-get-started [Accessed 03 December 2017]. 12. M.J. Callaghan, K. McCusker, J.L. Losada, J. Harkin, S. Wilson. "Using Game Based Learning in Virtual Worlds to Teach Electronic and Electrical Engineering, IEEE Trans. on Industrial Informatics, vol. 9, no. 1, pp. 575 - 584, Feb 2013. 13. Cognigy AI [ONLINE] https://www.cognigy.com/ [Accessed 03 December 2017].

PY - 2017/12/23

Y1 - 2017/12/23

N2 - The first generation of affordable consumer virtual reality headsets and related peripherals are now available. Question-Answering (QA) systems and speech recognition/synthesis has improved dramatically over the last decade. Virtual assistants, based on speech-based services are growing in popularity and can be used in a range of diverse application areas. This paper explores the practical use of virtual reality, IOT and voice driven virtual assistants in remote laboratories to facilitate visualization of electrical phenomena and to tutor students; guiding them through each stage of an experiment; presenting supplementary teaching resources when requested; accessing, controlling and configuring instrumentation and hardware and providing feedback with summative and formative assessment. The re-purposing of existing teaching material for use in an immersive environment with a virtual assistant is shown and the limitations and opportunities offered by the approach taken and the technologies used are discussed. The process of integrating test instrumentation, the board under test, a switching matrix and additional teaching resources into virtual reality using IOT is discussed and the inclusion of virtual assistants described. Two case studies of practical working examples of remote laboratories in virtual reality with a virtual assistant tutor are demonstrated and the viability and long-term opportunities for the use of virtual reality and virtual assistants in the context discussed

AB - The first generation of affordable consumer virtual reality headsets and related peripherals are now available. Question-Answering (QA) systems and speech recognition/synthesis has improved dramatically over the last decade. Virtual assistants, based on speech-based services are growing in popularity and can be used in a range of diverse application areas. This paper explores the practical use of virtual reality, IOT and voice driven virtual assistants in remote laboratories to facilitate visualization of electrical phenomena and to tutor students; guiding them through each stage of an experiment; presenting supplementary teaching resources when requested; accessing, controlling and configuring instrumentation and hardware and providing feedback with summative and formative assessment. The re-purposing of existing teaching material for use in an immersive environment with a virtual assistant is shown and the limitations and opportunities offered by the approach taken and the technologies used are discussed. The process of integrating test instrumentation, the board under test, a switching matrix and additional teaching resources into virtual reality using IOT is discussed and the inclusion of virtual assistants described. Two case studies of practical working examples of remote laboratories in virtual reality with a virtual assistant tutor are demonstrated and the viability and long-term opportunities for the use of virtual reality and virtual assistants in the context discussed

KW - Virtual Reality

KW - IoT

KW - Automatic Question-Answering (QA) systems

KW - speech recognition and synthesis

KW - virtual assistants

KW - voice user interface practical laboratories.

M3 - Conference contribution

BT - Unknown Host Publication

ER -

Callaghan MJ, Bengloan G, Ferrer J, Cherel L, El Mostadi MA, Gómez Eguíluz A et al. Voice Driven Virtual Assistant Tutor in Virtual Reality for Electronic and Electrical Engineering Remote Laboratories. In Unknown Host Publication. 2017