Conference-Based Usability Evaluation Methodology for Medical Web Applications

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Background: In the US, 100,000 deaths and 1,000,000 injuries occur each year due to medical errors. Whilst all medical errors are not caused by poorly designed software, many are. According to the 2007 ISO/IEC 62366 standard, a usability evaluation process must take place before the introduction of medical software into healthcare. Thus, software including Medicine 2.0 applications must also be evaluated for their usability. The standard method for assessing usability is via the ‘Usability Test’, which normally takes place in a ‘laboratory setting’. However, a laboratory-based approach can be expensive and time consuming.Objective: As such, we investigated the feasibility of carrying out a ‘Usability Test’ at a conference setting.Methods: In our previous work two Web-based systems were developed to visualise Electrocardiogram data. We carried out a usability evaluation of these applications at the 2010 International Conference on Computing in Cardiology. We recruited 17 delegates to partake in the evaluation. Each delegate attempted representative tasks using the software whilst ‘thinking aloud’. Each session was recorded using screencast software and a microphone.Results: From analysis of the video recordings, 70 previously unknown usability issues were discovered in the two applications. This alone indicates the utility of a conference-based approach. As part of the study, we also identified a number of advantages and disadvantages for carrying out a conference-based usability evaluation. The main advantage is the high concentration of experts at one geographical location. These experts also represent the international stage and it would be very expensive to arrange such worldwide experts to partake in a laboratory-based evaluation. Furthermore, in a conference setting, a usability evaluation can be carried out over a few days as opposed to weeks or months. This is contrary to a laboratory-based approach where weeks are dedicated to mailing invitation letters. A conference-based approach also allowed for effective advertisement of the study through the dissemination of information sheets within the delegate packs. It also allowed for face-to-face discussion regarding queries from potential subjects. Although delegates can volunteer to participate, a conference setting does also facilitate the solicitation of subjects. One of the most pertinent advantages is the fact that a conference-based approach does not require subjects to take additional time out and money to travel to a laboratory.Given the busy schedule of most conferences, a significant disadvantage is that a limited amount of time can be dedicated to each participant. Within this study, the average duration of a usability session was 34.69 minutes (SD=10.28). Given conference session breaks usually last somewhere between 30 minutes and one hour, this data shows the feasibility of integrating usability studies within a conference setting. We also found that only four user tasks could be performed within a 30-minute time period.Conclusions: Not only have we shown that the ‘Usability Test’ can be effectively carried out at a conference setting to detect ‘use errors’, we have also shown that this approach is less time consuming and more economic when compared to the conventional laboratory approach.
LanguageEnglish
Title of host publicationUnknown Host Publication
Number of pages1
Publication statusPublished - 24 Sep 2013
EventMedicine 2.0 Congress - London
Duration: 24 Sep 2013 → …

Conference

ConferenceMedicine 2.0 Congress
Period24/09/13 → …

Fingerprint

Cardiology
Video recording
Microphones
Electrocardiography
Medicine
Economics

Cite this

@inproceedings{c2212c711a1045df8fa8f5493f4fadfc,
title = "Conference-Based Usability Evaluation Methodology for Medical Web Applications",
abstract = "Background: In the US, 100,000 deaths and 1,000,000 injuries occur each year due to medical errors. Whilst all medical errors are not caused by poorly designed software, many are. According to the 2007 ISO/IEC 62366 standard, a usability evaluation process must take place before the introduction of medical software into healthcare. Thus, software including Medicine 2.0 applications must also be evaluated for their usability. The standard method for assessing usability is via the ‘Usability Test’, which normally takes place in a ‘laboratory setting’. However, a laboratory-based approach can be expensive and time consuming.Objective: As such, we investigated the feasibility of carrying out a ‘Usability Test’ at a conference setting.Methods: In our previous work two Web-based systems were developed to visualise Electrocardiogram data. We carried out a usability evaluation of these applications at the 2010 International Conference on Computing in Cardiology. We recruited 17 delegates to partake in the evaluation. Each delegate attempted representative tasks using the software whilst ‘thinking aloud’. Each session was recorded using screencast software and a microphone.Results: From analysis of the video recordings, 70 previously unknown usability issues were discovered in the two applications. This alone indicates the utility of a conference-based approach. As part of the study, we also identified a number of advantages and disadvantages for carrying out a conference-based usability evaluation. The main advantage is the high concentration of experts at one geographical location. These experts also represent the international stage and it would be very expensive to arrange such worldwide experts to partake in a laboratory-based evaluation. Furthermore, in a conference setting, a usability evaluation can be carried out over a few days as opposed to weeks or months. This is contrary to a laboratory-based approach where weeks are dedicated to mailing invitation letters. A conference-based approach also allowed for effective advertisement of the study through the dissemination of information sheets within the delegate packs. It also allowed for face-to-face discussion regarding queries from potential subjects. Although delegates can volunteer to participate, a conference setting does also facilitate the solicitation of subjects. One of the most pertinent advantages is the fact that a conference-based approach does not require subjects to take additional time out and money to travel to a laboratory.Given the busy schedule of most conferences, a significant disadvantage is that a limited amount of time can be dedicated to each participant. Within this study, the average duration of a usability session was 34.69 minutes (SD=10.28). Given conference session breaks usually last somewhere between 30 minutes and one hour, this data shows the feasibility of integrating usability studies within a conference setting. We also found that only four user tasks could be performed within a 30-minute time period.Conclusions: Not only have we shown that the ‘Usability Test’ can be effectively carried out at a conference setting to detect ‘use errors’, we have also shown that this approach is less time consuming and more economic when compared to the conventional laboratory approach.",
author = "Raymond Bond and Dewar Finlay and Chris Nugent and George Moore and Kyle Boyd",
year = "2013",
month = "9",
day = "24",
language = "English",
booktitle = "Unknown Host Publication",

}

Conference-Based Usability Evaluation Methodology for Medical Web Applications. / Bond, Raymond; Finlay, Dewar; Nugent, Chris; Moore, George; Boyd, Kyle.

Unknown Host Publication. 2013.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Conference-Based Usability Evaluation Methodology for Medical Web Applications

AU - Bond, Raymond

AU - Finlay, Dewar

AU - Nugent, Chris

AU - Moore, George

AU - Boyd, Kyle

PY - 2013/9/24

Y1 - 2013/9/24

N2 - Background: In the US, 100,000 deaths and 1,000,000 injuries occur each year due to medical errors. Whilst all medical errors are not caused by poorly designed software, many are. According to the 2007 ISO/IEC 62366 standard, a usability evaluation process must take place before the introduction of medical software into healthcare. Thus, software including Medicine 2.0 applications must also be evaluated for their usability. The standard method for assessing usability is via the ‘Usability Test’, which normally takes place in a ‘laboratory setting’. However, a laboratory-based approach can be expensive and time consuming.Objective: As such, we investigated the feasibility of carrying out a ‘Usability Test’ at a conference setting.Methods: In our previous work two Web-based systems were developed to visualise Electrocardiogram data. We carried out a usability evaluation of these applications at the 2010 International Conference on Computing in Cardiology. We recruited 17 delegates to partake in the evaluation. Each delegate attempted representative tasks using the software whilst ‘thinking aloud’. Each session was recorded using screencast software and a microphone.Results: From analysis of the video recordings, 70 previously unknown usability issues were discovered in the two applications. This alone indicates the utility of a conference-based approach. As part of the study, we also identified a number of advantages and disadvantages for carrying out a conference-based usability evaluation. The main advantage is the high concentration of experts at one geographical location. These experts also represent the international stage and it would be very expensive to arrange such worldwide experts to partake in a laboratory-based evaluation. Furthermore, in a conference setting, a usability evaluation can be carried out over a few days as opposed to weeks or months. This is contrary to a laboratory-based approach where weeks are dedicated to mailing invitation letters. A conference-based approach also allowed for effective advertisement of the study through the dissemination of information sheets within the delegate packs. It also allowed for face-to-face discussion regarding queries from potential subjects. Although delegates can volunteer to participate, a conference setting does also facilitate the solicitation of subjects. One of the most pertinent advantages is the fact that a conference-based approach does not require subjects to take additional time out and money to travel to a laboratory.Given the busy schedule of most conferences, a significant disadvantage is that a limited amount of time can be dedicated to each participant. Within this study, the average duration of a usability session was 34.69 minutes (SD=10.28). Given conference session breaks usually last somewhere between 30 minutes and one hour, this data shows the feasibility of integrating usability studies within a conference setting. We also found that only four user tasks could be performed within a 30-minute time period.Conclusions: Not only have we shown that the ‘Usability Test’ can be effectively carried out at a conference setting to detect ‘use errors’, we have also shown that this approach is less time consuming and more economic when compared to the conventional laboratory approach.

AB - Background: In the US, 100,000 deaths and 1,000,000 injuries occur each year due to medical errors. Whilst all medical errors are not caused by poorly designed software, many are. According to the 2007 ISO/IEC 62366 standard, a usability evaluation process must take place before the introduction of medical software into healthcare. Thus, software including Medicine 2.0 applications must also be evaluated for their usability. The standard method for assessing usability is via the ‘Usability Test’, which normally takes place in a ‘laboratory setting’. However, a laboratory-based approach can be expensive and time consuming.Objective: As such, we investigated the feasibility of carrying out a ‘Usability Test’ at a conference setting.Methods: In our previous work two Web-based systems were developed to visualise Electrocardiogram data. We carried out a usability evaluation of these applications at the 2010 International Conference on Computing in Cardiology. We recruited 17 delegates to partake in the evaluation. Each delegate attempted representative tasks using the software whilst ‘thinking aloud’. Each session was recorded using screencast software and a microphone.Results: From analysis of the video recordings, 70 previously unknown usability issues were discovered in the two applications. This alone indicates the utility of a conference-based approach. As part of the study, we also identified a number of advantages and disadvantages for carrying out a conference-based usability evaluation. The main advantage is the high concentration of experts at one geographical location. These experts also represent the international stage and it would be very expensive to arrange such worldwide experts to partake in a laboratory-based evaluation. Furthermore, in a conference setting, a usability evaluation can be carried out over a few days as opposed to weeks or months. This is contrary to a laboratory-based approach where weeks are dedicated to mailing invitation letters. A conference-based approach also allowed for effective advertisement of the study through the dissemination of information sheets within the delegate packs. It also allowed for face-to-face discussion regarding queries from potential subjects. Although delegates can volunteer to participate, a conference setting does also facilitate the solicitation of subjects. One of the most pertinent advantages is the fact that a conference-based approach does not require subjects to take additional time out and money to travel to a laboratory.Given the busy schedule of most conferences, a significant disadvantage is that a limited amount of time can be dedicated to each participant. Within this study, the average duration of a usability session was 34.69 minutes (SD=10.28). Given conference session breaks usually last somewhere between 30 minutes and one hour, this data shows the feasibility of integrating usability studies within a conference setting. We also found that only four user tasks could be performed within a 30-minute time period.Conclusions: Not only have we shown that the ‘Usability Test’ can be effectively carried out at a conference setting to detect ‘use errors’, we have also shown that this approach is less time consuming and more economic when compared to the conventional laboratory approach.

M3 - Conference contribution

BT - Unknown Host Publication

ER -