Abstract
Background
The evaluation of clinical skills and competencies is a high-stakes process carrying significant consequences for the candidate. Hence, it is mandatory to have a robust method to justify the pass score in order to maintain a valid and reliable objective structured clinical examination (OSCE). The aim was to trial the borderline approach using the two-domain global rating scale for standard setting in the OSCE.
Methods
For each domain, a set of six-point (from 5 to 0) scales were used to reflect high and low divisions within the ‘pass’, ‘borderline’ and ‘fail’ categories. Scores on the two individual global scales were summed to create a ‘summed global rating’. Similarly task-based checklists for individual stations were summed to get a total score.
Results
The Pearson's correlation between task-based checklist scoring and the two-domain global rating scale were moderate and significant. The highest R2 coefficient of 0.479 was obtained for station 7, and the lowest R2 value was 0.241 for station 14.
Discussion
There was a significant positive correlation between the two scales; however, the R2 value was not satisfactory except for station 7. The pass mark for the OSCE according to the borderline method was 64 per cent, which is higher than the arbitrarily set pass mark of 50 per cent.
Conclusions
This study confirms that the two-domain global rating scale is appropriate to assess the abilities of students within the framework of an OSCE. The strong relationships between the two-domain global rating scale and task-based checklists provide evidence that the two-domain global rating scale can be used to genuinely assess students’ proficiencies.
The evaluation of clinical skills and competencies is a high-stakes process carrying significant consequences for the candidate. Hence, it is mandatory to have a robust method to justify the pass score in order to maintain a valid and reliable objective structured clinical examination (OSCE). The aim was to trial the borderline approach using the two-domain global rating scale for standard setting in the OSCE.
Methods
For each domain, a set of six-point (from 5 to 0) scales were used to reflect high and low divisions within the ‘pass’, ‘borderline’ and ‘fail’ categories. Scores on the two individual global scales were summed to create a ‘summed global rating’. Similarly task-based checklists for individual stations were summed to get a total score.
Results
The Pearson's correlation between task-based checklist scoring and the two-domain global rating scale were moderate and significant. The highest R2 coefficient of 0.479 was obtained for station 7, and the lowest R2 value was 0.241 for station 14.
Discussion
There was a significant positive correlation between the two scales; however, the R2 value was not satisfactory except for station 7. The pass mark for the OSCE according to the borderline method was 64 per cent, which is higher than the arbitrarily set pass mark of 50 per cent.
Conclusions
This study confirms that the two-domain global rating scale is appropriate to assess the abilities of students within the framework of an OSCE. The strong relationships between the two-domain global rating scale and task-based checklists provide evidence that the two-domain global rating scale can be used to genuinely assess students’ proficiencies.
| Original language | English |
|---|---|
| Pages (from-to) | 551-556 |
| Number of pages | 6 |
| Journal | The Clinical Teacher |
| Volume | 11 |
| Issue number | 7 |
| Early online date | 24 Nov 2014 |
| DOIs | |
| Publication status | Published (in print/issue) - 1 Dec 2014 |