Ottawa Symposia 5 : Researching Workplace Based Assessment (WBA)

14:40 – 16:10 , 13th MARCH 2018
Brian Jolly (University of Newcastle, Australia), Jenny Weller (University of Auckland, New Zealand),  David Swanson (American Board of Medical Specialties, United States &University of Melbourne, Australia), Kichu Balakrishan Nair (University of Newcastle), Prof. Cees van der Vleuten (University of Maastricht)
Abu Dhabi National Exhibition Centre (ADNEC), Abu Dhabi

Details:

HSDs in health professions assessment have implications for trainees and teachers, but also for current and future patients. Robust evidence is required to support the use of WBA for this purpose.

There is evidence of reliability for WBA in multiple contexts. But no single WBA will provide all the information required for high-stakes decisions Critics of current analytic methods, such as estimated composite reliability using Generalizability theory, suggest this evidence ignores the confounding of training sites, rater stringency, and trainee proficiency inherent in unavoidable naturalistic designs, resulting in overestimates of the reproducibility of performance ratings, hence arguing against their use in making HSD, particularly across programmes in different locations.

The type of rating scale used may also influence WBA validity; ‘entrustability’ scales have become more prevalent. How their use might influence HSD and the issues above is beginning to be explored.