Symposia

Ottawa

Grading and Ranking in Assessment: Help or Hindrance in education of healthcare professionals

Dujeepa Samarasekera, Choon Eng Matthew Gwee (National University of Singapore, Singapore)

Today the team-care of patients is considered to enhance patient well-being including patient safety. Thus, the need for collaborative learning is now strongly emphasised in health professional education. Moreover, inter-professional education is also strongly advocated to optimise collaborative learning across professional disciplines and courses in preparation for future team-care of patients. However, grading and ranking are currently an important and a common practice in the

assessment of student acquisition of knowledge, skills and attitudes in health professional education. Grading and ranking in assessment tend to identify ‘star’ performers in a cohort of

students. Such ‘stars’ are likely to display individualistic and competitive behaviour, whereas the team-care of patients will require practitioners from different healthcare professions to display significant collaborative behaviour when taking care of patients collectively as a team. In recognition of the need for future healthcare practitioners to be equipped with such collaborative skills, health professional schools now try to nurture collaborative learning in several parts of their educational programmes, including inter-professional education. Much time and energy of academic faculty are invested in such programmes.

The question then is, will grading and ranking in assessment hinder efforts at promoting collaborative learning of students who are likely to become the next generation of leaders in the professions?

This Symposium will discuss the issues raised and share experiences from a global perspective.

 

Emerging Trends and Initiatives in the Implementation and Assessment of Entrustable Professional Activities (EPAs)

Machelle Linsenmeyer (West Virginia School of Osteopathic Medicine, USA), Olle ten Cate (University Medical Center, Netherlands), Carrie Chen (Georgetown University School of Medicine, USA), Claire Touchie (University of Ottawa, Canada)

Competency based education is being used around the globe to formulate the values of the medical profession in words that enable a new perspective on curricula, training, and assessment.  To help facilitate translation of competency expectations into practice, entrustable professional activities (EPAs) emerged.  EPAs link competencies to the tasks or responsibilities that can be entrusted to a trainee once sufficient, specific competence is reached to allow for unsupervised execution.  Framing competencies in the context of what physicians do in daily practice can allow for assessments that align with observations in the clinical environment.   Efforts to move these initiatives forward for assessment of learners in the health professions have been developing throughout the world.   Panel members will outline several of these initiatives and provide an opportunity for audience members to pose specific questions for discussion.

 

Out with the old, in with the new: Using conceptual frameworks to re-imagining research in selection and assessment

Kelly Dore (McMaster University, Canada), Chris Roberts (University of Sydney, Australia), Sarah  Wright (University of Toronto, Canada), Jonathan Dowell (University of Dundee, UK), Meghan McConnell (University of Ottawa, Canada)

This symposium will link psychometrics, selection, curriculum, and program outcomes in a socially accountable way. Internationally we have seen two main changing priorities in the selection. Firstly, the overall focus on selection parameters has shifted from identifying the most academically able trainees to achieving broader societal goals of increased representation. Secondly, an increasing emphasis on psychometrically driven selection (competency based) frameworks over locally defined selection methods. Given this paradigm shift, using a tool that withstands rigorous psychometric analyses without examining implications no longer seems adequate.  However, processes combining assessment tools within an overarching framework to assess diverse aspects of applicants’ abilities have not been well established. Logic models will be a case study of how application of conceptual frameworks can better address research across assessment by highlighting the critical components of the process, stakeholder perspectives, and evaluation methods.  Finally, the symposium will address some of the challenges of implementing (and studying) selection methods with differing theoretical underpinnings and stakeholder perspectives. The presenters bring a wealth of selection experiences and will share innovative theoretical insights from their different international contexts.

 

Emerging Concepts and Practical Applications of Validity

David Cook (Mayo Clinic College of Medicine and Science, USA), Meredith Young (McGill University, Canada), Claire Touchie (Medical Council of Canada, Canada), Susan Humphrey-Murto (University of Ottawa, Canada), Rodrigo Cavalcanti (University of Toronto, Canada)

Validity is a fundamental element in judging the quality and trustworthiness of assessments. Conceptions of validity have evolved from a “classical” model (content, criterion, construct), to a unified model supported by five evidence sources (Messick), to an argument-based approach focused on key inferences (Kane). While these distinctions can seem esoteric, understanding validity can help improve all aspects of assessment across the continuum from admissions to maintenance of competence. This symposium will highlight the importance of validity in evaluating assessment quality, and demonstrate the practical benefits of validation in the work of educators and researchers. We will contrast several validity frameworks, and describe strengths and weaknesses of each. We will then explore validity in four specific contexts:

1. Assessment of clinical reasoning

2. Assessment for high-stakes decisions

3. Consensus methods in assessment and research

4. Measurement of non-learning constructs such as cognitive load.

 

Researching Workplace Based Assessment (WBA)

Brian Jolly (University of Newcastle, Australia), Jenny Weller (University of Auckland, New Zealand), Joyce Moonen van Loon (University of Maastricht, The Netherlands), David Swanson (American Board of Medical Specialties, United States &University of Melbourne, Australia), Damian Castanelli (Monash University, Australia)

HSDs in health professions assessment have implications for trainees and teachers, but also for current and future patients. Robust evidence is required to support the use of WBA for this purpose.

There is evidence of reliability for WBA in multiple contexts. But no single WBA will provide all the information required for high-stakes decisions Critics of current analytic methods, such as estimated composite reliability using Generalizability theory, suggest this evidence ignores the confounding of training sites, rater stringency, and trainee proficiency inherent in unavoidable naturalistic designs, resulting in overestimates of the reproducibility of performance ratings, hence arguing against their use in making HSD, particularly across programmes in different locations.

The type of rating scale used may also influence WBA validity; ‘entrustability’ scales have become more prevalent. How their use might influence HSD and the issues above is beginning to be explored.

 

The Role and Effect of Accreditation Systems and Competency based Education in Advancing Social Accountability: a post Tunisia Summit Assessment

Bob Woollard, Boelen Charles, William Burdick

Through engagement of the “pentagram partners” (community, management, policy makers, health professions and academies.) in a process of appreciative inquiry to gather further evidence from around the world of successful innovations in education, accreditation and action towards social accountability in medical education.  The symposium will build upon this experience to re-assess the post Summit accomplishments and prioritize next steps.

ICME

Towards a competency-based curriculum: Aligning and mapping competencies

Ahmed Al-Rumayyan, Mohi Eldin Magzoub (King Sau Bin Abdulaziz University For Health Sciences, KSA), Hossam Hamdy (Qatar University, Qatar)

Competency and outcome-based medical education is an important strategy in medical education for making the curriculum relevant to medical practice and the health system. The balanced and comprehensive incorporation of all competencies in the curriculum remains a big challenge for medical educators and curriculum planners. Furthermore the assessment of certain competencies such as professionalism and other soft skills with reasonable validity and reliability is an unresolved problem. Proper alignment and and mapping of competencies and learning outcomes with different parameters of the curriculum such as learning objectives, teaching instructions, scheduling with each other and with assessment may provide a reasonable solutions to these challenges.

 

Patient safety in a Modern day World – Learning how to Recognize, Address, Decrease and Eliminate Risks in Medical Care while Maintaining Best Practices.

Prof. Mohamud Verjee, Dr. Sunanda Holmes, Prof. Adeel Butt, Dr. Paul Amuna

Servant leadership lessons include the creation of a learning environment, cultivating trust and stimulating empathy between leaders, trainees and employees. Social nurturing is integral to the effective delivery of safe, high standard care, taking into account all medical advances available to the profession up to today. Discussing the style of best teachers who make errors and making them learning points for others is a matter of course and not just of fact. Systemic change requires interprofessional dialogue. The rate of reported mistakes in care does not match the actual incidence of errors in practice, and this aspect of patient safety should be aired openly and not be kept covert. Globally, the trend is the same, with no uniform protocol of policy, and a lack of consistency in either warning or penalty for the institution or persons involved. Silence does not mean that all is well. Systemic problems may lie in the design of work environment and true teamwork. Suggested solutions from the Great Ormond Street Hospital (GOSH) study, with other examples such as the Toyota and Ferrari models in industry teach us applied principles of care and failsafe opportunities to prevent damage, improving efficiency of performances at every possible occasion.

 

Empathy in medical training; translating principles into real world practice.

William T. O’Connor (University of Limerick Medial School, Ireland)

Empathy is particularly important in considering others especially when making decisions and is therefore a basic component of all helpful human relationships including effective, therapeutic interventions. Yet empathy is often lacking in professional practice, as frequently reported by patients [1]. In this regard, it has been reported that empathy significantly declines in the third year of medical school, just when extensive exposure to clinical settings typically first occurs [2]. This main aim in this proposal is to address a gap in medical education by providing the scientific rationale for establishing opportunities for creative and effective empathy and engagement in medical practice.

 

Integrating ‘Patient Safety’ in undergraduate medical curriculum

Rahila Yasmeen, Rehan Ahmed Khan (Riphah International University, Pakistan), Gohar Wajid (WHO, Egypt), Ahsen Sethi (Khyber Medical University, Pakistan)

‘Patient Safety’ concepts and principles are important to understand in order to integrate its content in the undergraduate medical curriculum. This symposium will help in understanding and aligning the content of ‘Patient Safety’ in medical curriculum.

Patient safety in a modern day world – learning how to recognize, address, decrease and eliminate risks in medical care while maintaining best practices.

Prof. Mohamud Verjee (Weill Cornell Medicine-Qatar, Qatar), Sunanda Holmes (Weill Cornell Medicine-Qatar, Qatar), Adeel Butt (Hamad Medical Corporation, Qatar) Paul Amuna (Primary Health Care Corporation, Qatar)

Servant leadership lessons include the creation of a learning environment, cultivating trust and stimulating empathy between leaders, trainees and employees. Social nurturing is integral to the effective delivery of safe, high standard care, taking into account all medical advances available to the profession up to today. Discussing the style of best teachers who make errors and making them learning points for others is a matter of course and not just of fact. Systemic change requires interprofessional dialogue. The rate of reported mistakes in care does not match the actual incidence of errors in practice, and this aspect of patient safety should be aired openly and not be kept covert. Globally, the trend is the same, with no uniform protocol of policy, and a lack of consistency in either warning or penalty for the institution or persons involved. Silence does not mean that all is well. Systemic problems may lie in the design of work environment and true teamwork. Suggested solutions from the Great Ormond Street Hospital (GOSH) study, with other examples such as the Toyota and Ferrari models in industry teach us applied principles of care and failsafe opportunities to prevent damage, improving efficiency of performances at every possible occasion.