Pre-Conference Workshops

OTTAWA

Ottawa PCW 1

Blueprinting Assessment

Carmel Tepper (Bond University, Australia)

This is a highly interactive workshop where participants will use their own laptops to create blueprints based on their own faculty assessment needs. Participants will use excel® to create Year level blueprints that show assessment types and weightings, as well as exam level blueprints that show points per exam, per item type and per learning outcome.

Level: Introductory

 

Ottawa PCW 2

Selection methods for assessing non-academic attributes in medical education

Fiona Patterson (Work Psychology Group, UK)

Internationally, medical admissions are high-stakes and competitive. Therefore, selection methods should reliably identify whether candidates are likely to succeed in practice. Research shows that, as well as academic attainment, non-academic attributes are important predicators of performance and training. SJTs and MMIs may offer reliable and valid assessment solutions. This workshop summarises Medical admissions research, provides opportunity to practice SJT and MMI design, and gives insight into practicalities e.g. assessor calibration.

Level: Introductory

 

Ottawa PCW 3

Using OSTEs (Objective structured teaching encounters) to train, develop and assess clinical teachers

David Taylor (University Of Liverpool, UK); Hossam Hamdy (Gulf Medical University, UAE)

We use a combination of formal face-to-face instruction, discussions in small groups and role play activities. Each participant will be involved in developing the problems, and assessment schedules, which are then used in OSTE stations.  They will have the possibility to assess and critique their performance and that of their peers.  The emphasis is on ensuring reliable and valid assessment, which provides feedback of direct use to the participant.

Level: Introductory

 

Ottawa PCW 4

Psychometrics for the non-Psychometrician: A Practical Perspective

John Boulet, (ECFMG, USA); Jeanne Sandella (National Board of Osteopathic Medical Examiners, USA)

The primary goal of any assessment is to provide a measure, or measures, of knowledge, skill, or attitude that reflects actual ability.  Evidence to support the validity and reproducibility of assessment scores must be obtained if meaningful inferences regarding ability are to be made. Various exercises will provide participants with the knowledge and tools needed to gather evidence to support arguments that their assessments yield valid and reliable scores.

Level: Introductory

 

Ottawa PCW 5

The 10 Most Avoidable Assessment Flaws

Ara Tekian (University Of Illinois At Chicago, USA); John Norcini (FAIMER, USA)

Assessment drives, creates, and ensures learning, yet we often use methods and processes that are seriously flawed. In this workshop, we will describe our top 10 assessment flaws. Included will be problems such as the misalignment of purpose and method, failure to create a blueprint, misuse of scores, lack of reliability, and failure to apply a rational standard-setting process. For each flaw, we will propose remedies intended to improve the quality of individual methods and the assessment system as a whole.

Level: Intermediate

 

Ottawa PCW 6

Can we ‘personalise’ large scale standardised assessment? Implementing and measuring the impact of Sequential Testing in practice

Richard Fuller, Matthew  Homer (University Of Leeds, UK)

SQT methodologies are relatively new assessment formats, where all candidates take a main ‘screening’ test, with lower performing candidates subject to an additional (sequential) test of similar magnitude. This assessment design helps us deal with the challenges of feasibility, quality and cost whilst undertaking and delivering ‘fair’ high stakes assessment.

Level: Intermediate

 

Ottawa PCW 7

Authentic assessment of nontraditional discipline-independent (non-technical) skills in basic science curricula

Wojciech Pawlina, Nirusha Lachman, Natalie Langley (Mayo Clinic College of Medicine and Science, USA); Darrell Evans (Monash University, Australia)

This interactive workshop will use practical exercises and round-table discussion emphasizing incorporation and assessments of non traditional discipline-independent skills (NTDIS) in basic science curricula. An initial overview of existing literature in this area will be followed by group work reviewing case materials, examples of practical methodologies, and challenges in authentic assessments.  The workshop will conclude with advice based on researched effective practice and guidance on how to implement and assess NTDIS.

Level: Intermediate

 

Ottawa PCW 8

Using the Rasch model to improve assessments and investigate academic integrity issues

Deborah O’Mara, Ms Imogene Rothnie (Sydney Medical School, Australia)

This workshop will provide an introduction to the Rasch model from IRT. Much of the workshop will be interactive to provide participants with experience interpreting Rasch output using de-identified medical education data based on multiple choice questions. Guidelines for applying Rasch estimation using Winsteps software will be demonstrated. Practical examples of data preparation, item analysis, test equating with item anchors will be considered as well as person ability statistics.

Level: Introductory

 

Ottawa PCW 9

Use of Generalizability Theory in Designing and Analyzing Objective Structured Clinical Examinations (OSCEs) and Oral Examinations

David Swanson (American Board of Medical Specialties, USA & University of Melbourne Medical School, Australia)

Assessments involving multiple sources of measurement error (eg, rater stringency, task difficulty, content specificity) are commonly used in the health professions to measure clinical skills. Classical test theory does not furnish adequate tools for investigation of psychometric characteristics of such assessment methods. This workshop provides an introduction to generalizability theory, which does supply the appropriate tools to estimate the reproducibility of scores and evaluate alternate approaches to test design and use of testing resources.

Level: Intermediate

 

Ottawa PCW 10

Development and Implementation of online assessment systems (OAS)  for health professions education based on educational frameworks: practical tips for success .

Vishna Devi Nadarajah (International Medical University, Malaysia); HUI MENG ER (International Medical University, Malaysia)

At the workshop, we show participants how to propose, project manage and develop OAS contextual to their needs. We share practical tips from our experience in developing an integrated, reliable, secure and valid OAS for outcomes based education with a range of assessment tools for health professions education. Participants will be further engaged by working in groups, identifying resources and how to overcome challenges using checklists developed from stakeholders perspectives.

Level: Intermediate

 

Ottawa PCW 11

Using Assessment to Diagnose and Treat the Failing Clinical Student

Debra Klamen, Anna Cianciolo (Southern Illinois University School of Medicine, USA)

Students fail clinical performance examinations.  Yet we do not  have explicit guidelines to help them.  Remediation is often rotely applied and/or too general.  The core problem behind the failure is never clarified.  “Treatments” need to be specific and targeted.  This workshop will teach participants to use assessment strategies to  ‘diagnose’ and ‘treat’ those learners who are deficient in clinical performance through practice on anonymized student cases.

Level: Introductory

 

Ottawa PCW 12

Strategies to use portfolios as a meaningful part of competency-based assessment across the continuum of education in the health professions: examples from the broader world of education and applications for health professions education.

Lindsey Lane, Janice Hanson (University Of Colorado, USA); Ulcca Joshi Hansen (University of Denver, USA)

Presenters will share the opportunities and challenges of using portfolios in health education, including the theoretical and practical alignment with learning and assessment theory. Attendees will synthesize and apply this information, developing a portfolio proposal for their institution/setting. This will include a rationale statement, an outline for portfolio structures, recommendations for specific data/pieces that meaningfully represent the clinical performance of their learners, and proposed data collection processes.

Level: Intermediate

 

Ottawa PCW 13

Validity and Your Assessments: using a contemporary validity framework to evaluate assessments

Katharine Boursicot; (Health Professional Assessment Consultancy, Singapore); Trudie Roberts (University of Leeds, UK); Richard Fuller (Curtin University, Australia) Mabel Yap (Ministry of Health, Singapore)

Notions of validity in assessment are often confused; contemporary models of validity described by Kane (2013) provide a holistic approach to scrutinise assessments in a systematic manner.

Level: Intermediate

 

Ottawa PCW 14

The “Learning Analytics” tidal wave of process measures:  What are the implications for Health Professions Assessment?

Martin Pusic (NYU School Of Medicine, USA); Stanley Hamstra (Accreditation Council for Graduate Medical  Education, USA); Rodgrigo Cavalcanti (University of Toronto, Canada)

There will be presentation of a case description (U.S. Dept of Education Definition of Learning Analytics & Examples) followed by the Group work – consider definition of learning analytics and “what has changed”? The Gartner Framework will be presented with discussion of Predictive and Prescriptive Analytics followed by the Group work – Implications for a Health Professions School (How would/should I change my practice?).In the last there will be Summary of Current Research Directions

Level: Advanced

 

Ottawa PCW 15

Assessing the Quality of Objective Structured Clinical Examinations: An Introduction to Psychometrics

Naveed Yousuf, Rukhsana Zuberi (Aga Khan University, Pakistan)

Despite the objective, standardized and structured method of OSCEs, it is imperative to ensure that decisions based on performance in OSCEs are valid, reliable and in accordance with the basic standards of testing and assessment. These results must also inform the curriculum, the test administrators and the students of areas of strength and areas for improvement.

Level: Intermediate

 

Ottawa PCW 16

Programmatic Assessment in competency-based education and assessment: From Theory to Practice

Sylvia Heeneman, Marjan Govaerts, Suzanne Schut (Maastricht University, Netherlands)

In this workshop, the principles of PA are explained.  Context-specific factors for success as well as issues of concern will be discussed using examples of current (best) practices. In small groups, participants will discuss opportunities and barriers for implementing PA in their context.  There will be ample opportunity to examine how the model of PA can be used for implementing novel programmes of assessment tailored to the (local) context.

Level: Intermediate

 

Ottawa PCW 17

Assessment of clinical reasoning: from principles to practice

Emma Willert,  Harish  Thampy (University Of Manchester, UK); Dr Subha  Ramani (Harvard Medical School, USA)

Clinical reasoning is a key skill that refers to the process of generating diagnostic hypotheses. Assessing clinical reasoning remains challenging for many supervisors and forms the focus of this workshop.

Using clinical scenarios and scripts, brainstorming and small group exercises, we plan to discuss frameworks for assessment of clinical reasoning at multiple levels of training, from undergraduate to postgraduate.

Level: Introductory

 

Ottawa PCW 18

Effective Work-based Assessment in the CBME Era: Meeting the Needs of Both Patients and Learners

Eric Holmboe (ACGME, USA)

The effective use of work-based assessments used in the course of caring for patients and populations is essential to the successful implementation of competency-based medical education (CBME). This will be an interactive, hands-on workshop to help participants get the most out of their work-based assessments (e.g. direct observation, evaluation forms, multi-source feedback and clinical performance data). Participants will complete an action plan blueprint to help them make changes in their own assessment approaches. A toolkit of assessment resources will be provided.

Level: Intermediate

 

Ottawa PCW 19

Introduction to Progress Testing

Adrian Freeman (University Of Exeter Medical School, UK); Dr Lee Coombes, Steve Riley (University of Cardiff Medical School, UK); Warwick Bagg (University of Auckland Medical School , New Zealand); Carlos Collares (University of Maastricht Medical School, Netherlands)

The presenters are experienced in all aspects of progress testing, from the initial design and delivery through to the post test analysis and interpretation of psychometrics.  With many variations of the test itself, the workshop will discuss general principles that are common across all tests, examine best practice, and look at the challenges of introducing a progress test into a medical school programme

Level: Introductory

 

Ottawa PCW 20

Using virtual-reality simulators to assess competence in technical skills

Lars Konge, Martin Tolsgaard, Amandus Gustafsson (Copenhagen Academy for Medical Education & Simulation (CAMES), Denmark)

Virtual-reality simulators are excellent for learning technical skills. Training to a predefined proficiency level directs training, motivates the trainees, and ensures basic competency before operating on patients. However, development of clinically meaningful proficiency tests with sufficient evidence of validity can be a challenging task. The facilitators of this workshop have published more than 30 scientific studies on the development and validation of simulation-based tests. They will give valuable tips and tricks on feasibility, the five sources of validity evidence, standard setting, and on how to publish validation studies.

Level: Advanced

ICME

ICME PCW 1

Objective Structured Teaching Evaluation: A collaborative approach

Zarrin Siddiqui (University Of Western Australia, Australia); Rahila Yasmin (Riphah International University, Pakistan)

This workshop will be an opportunity for participants to; engage in a collaborative exercise for developing an OSTE blueprint through identification of the skills to be addressed, preparation of the scenario and development of the assessment tool.

Level: Intermediate

 

ICME PCW 2

How to conduct a self-study based on standards of an accreditation system

Azim Mirzazadeh (Council for Undergraduate Medical Education, Iran); Tahereh Changiz (University of Medical Sciences, Iran);
Roghayeh Gandomkar (Tehran University of Medical Sciences, Iran)

Accreditation received more attention in the last decades in 5 continents. WHO and WFME have started a joint program for promoting accreditation worldwide and based on ECFMG policy, after 2023, only those applicants who graduated from an accredited medical school could sit the licensing examinations. In this workshop, after a brief description of basic concepts about accreditation system, you have a lot of opportunity during interactive group exercise to use a simple tool for collection and analysis of data according to standards of an accreditation system.

Level: Intermediate

 

ICME PCW 3

Writing reliable Multiple Choice Questions (MCQ)s, and using item analysis for post-exam analysis

Susanna Martin, Joshua Lloyd (University Of Saskatchewan, Canada)

Participants will work in groups to identify errors in sample MCQs. Subsequent facilitated discussion will explore and generate a list of common pitfalls. Subsequently, interpretation of item analysis reports will be discussed and reports provided for the sample questions. Groups will work to revise them, followed by group discussion. Participants are encouraged to bring their own questions for group critique in the final part of the session.

Level: Introductory

 

ICME PCW 4

The World Café: Transition from “Stop Talking & Get to Work” to “Start Talking & Create Together”

Ahsan Sethi (Khyber Medical University, Pakistan)

Participants will gain insight into World Café as an educational methodology for hosting large group dialogue and will be able to develop a plan for testing World Café at your workplace and to relate World Café to your experiences and debate its value in enhancing learning.

Level: Introductory

 

ICME PCW 5

Best in Design and Adding Value: Applied Medical Education Research

Gabriela Berger (University Of Queensland, Australia)

This 3 hour workshop offers a toolbox of approaches which teaches aspiring medical educators research essentials. It will allow a fundamental understanding of applied research design and will include methodological approaches on how to select appropriate topics and write research questions to formulate research proposals that add value. Good project design is crucial; it ensures that the results gained are meaningful and can be applied in the real world for the benefit of patients.

Level: Intermediate

 

ICME PCW 6

Leadership and Emotional Intelligence Competencies for Medical Education and Healthcare Professions

Yawar Hayat Khan, Rahila Yasmeen (Riphah International University, Pakistan); Fadil Citaku (Academy of Leadership Sciences Switzerland, Switzerland)

Upon completion of this workshop, the participants should be able to: Develop a greater understanding of the range of perspectives about leadership in medical and healthcare organizations. Prepared as a leader to be more discerning about how to enact the role of leader within medical or healthcare organization. Develop the understanding of EI which serve to solve problems in healthcare organizations.

Level: Intermediate

 

ICME PCW 7

How to integrate Patient Safety and Quality into Medical Curriculum

Paul Barach (Wayne State University, USA); Matiur Rahman, Zakiuddin Ahmed (Riphah International University, Pakistan)

Patient safety currently receives almost no place in most Medical School curricula leading to lack of awareness about importance of Patient Safety practices to avoid harming the patients. This workshop will develop a plan for introducing Patient Safety in the Medical Schools especially in Developing countries where the situation of Patient Safety is far from satisfactory.

Level: Intermediate

 

ICME PCW 8

A workshop WORKSHOP: Designing, developing and conducting an effective workshop

Lamia Soghier (Children’s National Health System, USA) Beverley Robin (Rush University Children’s Hospital, USA); Jennifer Owens (George Washington University, USA)

Workshops comprise a large percentage of conference presentations, yet educators do not have formal training in their design or conduct. This workshop will introduce participants to elements of effective workshops through small group activities and facilitated discussions. Using our “Workshop workbook”, participants will design a workshop on a topic of their choice. The importance of enhanced interactivity, developing an agenda, teaching aids, methods of conducting a workshop, and use of evaluations will be highlighted.

Level: Intermediate

 

ICME PCW 9

Virtual Patient Learning (VPL): The future of Problem Based Learning (PBL)

Hossam Hamdy (Gulf Medical University UAE); Ayad Al Moslih (Qatar University, Qatar); Giulio Tavarnesi, Mr. Andrea Laus (LIFELIKE SA, Switzerland)

To introduce medical educators to a major medical education innovation using artificial intelligence and advanced cinematographic techniques which will affect the implementation and future direction of Problem Based Learning. It transform the traditional PBL format to a virtual patient the students or residents can interact with.

Level: Advanced

 

ICME PCW 10

Facilitate your students using OMP: A Time Efficient Teaching Skill

Muhammad Nadim Akbar Khan, Saadia Sultana (Riphah International University, Pakistan)

OMP is a teaching method designed to help faculty to teach efficiently when otherwise pressed for time. This workshop has been designed for both busy clinical preceptors and medical students to enhance shared responsibility for learning. A PPT presentation of five skills of this microteaching method will help participants to understand the steps of OMP. In the next phase one group of participants will be given a vignette to conduct OMP by role play. Other group will critique the performance of first group by constructive feedback.

Level: Introductory