MBChB - Year 2 - Integrated Health Systems Part 1A (Pre-clinical)

Context: Prior to COVID, assessments were invigilated and comprised  Short Answer Questions (SAQs), Multiple Choice Questions (MCQs), Extended Matching Item (EMI) papers and Objective Structured Practical Examinations (OSPI) tests. Due to varied levels of internet access and home learning environments, as well as the lack of invigilation, assessments moved to online versions of OSPI (using illustrations instead of specimens), and the MCQ and EMI papers were maintained. The SAQs were dropped due to concerns around logistical and technical challenges associated with written questions. One challenge was allowing students sufficient time to compensate for possible internet connectivity issues, but not providing enough time for students to treat the assessment as an open-book exam.

Purpose: There was a greater emphasis on continuous assessment. The SAQs previously assessed more application skills, whereas the MCQ and EMI assessments focused more on core theoretical competencies. The MCQs are now being recreated to be more integrated in terms of assessing both core knowledge and applied skill.

Process: The assessment was open for 24 hours (accounting for those students whose home conditions or connectivity was better at particular times of day), but the duration of the assessments was limited to 1 hour. Multiple short assessments (with smaller mark allocations) were used rather than 1 long assessment as this was considered less vulnerable to connectivity issues. Having the assessment open for 24 hours meant that collusion was possible, but there was no indication of that happening. The pass mark for all papers was lowered to 50%, whereas pre-COVID, some papers (assessing core competencies) had a pass mark of 65%.

Outcomes/ Lessons learned: The initial time required to develop new questions was counterbalanced by the times saved using computer-marked assessments. Having multiple smaller assessments was perceived by some students as being very intense and pressurised. Some integration and skill application was lost in doing away with SAQs (so those will be brought back), but integration has been a weakness for some time. The class average went up and there were fewer failures in ERT, but despite lowering the pass mark, the standard didn’t decrease. COVID necessitated some valuable changes to the assessment practices.

Recommendations: Based on the ERT experience, retaining the continuous and integrated assessment practices, and reintroducing opportunities to assess applied skills would be a valuable enhancement of the assessment overall practice.