IV. Limitations of Constructed Response Items
B. Time Intensity for both Students and Scorers
The time intensity of constructed response items, which require students to craft thoughtful answers and educators to evaluate them, can impact the design of assessments. Exams heavily relying on these items may become too long, leading to student fatigue and reduced performance. To address this challenge, institutions are exploring technological solutions such as digital submission and partial automated scoring to make the process more efficient.
BASIC Information
Time intensity in the context of constructed response items refers to the substantial amount of time it takes for students to craft answers and for educators or scorers to evaluate them. Unlike objective questions that can be quickly answered and auto-graded, constructed response items demand thoughtfulness in crafting answers and a comprehensive evaluation process.
PA (PA) programs are rigorous and require students to understand and apply vast amounts of medical knowledge. When students are asked to answer constructed response items, they must recall information, formulate a comprehensive response, and articulate it well. Meanwhile, instructors or scorers must dedicate significant time to read, understand, and fairly evaluate each student's response.
INTERMEDIATE Information
The time intensity of constructed response items can influence the overall design of assessments. If an exam heavily relies on these items, there's a risk it might become too long, leading to student fatigue and reduced performance. On the other hand, instructors might feel pressured to reduce the number of constructed response items, potentially compromising the depth and breadth of assessment.
While it's tempting to include numerous constructed response items to cover a wide range of topics, quality should be prioritized over quantity. A few well-crafted items can provide deeper insights into a student's understanding than numerous shallow ones.
In a cardiology module, a PA instructor wants to test knowledge on various heart conditions. Instead of asking detailed constructed response questions about every condition, the instructor might focus on a few major diseases, asking students to detail their pathophysiology, treatment options, and patient care considerations. This approach balances the depth of assessment with the practicality of grading time.
ADVANCED Information
To combat the time intensity challenge, many institutions are exploring technological solutions. Software tools that allow for digital submission, streamlined feedback, and even partial automated scoring can aid in reducing the time burden on scorers. While full automation isn't viable given the nuanced nature of constructed response items, technology can certainly assist in making the process more efficient.
READ
Student strategies when taking open-ended test questionsA significant consequence of the time-intensive nature of grading constructed response items is the potential delay in feedback. In medical education, timely feedback is crucial for reinforcing correct understandings and rectifying misconceptions. Delays can impede learning progress and might lead to the entrenchment of wrong ideas.
Holistic Assessment and Portfolios:
An advanced approach to tackle the time intensity issue is to incorporate constructed response items within a broader holistic assessment strategy. For instance, PA students might maintain portfolios where they regularly input reflections, case analyses, and treatment plans. Instead of grading each entry individually, instructors can assess them in chunks or at key milestones, providing comprehensive feedback. This approach values depth and ongoing assessment while managing time demands.
A PA student is on a clinical rotation in rheumatology. Over a month, they encounter various cases of autoimmune disorders. Instead of daily detailed reflections, the student submits a weekly case analysis, integrating all the learnings from that week. The instructor then provides feedback on these weekly analyses, ensuring that feedback is timely yet comprehensive, without being overwhelmed by the daily grading of detailed entries.