IV. Limitations of Selected Response Items
C. Limited in Measuring Complex Skills
Selected response items, such as multiple-choice questions, are useful for assessing basic knowledge but may not effectively measure complex cognitive or practical skills. In fields like medicine, which require a range of skills from foundational knowledge to critical thinking and hands-on abilities, selected response items may only assess the foundational layer. To create a more comprehensive evaluation, educators should use a combination of assessment tools and methods.
BASIC Information
While selected response items, such as multiple-choice questions, are excellent tools for assessing a student's knowledge on specific facts or basic concepts, they can sometimes fall short when trying to measure complex cognitive or practical skills.
Surface Over Depth: These questions can often prioritize recall over analysis, synthesis, or evaluation.
Isolation from Real-world Scenarios: The structure of such questions may not mimic the complexities of real-life situations where a combination of skills is necessary.
Inability to Measure Practical Skills: They can assess what a student knows but not always how they would act or perform.
INTERMEDIATE Information
Medicine, as a field, involves a multifaceted set of skills that range from foundational knowledge to critical thinking, clinical reasoning, communication, and practical hands-on skills. Selected response items might be adept at probing the foundational layer but may not effectively touch upon the advanced layers.
READ
Does developing multiple-choice Questions Improve Medical Students’ Learning? A Systematic ReviewContextual Judgments: Real-world medical scenarios often require the PA to make judgments based on various contextual cues. A multiple-choice format may not capture these nuances.
Interdisciplinary Integration: Clinical cases often require the integration of knowledge from multiple disciplines, a skill challenging to assess with simple selected response items.
Hands-on Proficiency: Procedures, from drawing blood to surgical interventions, need practical assessments rather than theoretical ones.
A question might ask, "Which of the following is the first-line treatment for Condition X?" This tests knowledge but doesn't assess the student's ability to consider patient history, potential allergies, other medications, or the communication skills needed to discuss the treatment with the patient.
ADVANCED Information
Understanding the limitation of selected response items in assessing complex skills is the first step. To create a comprehensive evaluation, educators need to employ a blend of assessment tools and methods.
READ
Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paperClinical Case Simulations: Virtual or real-life simulations of clinical cases can allow students to showcase their diagnostic reasoning, decision-making, and even interpersonal skills.
Objective Structured Clinical Examinations (OSCEs): These are structured practical tests where students rotate through stations, each assessing different competencies, from communication to practical skills.
Portfolio Assessments: Over time, students can compile a portfolio of their clinical encounters, reflections, and feedback, offering a comprehensive view of their skills and growth.
Peer and Self-Assessments: Encouraging students to evaluate themselves and their peers can foster self-awareness and critical evaluation skills.
Rather than just asking about the steps in a particular procedure, like a lumbar puncture, an OSCE station might have a student demonstrate the procedure on a mannequin. Simultaneously, another station might have the student explain the procedure to a "patient" (played by an actor) to assess both their knowledge and communication skills. This multifaceted approach offers a more accurate and holistic evaluation of the student's abilities in real-world scenarios.