IV. Limitations of Constructed Response Items
C. Potential for Ambiguity
Ambiguity in constructed response items occurs when questions are unclear, open-ended, or can be interpreted in different ways. This can confuse students and make it difficult for them to understand what is being asked and how to provide an appropriate response. Ambiguity can have negative effects on learning, such as reducing students' confidence, leading to inconsistent feedback, and potentially inaccurately assessing their knowledge.
BASIC Information
Ambiguity in constructed response items arises when questions are unclear, open-ended, or can be interpreted in various ways. This can lead to confusion for students, making it challenging for them to discern what's being asked and how to craft an appropriate response.
In the context of medical education for PA students, ambiguity can be especially problematic. If students misinterpret what's being asked, they might provide an accurate answer to a different question, leading to potential scoring issues and misjudgments about the student's knowledge and competence.
INTERMEDIATE Information
Several factors can introduce ambiguity into constructed response items:
Vague wording: Using non-specific terms or phrases.
Lack of context: Not providing a clear scenario or background.
Overly broad questions: Asking about vast topics without narrowing the focus.
To reduce ambiguity, questions should be:
Clear and concise.
Context-specific, especially in clinical scenarios.
Direct in what they're asking, with a clear expectation of what constitutes a full answer.
In a module about respiratory conditions, a PA instructor wants to assess knowledge on causes of shortness of breath. Instead of an ambiguous question like "Describe factors causing shortness of breath," the instructor could ask, "A 65-year-old male patient, a smoker with a history of hypertension, presents with sudden shortness of breath. Discuss the three most likely causes in this scenario." This provides context and sets clear expectations for the student.
ADVANCED Information
Ambiguity in questions not only affects the assessment process but can also impact learning:
Reduced Confidence: Students might lose confidence if they frequently encounter ambiguous questions, unsure if they're misinterpreting them or lacking knowledge.
Feedback Discrepancies: Ambiguous questions can lead to varied student answers, making feedback inconsistent and potentially confusing.
Inaccurate Assessment of Knowledge: Scorers might misjudge a student's knowledge based on their response to an ambiguous question, potentially overlooking gaps or misconceptions in understanding.
While reducing ambiguity is generally the goal, there's a concept known as "constructive ambiguity." This is where a slight degree of ambiguity is introduced deliberately to mimic real-world situations where not everything is clear-cut. It pushes students to think critically and make judgments based on the information available. However, it's essential to use this technique judiciously and ensure that the purpose is clear to both students and scorers.
A PA student in a clinical rotation is presented with a case: "A 30-year-old female presents with intermittent abdominal pain over the last six months. Recent labs show iron-deficiency anemia. Discuss the potential diagnoses."
While conditions like Crohn's disease or celiac disease might be top considerations, the scenario is left slightly ambiguous to mimic the uncertainties of real-world diagnoses. The student is expected to navigate this ambiguity, consider various options, and present a reasoned differential diagnosis.