VI. Scoring Constructed Response Items
A. Development of Rubrics
A rubric is a scoring tool that outlines criteria and levels of quality for a task. It is used to assess student responses in a standardized manner. Rubrics should be regularly updated to reflect advancements in knowledge and best practices, and can also be used for peer and self-assessment to promote deeper understanding.
BASIC Information
A rubric is a scoring tool that lists criteria for a task, along with descriptions of levels of quality for each criterion. It provides a standardized means of assessing student responses.
In the context of PA (PA) training, rubrics ensure that evaluations of students' clinical knowledge, diagnostic abilities, and other key skills are consistent, transparent, and fair.
INTERMEDIATE Information
Criteria: These are the elements or features that are expected in the response.
Levels of Quality: Typically, a rubric will outline several levels of performance for each criterion.
Descriptors: These provide a detailed explanation of what is expected at each level of quality.
Holistic Rubrics: Score the overall performance on an item without breaking it down criterion by criterion.
Analytic Rubrics: Assess performance based on several criteria, providing separate scores for each.
In PA training, where clinical scenarios often require multi-faceted responses, analytic rubrics can be particularly useful. They ensure that all aspects of a student's answer, from diagnosis to treatment planning, are thoroughly evaluated.
Task: Propose a treatment plan for a patient with Type 2 Diabetes Mellitus.
Analytic Rubric:
Diagnosis (5 Points Max):
5 Points: Accurately identifies and justifies the diagnosis, referencing patient history, and lab results.
3 Points: Identifies the diagnosis but lacks full justification.
1 Point: Inaccurate or incomplete diagnosis.
Treatment Strategy (5 Points Max):
5 Points: Proposes a comprehensive strategy, including lifestyle changes, medications, and follow-up plans.
3 Points: Mentions some appropriate treatments but lacks a comprehensive approach.
1 Point: Proposes inappropriate or insufficient treatments.
ADVANCED Information
Rubrics, especially in the medical field, must evolve with advances in medical knowledge and best practices. Regularly revisiting and refining rubrics ensures their continued relevance.
Incorporating Peer and Self-assessment:
Advanced educational strategies can include peer and self-assessment using rubrics. This encourages students to critically evaluate their own knowledge and the knowledge of their peers, fostering deeper understanding.
Balancing Complexity and Usability:
While rubrics should be comprehensive, they must also be user-friendly. Overly complex rubrics can be difficult for educators to apply consistently and for students to understand.
Feedback Mechanism:
A well-designed rubric not only assesses but also educates. By understanding where they lost points, students can identify areas for improvement. This feedback mechanism is vital in medical education, where continuous learning is paramount.
Task: Conduct a mock patient counseling session on the risks and benefits of anticoagulant therapy.
Analytic Rubric:
Communication Skills (10 Points Max):
10 Points: Exhibits clear, empathetic, and patient-centered communication throughout.
7 Points: Generally clear communication but lacks consistent empathy or patient-centricity.
4 Points: Some clear explanations, but significant lapses in empathy or clarity.
1 Point: Inadequate communication skills exhibited.
Knowledge of Anticoagulant Therapy (10 Points Max):
10 Points: Accurately and comprehensively explains risks, benefits, and alternatives.
7 Points: Accurate explanations but lacks depth or misses key details.
4 Points: Some accurate information provided, but multiple inaccuracies or omissions.
1 Point: Lacks accurate understanding of anticoagulant therapy.