Abstract
Introduction
Curricular mapping is essential in pharmacy education to evaluate student learning progression and identify curricular gaps. Faculty assessment tagging can be labor-intensive and subject to variability. Artificial Intelligence (AI) systems, such as CompetencyGenie®, can automate tagging in a scalable manner. This case report evaluates CompetencyGenie’s® reliability relative to faculty tagging of pharmacy assessment items.
Methods
Faculty-tagged pharmacotherapy questions across four required domains were compared to CompetencyGenie® recommendations. Percentage agreement and Cohen's Kappa (κ) were used to quantify the degree of agreement. Results
Sixty-two tagged questions were analyzed. Agreement was 59.7% for Bloom’s Taxonomy (κ = 0.57), 47.3% for COEPA (κ = 0.24), 71.7% for Didactic Topics (κ = 0.30), and 71.7% for Disease States (κ = 0.69).
Discussion
Despite variable results across domains, fair levels of agreement were observed for Bloom’s Taxonomy and Disease States. AI-assisted tagging may support faculty judgment, reduce workload, and streamline curricular mapping.
Rights
© 2026 The Authors
Recommended Citation
Stewart, Morgan P.; Nguyen, Kevin; Castleberry, Ashley; Wu, Li-Wei; Park, Chanhyun; Metri, Justine; and Nohria, Rahul, "Tag, You Are It: Evaluating Artificial Intelligence and Faculty Alignment in Question Categorization Across Four Competency Frameworks" (2026). The Interdisciplinary Journal for Research on Pedagogy and Learning Design, Vol. 1, Issue 1., Fall 2026. 15.
https://openprairie.sdstate.edu/ijrpld/15
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License