When considering effective assessment practices, what processes come to mind first? Engaged committee members, comprehensive assessment plans with timelines, and closing the assessment loop are likely at the forefront.
However, one of the most impactful elements of the assessment process is undoubtedly informed, data-driven decision-making. It serves as a cornerstone of our endeavors. Assessors gather data, analyze it, make informed decisions, implement changes, and then reassess. This approach isn’t limited to formal assessments; it extends from deciding what to prepare for your family’s dinner to complex tasks like revising a doctoral program curriculum.
The Use of Category Tagging in Assessment
In health professions education, curricular innovations are often evaluated using category tagging at the item level in individual course assessments, which then contribute to a comprehensive curriculum map. Let’s delve into each of these levels more closely.
Category tagging. Programs determine the categories most beneficial to track within their educational structure. Effective category tagging often involves collaboration with stakeholders, including faculty, students, and administrators, to develop a set of relevant categories. Common categories in health professions education encompass:
- Program learning outcomes
- Course learning outcomes
- Lecture/lab learning outcomes
- Didactic topics
- Disease states
- Licensing exam blueprint
- Level of learning (for example, Bloom’s Taxonomy)
Item classification. Assessment items on quizzes and/or exams are tagged with the specified categories, establishing an item classification system. This system is instrumental in organizing item banks, selecting appropriate items for assessments (exam blueprinting), monitoring student performance, and evaluating teaching effectiveness.
Curriculum mapping. Curriculum mapping holds immense significance in the accreditation process for health professions education. Programs are required to chart out what content is being taught and where to ensure coverage of critical topics. Multiple curriculum maps are often utilized to visually represent various aspects of the curriculum, including topics (with depth of learning on each topic), disease states, learning outcomes, instructional methods, and assessment methods.
A Comprehensive Look at Benefits of Category Tagging in Higher Education
Consistent, accurate, and comprehensive application of category tagging across a program elevates its status to a pivotal tool, significantly enhancing educational outcomes in health professions education. A critical aspect of category tagging lies in its role of tracking and assessing core professional competencies throughout both pre-clinical and clinical education. Category tagging doesn’t merely enhance the educational landscape for health professions programs; it serves as a cornerstone for competency-based education, guaranteeing that students are sufficiently prepared for the demands and expectations of their roles in healthcare.
Importance of category tagging for tracking core professional competencies. Competencies in health professions education encompass a spectrum of skills, knowledge, behaviors, and attitudes essential for effective practice in the healthcare domain. Categorizing assessment items based on cognitive levels and profession-specific competencies offers a structured and insightful approach to systematically track and evaluate students’ development and mastery of vital skills and knowledge required for their respective professions.
Benefits to the curriculum. By aligning the delivery and methodology of content with student performance in each category, programs can utilize data to make well-informed decisions about their curriculum. By tagging assessment items related to disease states, a medical program can ensure comprehensive coverage of critical conditions and tailor teaching approaches to improve understanding. This approach enables the identification and filling of gaps in content, addressing low student performance, and facilitating the monitoring and celebration of curricular innovations.
Benefits to instructors. Instructors can proactively and retroactively utilize categorized data to enhance teaching and learning. Item mapping aids in crafting targeted exam blueprints that precisely align with intended learning objectives. Consistent low performance in specific program learning outcomes can prompt instructors to modify their instructional strategies, thereby improving understanding and retention. Moreover, feedback from student performance in various competency-based categories empowers faculty to make data-driven decisions, refining teaching strategies to best support student learning.
Benefits to students. When shared with students, individualized feedback on their performance within the defined categories enhances the learning experience and enables students to understand their strengths and weaknesses in relation to critical competencies. This feedback can be provided after individual assessments (quizzes and/or exams), at the end of a specific period (semester or year), or cumulatively. For example, feedback on their performance related to specific disease states guides students in focusing their study efforts and gaining a deeper understanding of critical medical conditions. Ultimately, this process empowers students to develop their metacognitive skills, allowing them to strategize their study approach for future assessments and, in the long run, their professional practice.
I Get It, But…
The benefits of curriculum mapping and category tagging of assessment items is clear. But it can be the most time-consuming part of the assessment process. One of the categories that is often the hardest to tag is the level of learning (e.g. Bloom’s Taxonomy). This is where we can get some help from AI to work more efficiently. It is time that we considered automated exam item classification to simplify and automate this manual process.
Can Artificial Intelligence (AI) Support Category Tagging in Health Professions Education?
CompetencyGenie™, developed by Enflux, is an AI-powered tool that provides seamless categorizes exam item classification according to cognitive levels and competencies. By automating the manual process, it streamlines assessment classification, allowing educators to measure curriculum effectiveness and pinpoint areas for improvement. This tool is really easy to use and can handle single questions or a batch of questions at the same time. Moreover, users have the flexibility to choose between the original or revised Bloom’s Taxonomy. Additionally, there’s an option to request access to actively developed, discipline-specific competency classifications.
Is this tool flawless? No. To be frank, educators aren’t perfect at category tagging either. AI-assisted tools like CompetencyGenie can efficiently initiate the tagging process, which educators can then review and validate.
Explore the tool to determine its time-saving potential. Assess its accuracy in alignment with your content. Evaluate if it enhances your tagging process. Additionally, consider engaging in educational scholarship to collectively enhance our understanding of how these powerful tools can benefit our curricula, faculty, and students.
