In an ideal world, medical school assessments would provide a clear measure of how well students have mastered required competencies, aligning with the ultimate goal of preparing capable practitioners. However, faculty often face challenges such as limited time and difficulties in evaluating competencies effectively, particularly when classifying quiz and exam questions. While competency-based education is widely adopted in medical schools, achieving accurate category tagging is essential for obtaining meaningful insights into student performance.
You can’t measure curriculum effectiveness without consistent classification. To evaluate student performance effectively, ALL faculty need to tag ALL items correctly ALL of the time. It’s a major part of unlocking accurate data analytics in medical education and demonstrating accreditation compliance.
There are many other benefits of exam item classification. But first, let’s address the elephant in the room. There’s a good reason faculty dislike classifying items. The process has been time-intensive and challenging for educators.
It doesn’t have to be this way. Data analytics platforms like Enflux allow instructors to evaluate the reliability and validity of assessments. When paired with a powerful AI tool CompetencyGenie™, which classifies and tags exam items, faculty members can eliminate manual tasks and guesswork, save time, and enhance educational outcomes.
Exam Item Classification for Medical Education
Fair and effective assessment is critical to medical education, as student performance needs to meet state board standards. Consequently, competency-based education has become the preferred teaching model for most health professions programs, as it evaluates performance based on the demonstration of required skills and knowledge.
While it sounds good in theory, many instructors need more support to ensure curriculum alignment. Moreover, proving assessment effectiveness and even fairness is difficult without a good strategy and proper data.
What does the typical process look like?
First, faculty members identify the most relevant competencies and frameworks to guide their assessments. These are then organized into categories that align with the program’s goals. Common examples of such categories include the USMLE and COMLEX exam competencies, Miller’s Pyramid of Clinical Competence, practice-based learning, and course-specific learning outcomes.
Through exam item classification, educators match or tag specific exam items with a category to measure how well students master the required competencies. This system facilitates assessment mapping, finding curriculum gaps, and curriculum alignment with program outcomes.
For example, an instructor may want to assess students’ abilities to evaluate alternative treatment options for a herniated disk. By using Enflux’s competency-based tagging, the instructor can quickly identify whether existing exam items align with this clinical decision-making skill. If no items match this intent, it signals the need to update the assessment to better evaluate students’ competency in making informed treatment decisions.
Additionally, exam item classification can help reduce guesswork in understanding student performance. Since students learn at different paces and may progress unevenly through course content, analyzing assessment data becomes crucial. For example, an instructor using Enflux might identify that 40% of the class performed poorly on specific questions within a particular category. This insight could lead to a review of whether the content was adequately taught, the questions were effectively designed, or if additional instructional time is needed for that category. By providing a clear structure for evaluation, exam item classification supports instructors in guiding students toward mastering competencies.
Integrating Exam Classification in the Curriculum Mapping Process
In medical education, curriculum mapping, category tagging, and exam item classification are interconnected processes critical for achieving accreditation and maintaining educational quality. Think of them as complementary pieces of a puzzle that together create a complete picture.
- Curriculum mapping: This process addresses the question, ‘What do we want to teach?’ It ensures alignment between the curriculum and overarching learning objectives or competencies.
- Category tagging and exam item classification: This step connects specific learning objectives or competencies to individual exam items, ensuring that assessments accurately reflect the intended goals of the curriculum.
Together, these tools provide a comprehensive framework for curriculum alignment, gap identification, and effective assessment design, ensuring consistency and meeting accreditation standards.
Some of the benefits of exam classification and tagging are:
- Organizing content: Health professions course content teaches a complex set of knowledge, skills, and behaviors. Mapping, classifying, and tagging competencies offer a good structure for the entire program. Moreover, they reveal insights into organizing course content to best support learning outcomes.
- Ensuring curriculum alignment: Accurate exam item tagging and classification help generate meaningful data, enabling educators to evaluate and enhance curriculum effectiveness. Enflux provides data visualization and dashboards for higher education, including a curriculum effectiveness dashboard that monitors student academic performance over time. These tools offer valuable insights into the impact of curriculum changes. Additionally, instructors can use the platform’s curriculum gap analysis to identify discrepancies between the intended learning outcomes outlined in the curriculum map and the actual outcomes reflected in their teaching and assessments.
The Curriculum Effectiveness and Gap Analysis dashboard
- Improving learning outcomes: Competency-based learning focuses on student-centered teaching methods, where learners demonstrate mastery of specific skills and knowledge. While true competency-based assessments often rely on observations, such as evaluating a student’s ability to perform clinical tasks or make informed decisions in patient care, tools like mapping, tagging, and classifying exam items can still play a complementary role. By aligning quizzes and exams with key competencies, educators can identify areas where students need further support. This data-driven approach allows instructors to adjust their teaching strategies, ensure better alignment with learning objectives, and contribute to overall student success.
If you’re thinking: well, this is a lot! Indeed, who has the time and easy way to do this?
Data analytics in medical education effectively addresses this challenge. Enflux, a comprehensive data analytic and decision support system for higher education, exemplifies how meaningful and clean data can empower educators to make timely, informed decisions, support student success, and drive continuous improvement. Our approach helps level the playing field for students. Additionally, the CompetencyGenie™ Chrome extension for ExamSoft users does the job in a few seconds within the platform.
This allows instructors to focus on curriculum development and instructional strategies, as the AI-powered solution simplifies the classification process. Most importantly, it enables the provision of actionable feedback and the identification of growth areas, offering students critical support when they need it most.
Real-Life Examples: Achieving 100% Faculty Compliance with Enflux
Many higher education institutions have gone through similar challenges before adopting comprehensive data analytics tools or integrating AI into their processes.
Florida A&M’s Pharmacy School Journey to Excellent NAPLEX Pass Rates
Just a few years ago, Florida A&M’s College of Pharmacy had a major problem. A 26% pass rate decline on the NAPLEX exam raised major alarm bells.
After working with Enflux, the data revealed that 73% of the faculty did not tag all their exam questions. Those who did only managed a 3% compliance for NAPLEX! Their improvement plans lead to reaching 100% tagging coverage for all relevant categories.
The final result? A 87-92% NAPLEX pass rate, which they easily maintain now that they have a proper categorization system.
Franklin Pierce University Streamlines Categorizing And Tagging Exam Items
Dr. Stephani Cuddie, Director of Online Administration & Training at Franklin Pierce University, was on a mission to support her faculty. They turned to Enflux to address an urgent need: an important accreditation visit was approaching in just one month.
To help her faculty, Dr. Cuddie utilized CompetencyGenie™ by Enflux for exam item tagging and categorization. With 90% of the items tagged in a short period, she was able to streamline the evaluation process, ensuring a comprehensive and precise assessment of the curriculum, while saving time and minimizing errors.
An example from a Physician Assistant program identifying competencies for the PANCE board exam
Moreover, proper classification unveiled redundancies as multiple courses taught the same content and assessed the same learning outcomes. This was the perfect opportunity to guide faculty in choosing the right competencies for each subject.
Effective Assessment Strategies for Health Professions Education
Educators and administrators, building confidence in your courses and assessments hinges on robust data insights.
A successful competency-based program relies on effective strategies tailored to your specific needs. Among these, exam item classification, curriculum mapping, and category tagging are essential tools that can enhance effectiveness of your medical education program.
Furthermore, automated exam item classification simplifies the entire assessment process for educators. After tagging questions in ExamSoft using the CompetencyGenie™ Chrome extension, the data integrates seamlessly with Enflux’s analytics platform. This integration provides users a comprehensive view of their results, enabling them to evaluate curriculum effectiveness, conduct gap analyses, and monitor student outcomes across required categories. While CompetencyGenie™ streamlines the initial tagging process, Enflux serves as the cornerstone for transforming that data into actionable insights, offering a holistic perspective and allowing educators to make informed decisions to enhance curriculum alignment.
In short, tools like Enflux empower educators to analyze data effectively, refine assessments, and ensure alignment with accreditation standards—all without the need for spreadsheets! Discover how these innovative tools and strategies can help you drive student success and make a lasting impact on your educational programs.