Enflux provides a decision support platform for higher education, aggregating data in real-time and providing insightful, actionable reports to improve overall performance and student success. In November of 2019, Florida A&M University’s College of Pharmacy, an ExamSoft customer since 2015, began using Enflux. In the four months that followed, the school moved faculty compliance for tagging exam outcomes from 23 percent to 100 in all tracked categories. The school also attributes exam pass rates, which rose from 75 percent, to 82.70 percent, in large part to curriculum improvements driven by real-time data delivered by Enflux’s dashboards. Florida A&M’s Executive Director of Information Technology is the school’s Enflux liaison.
In 2016, Florida A&M’s pharmacy school noted a 26% decline in student passing rates on the NAPLEX exam, drawing administration and faculty attention. To reverse the trend and improve accreditation compliance, the school set out to strengthen its curriculum. In the process, it identified key areas requiring immediate attention, including a need to: automate data extraction; analyze and organize program performance, down to the student level; prioritize areas of opportunity to increase efficiencies; and generate action plans easily shareable among faculty members.
To improve curriculum, the school also needed to find a way to ensure that every professor was tagging every exam question to CAPE, ACPE, Blooms, and NAPLEX categories. And for accurate tagging analysis, the IT Executive Director needed to be able to exclude from reports the tags of professors who’d created exams in 2015 through 2017, but who were no longer employed by his school, since only the faculty individual member who originally created the exam questions could identify the outcomes those questions sought to assess.
“We needed to make decisions and take action early in the semester instead of waiting until finals when students might possibly be failing, and then try to figure out what, if anything, we could do,” A&M’s Executive Director of IT said. “But, as with most schools, our faculty were already extremely busy and didn’t always have time to run ExamSoft reports. Even when they did, those reports didn’t compile results in a centralized place.”
In October of 2019, Florida A&M’s team participated in an Enflux capabilities demonstration. “I liked the product immediately because it can display on dashboards all the information we need to track,” the executive director said, referring to Enflux’s native Categories Per Item, Assessment Effectiveness, Curriculum Effectiveness, Student Performance, MPJE, and PCOA dashboards, which enable faculty to begin immediately making decisions based on real-time information. “As soon as I saw it, I knew we had to have it.”
Upon contract execution, A&M’s Executive Director of IT met with Enflux’s Customer Success Team to explain their unique needs and environment. “A week later, I was logging in and looking at my dashboards,” he said, “Enflux’s development team had already written all the code. So everything was matched to our ExamSoft data. Once that data migrated over, the dashboards were automatically populated.” Next, Enflux trained the pharmacy school’s entire faculty, on-site, in November. “From signing the contract to having our faculty trained and ready to go, took less than a month,” the executive director said.
Using Enflux’s Categories Per Item dashboard, A&M’s team was able to see, for the first time, that 73 percent of the school’s faculty had not tagged all of their exam questions. Enflux dashboards further revealed compliance in each of four tagging categories: 74 percent for CAPE, 82 percent for ACPE, 77 percent for Blooms, and 3 percent for NAPLEX. As a result, the dean identified tagging as a top priority for faculty and issued a February 28, 2020 deadline for 100 percent compliance. To track progress, in January, A&M began using Enflux to generate weekly ActionPlans® showing faculty headway.
Armed with these new insights, A&M’s team also probed why NAPLEX tagging was dramatically lower than the other three categories, and learned that most faculty believed NAPLEX tags didn’t apply to their course material. “That’s not true, though it was a widely held misperception,” A&M’s IT Executive Director said. As a result, he enlisted a faculty member, who is also a NAPLEX test writer, to train faculty on properly using NAPLEX tags.
On February 10, 2020, the Executive Director of IT generated a tagging snapshot that showed an interesting trend: NAPLEX tagging had, in fact, skyrocketed, while every other category declined. “Enflux dashboards enabled us to see this pattern,” he said. “So we asked faculty to go back and tag every exam question, beginning in 2015, while keeping up with their current tags.” Professors initially balked at the new directive, questioning the need to go back to 2015. In response, the executive director underscored the importance of acquiring longitudinal data. “If you have questions that have appeared on exams over time, you can see if students are performing better on that question based on the changes in your teaching, your remediation, or the review practices you do for your exams,” he explained.
For added compliance incentive, the dean concurrently announced that faculty whose questions weren’t tagged by February 28 would face corrective action. “Typically you don’t have to enact these types of measures with faculty,” A&M’s IT Executive Director said. “But Enflux is only as good as the data we provide. Good data produces good insights and the ability to make good decisions. Having categories tagged correctly was critically important to achieving our goal of improving student’s NAPLEX first-time pass rate.”
In the final weeks before the dean’s deadline, Enflux began updating dashboards three times a week, rather than once. “This enabled me to frequently update the faculty and send custom screenshots to individual professors,” said the executive director of IT, who also used Enflux to create individual ActionPlans, with due dates, for each professor. In addition, each week, the executive director created a training video on how to do bulk tagging in ExamSoft, making the task much faster and easier, and further boosting compliance.
As an extra incentive, the Dean also began recognizing faculty who achieved 100 percent tagging compliance, displaying their photos, along with congratulatory messages, on digital monitors across campus, and by issuing certificates of recognition at staff meetings.
Florida A&M faculty also implemented other Enflux dashboards, including Student Performance. They referenced an actual student, whose name was changed, to illustrate how this dashboard supported student success:
“John Doe scored a 66.5 on his first Individualized Health/Patient Care exam,” they said. “Our metric is 70 percent. John’s score appeared on the “Insight” field of his dashboard, triggering a meeting with his professor. Enflux ActionPlan’s “Data Driven Decision” field enabled the professor to document that John’s low score was due to poor performance in class. The field also noted that John was in an at-risk population. John’s “Data-Driven Objective” was to receive at least 70 percent in this class. His “Measurable Target” was 70 percent or better on his next exam. To provide additional support, a Student Service Advisor was automatically assigned to meet with John by a certain deadline, to determine what factors led to his low grade. During the meeting, John said he needed tutoring and, as a result, was matched with a tutor from the school’s volunteer pool. John’s Student Advisor left comments in John’s Enflux ActionPlan regarding her findings from their meeting, tagging ACPE standards for “Student Achievement and Readiness,” noting tutoring as a next step, and attaching related documents to John’s Enflux record. As a result, John scored an 84 on his next exam. The Student Advisor noted the improved score and John’s ActionPlan was closed.”
Student Performance Dashboards are also helpful during faculty office-hour visits with students, enabling professors to tell students what scores they need to make on future exams in order to pass the course. Actionable feedback can also prompt students to proactively consider steps they can take prior to upcoming exams (e.g., mentoring, requesting separate study sessions with the professor on a specific topic, after-hours videoconference meetings), to improve their success.
As Florida A&M implemented Enflux, the Executive Director of IT identified several needs that Enflux’s native platform didn’t address. “In each case, I met with their Customer Success and Development teams and, a week or two later, they’d provide a solution to meet our needs,” he said. “When a customer has an issue, they listen and respond very quickly.”
The executive director recommends that schools designate one individual to serve as the school’s primary Enflux liaison, assuming responsibility for the pace and sequence of implementation. “At Florida A&M, I am the Enflux Champion—it’s in my job description,” he said. “I told the Dean I’d be glad to take on this new role, as long as I was also given faculty access and support. The Dean was quick to delegate authority along with responsibility, and that’s been another key aspect to implementation success.”
After a month of Enflux implementation, Florida A&M’s pharmacy school achieved 100% tagging compliance in all four targeted categories.
The school now uses Enflux for documentation and ACPE reports. “We used to have to pull self-study information from a number of disparate sources,” A&M’s Executive Director of IT said. “Whereas now we use Enflux to pull all our information from a centralized source.”
Based on Enflux dashboards, the Florida A&M also produces a report showing ACPE the school’s plans for compliance and what it is doing to improve NAPLEX scores.
By using Enflux as an early-alert system for intervention and remediation, A&M is able to intervene after the very first exam, making data-driven decisions and feeling confident about those decisions.
Florida A&M can now document that its curriculum produces an exam pass rate of 87% to 92% “in large part due to Enflux and the ability they’ve given us to look at data ahead of time instead of waiting until the end,” their team said.
The number-one reason Florida A&M bought Enflux is that the metric on which the university is judged is NAPLEX first-time passing rate.