Over the past several months, we have seen three new reports released by Stanford’s Center for Research on Education Outcomes (CREDO) on charter school performance in Idaho, Ohio, and, most recently, Pennsylvania. Much like their predecessors, these reports seek to utilize a so-called “virtual twin” methodology to examine the performance of charter school students in both brick-and-mortar and online charter schools, in comparison to their traditional public school counterparts. These new reports find varying levels of charter school performance within each state, but all three make the same summary statement about the perceived failure of online charter schools when it comes to math and reading growth.

Since the very first CREDO virtual twin report came out four years ago, we have responded consistently to highlight both the positives and potential downsides of the virtual twin methodology. We appreciate that CREDO researchers have attempted to control for the many factors that impact student performance, highlighting both student characteristics and prior test performance as outlined in the latest Pennsylvania report.

However, the virtual twin model does not include several additional important factors that disproportionally affect students in online charter schools. A match for these missing factors is necessary to ensure virtual twins are being accurately compared, and despite our attempts over the past four years to encourage the inclusion of these factors, the methodology has yet to be revised to include them. These factors are:

• Length of Enrollment
• Enrollment Date or Time of Year
• Reason for School Switch

Student Mobility

These 3 factors are associated with one thing, the movement of students from one school to another, also called student mobility. While I will not belabor each of these factors, I do want to highlight some key pieces of evidence that point to the impact and importance of mobility on school success measures. There are decades of research with numerous studies finding that student mobility can negatively affect everything from standardized test performance to the likelihood of an on-time graduation. And there are two studies that are particularly interesting in light of these CREDO report findings:

• This study in California found that not only are there impacts on achievement at the student level but that even non-mobile students have lower average mathematics test scores if they attend a high school with a high mobility rate.
• This meta-analysis of twenty-six studies on mobility and achievement determined that 25 out of the 26 studies found negative effects of mobility on reading and math assessment results, with mobile students being 3-4 months behind in achievement.

The impact of mobility is still relevant today, in both brick-and-mortar and online schools. In Texas, our research found that students in highly-mobile schools, whether charter schools or traditional public schools, had significantly lower state assessment proficiency rates than their peers in non-highly-mobile schools in all subjects. These gaps ranged from around 10-20 percentage points as evidenced on this chart.

Source: TEA STAAR 2017-2018 Aggregate Results with Analysis completed by K12 Academic Policy Team

The researchers at CREDO do take a step toward acknowledging the impact of mobility by taking a look at the performance of students in charter schools in Pennsylvania compared to their traditional school counterparts and find that the majority of the negative impact on student growth happens in the first year that a student enrolls at the charter school, which lines up with the mobility research outlined above.  Many online charter schools have mobility rates that approach 50% and thus the impact of those first-year students could explain much of the perceived negative performance of the online sector using this virtual twin methodology.

However, it is worth noting that despite taking this look at persistence for the overall charter sector, this same analysis was not completed for online charter schools in the report.

Source: CREDO Pennsylvania State Report Page 32

So you may be thinking at this point, “So what?  Mobility matters but now what?”  Here are three reasons why it matters:

• Decades of research has proven that, by the very act of choosing to leave a school, the student becomes fundamentally different than their so-called “virtual twins” who remain in the traditional public school. In fact, a peer-reviewed study by Pearson found that after adjusting for student demographic factors,  district-mean student mobility, and school-mean socio-economic status there was no statistical difference in the rate of students reaching proficiency in math and reading between student cohorts in Connections Academy online schools and cohorts in brick-and-mortar schools. It may be that much of the approximate 100 fewer days of math and reading instruction CREDO found for Pennsylvania online charters could be attributed not to the school itself but to the impact of mobility, which would line up with the findings of the meta-analysis showing that mobile students were 3-4 months behind in achievement.

• States are beginning to acknowledge the impact of mobility on student and school results by building mobility factors into accountability systems and policies and other states should give this some thought. There are several ways that states are currently handling mobility and I want to specifically point out the Arizona approach that weights student test results by length of enrollment, meaning that schools are held increasingly accountable for the students the longer they are enrolled.

• These CREDO reports all close with similar recommendations regarding online charter schools; “it is the poor performance of online charter schools that drags down the overall charter impact on student academic growth and additional oversight is needed in order to assure adequate performance.” The reality is that this methodology does not account for the distinct qualities of these online schools or the students they serve. So on one side we have a flawed “virtual twin” methodology finding that “any potential benefits of online schooling such as mobility and flexibility in curriculum are drowned out by the negative impacts on academic growth” and on the other we have thousands of stories of students who have chosen this model and found success, often after thinking success was completely beyond their reach. It’s past time for the voices of these families to “drown out” these flawed studies and for researchers to look a little deeper into online charter schools.

For quotes or more information, please contact K12 media at press@k12.com