By John Merrifield, President, Institute for Objective Policy Assessment – https://policytalk.org
As Nathan Gray and I document in our new book – Unproductive School Choice Debates: All Sides Assert Much that is Wrong, Misleading, and Irrelevant – there is a long history of unprincipled attention to alleged outcomes of the US experience with minor expansions of school choice eligibility. An especially egregious recent example is “Apples to outcomes? Revisiting the Achievement v. Attainment Differences in School Voucher Studies,” from the Brookings Institution Brown Center. An equally egregious example is chapter seven of Diane Ravitch’s (2010) The Death and Life of the Great American School System. Many of the voucher studies alluded to (not described!?) there and in the Brookings article are wrong. Some analysts assess numbers masquerading as data. And it is common for statistical manipulations to control away central issues.
But the much bigger problem is that the vast majority of school choice studies, including most of the most rigorous, are misleading. They have low relevance to a “Nation at Risk.” It’s a nation in need of gradual, transformational school system change, not minor re-shuffling of enrollment within static, terribly low-performing school systems dominated by traditional public schools persisting in futile attempts to make one size fit all, with a fringe of mostly struggling, mostly sectarian private schools.
We can learn some important things from the US experience with minor expansions of school choice eligibility, but NOT by lumping them together and then making broad-brush generalizations far beyond the undisclosed, very restrictive conditions that were actually present. Ravitch’s chapter 7 does not specify the key elements of any of the many choice programs asserted as compelling evidence that choice has been a failed strategy and will continue to be; a reckless, unprofessional practice repeated by the Brookings article.
To use the effects of school choice expansions to make good policy choices, we have to connect each program’s specific conditions to its outcomes. The most important of those conditions is consistently the tiny scope that causes the findings to tell us what happens when we move children within a static system widely seen as dysfunctional. We need to change that system, not move children within it.
In the limited space available, let’s look at the much-ballyhooed program – one of two – that appeared to cause participants to learn less than if they had remained in their assigned failed (yes!) school. The Brookings article did not describe the program’s scope, or eligibility criterion. Its source was a NY Times article, not the peer-reviewed report that actually connects the program’s narrow scope to the measured early test score declines, that rebounded (not reported) back to the usual finding of small effects from a small program. Indeed, a key aspect of the Louisiana program that appeared to lead to learning loss is that eligibility was limited to victims of assigned schools formally designated as failed. So, already struggling children thrust into a new environment suffered, at least temporarily, from their dislocation, perhaps because their new curriculum was overwhelming, or just new to them. To enroll voucher users, private schools had to accept a very small sum as full payment, which caused the better private schools to be unavailable to the voucher users and precluded the formation of new schools to serve voucher users. The Brookings article did not report any of those reasons why we could expect little benefit – maybe even a temporary backslide – from a very narrowly targeted program. I find it very hard to accept that such negligent reporting could just be the result of sloppy scholarship. It is most certainly hurting children, and definitely jeopardizing a nation already at risk because of terribly low-performing school systems.