School performance reviews mislead : Stltoday
The annual review of school district performance by the Missouri Department of Elementary and Secondary Education claims to show how well each school district in the state performs. It doesn’t. All it really shows is the relative wealth of the population in each school district.
Compare the performance scores with rates of participation in the federal school lunch program. The correlation between poverty and low performance scores is striking. I made such a comparison for 52 local education agencies in metropolitan St. Louis that operate high schools. I excluded education agencies that did not offer high school from the comparison, because the challenges of high school and elementary school are so very different as to make any comparison between the two meaningless. I used participation in the free and reduced price lunch program as the measure of poverty, even though it does not register differences in the intensity of poverty, for example, or distinguish between someone new to the program and someone who has always lived in poverty. The degree to which different levels of participation in the subsidized lunch program tracked accreditation levels, however, was stunning.
DESE ranks districts into four levels of accreditation.
In only four of the 20 districts whose scores would earn them the “accredited with distinction” label were more than one-third of the students eligible for free or reduced price lunch, and none of those enrolled more than 3,000 students.
In none of the 27 districts that would earn accreditation without distinction were fewer than one-third of the students eligible for free or reduced price lunch (and in only two were fewer than 40 percent of the students eligible for subsidized lunches). In only one of those 27 districts were more than 60 percent of the students eligible for the federal lunch program, and that district had the lowest performance score of the group.
In the three districts that would fall into the “provisionally accredited” category, 62 percent, 72 percent, and 90 percent of students participated in the subsidized lunch program.
In all four of the districts that would be unaccredited (five if one includes the Construction Career Center), the proportion of students eligible for subsidized lunches exceeds 85 percent.
One reason that DESE’s performance review tells us more about family wealth than school performance is that it measures things on which families have great influence. Fully one-half of the possible 140 points on the accreditation scale are based on student scores on tests purchased from textbook publishers. Another 20 points are based on the results for students taking college entrance exams. The other 50 points are based on graduation rates (30 points), attendance (10 points), and the proportion of graduates placed in college or a job within six months (10 points).
The problem with the tests is that they measure so much more than just academic achievement. They measure a complex combination of traits, which includes achievement, but also includes culture and motivation. DESE and test publishers ignore that complexity by assuming a sameness to families and people that doesn’t exist.
One culture? Not quite. A question that assumes familiarity with golf, for example, introduces a cultural bias against students who grow up in families and neighborhoods unfamiliar with the game.
Everyone is equally motivated? Not so. Psychologists have found that some individuals are internally motivated to always try to do their best on tests, even it they can’t perceive of any benefit from doing so, but others aren’t. It is possible that in life children from more privileged backgrounds more often learn that effort brings rewards, and children from disadvantaged circumstances more often learn that it doesn’t.
Recent research also shows that poverty affects one’s performance on tests. When people living in poverty are most secure, they perform as well on cognitive tests as people who are well-to-do. But, when they are most financially stressed, they perform worse on the same tests.
In science, one has to isolate a variable to measure its response. Missouri’s achievement tests, in failing to isolate academic achievement from cultural influences, motivation, and poverty, fail to really tell us anything about academic achievement or school performance.
Peter Downs is a former member of the St. Louis city school board and author of “Schoolhouse Shams: Myths and Misinformation in School Reform,” from Rowman & Littlefield Education.
