The relationship between education spending and test scores

Good public policy is impossible without understanding causality. The most common logical mistakes in the policy debate is the tendency of confusing correlation with causality. One common reason correlation is confused with causality is the failure to make demographic adjustments.

A demographic adjustment without some form of exogenous variation can rarely tell us what the definitive truth is. But it can often be used to discard weak arguments. Usually, I focus on criticizing the left in this regard, but let's make an effort to be intellectually honest and scrutinize the right for once.

The argument of the right is that educational spending is wasteful, and doesn't produce outcomes. They base this on the fact that the United States already spends more on education than almost all other nations, and more than ever in history.

John Stossel for instance writes that:

"If money were the solution, the problem would already be solved"

The difficulty is that education spending goes up when social needs go up. This is the classical head-ache pill example: just because people with worse head-aches take more pills, it doesn't mean the pills are ineffective or even causing the pain. The same argument is also true of spending on the police. Cities with more crime have more police, not because the police are causing higher crime rates, but because they are a response to high crime.

So for instance the entity that spends the most on education and has the worse test scores in the U.S is the District of Columbia. This doesn't prove anything, because D.C also has much more poverty and many more disadvantaged students. The relevant policy question is what would happen to test scores if D.C spent less.

Let's start with a simple cross-country regression of the normal, unadjusted PISA-scores (the ones the media keeps reporting) and education spending per pupil, as measured by the OECD.



It turns out that the correlation is negative (-0.2). The United States spends the second most of any country, and has below average test-scores. Ethnically homogeneous Japan, South Korea and Finland spend at average rates and have the best test scores. Tiny, ethnically homogeneous and "hungry" Estonia spends less than half as much as the United States and Norway on education but has far better test scores.

Argument over. Doing the comparison in the New York Times way, Conservatives win, and Liberals lose.

But what if we do it my way instead? Let's do a crude demographic adjustment, and compare European-origin groups to other European-origin groups, just as I did before. The methodology is explained and defended in detail here.

For Europe, I remove first and second generation immigrants. For U.S, I remove Latinos, Asians and all other groups than non-Hispanic whites (two thirds of the U.S population is non-Hispanic white).

Now, prepare to witness the power of the simple demographic transformation.



The relationship between spending and outcomes is now reversed. More spending is indeed associated with better PISA scores. The United States does well, as predicted by how much she spends on education. Finland with her admired education system is still an outlier, which I guess indicates there really is something special about Finland and their school system.

The graph suddenly makes more intuitive sense, doesn't it?

But we have more variation in data we have not used. Why not break out each U.S state and treat it as a separate observation? We have comparable data on 8-grader test scores for each state from the NAEP which I have converted to PISA-equivalents (method explained here).

We also have access to education spending per pupil for each U.S state, courtesy of U.S Census.

Again first we do the standard, unadjusted PISA-scores for this pooled sample of countries and U.S states.



A flat, slightly downward sloping curve, countries and states that spend more don't appear to score any higher.

Notice that Utah, Finland and South Korea outscore California, even though they spend half as much (they don't need to spend more, because they lack California socially disadvantaged masses). Minnesota beats Sweden, but generally U.S states do poorly, even though they spend more than Europe.

Another victory for Republicans. Clearly doubling money on education has no effect, and may even - somehow - make your countries children score a little less.

Now, look at the exact same graph, only with my standard crude demographic adjustment (For Europe, first and second generation immigrants are removed. For U.S states, all demographic groups other than non-Hispanic whites are removed from the sample).



Again the results are reversed after a demographic adjustment. The advantage of the U.S and the advantage of rich U.S states that spend a lot on education suddenly becomes apparent. More public spending on schools is associated with better outcomes. The mystery of the missing correlation is solved.

Lastly, let's look at test scored of African Americans and state educational spending.



Of course these graphs alone do not prove that the driving factor is education spending itself. In theory, it could be because students in rich states are different, for example in terms of family income. But the relationship still represents powerful suggestive evidence in favor of public expenditure on education.

However the left in the United States doesn't use this argument, because they are ideologically averse to demographic adjustments having to do with race and ethnicity (most of them consider all statistical generalizations about race and ethnicity somehow offensive, regardless of why you are doing it).

The result of liberals political correctness is that they are depriving themselves of a very important argument in a very important debate.