Ohio State University researchers have developed a new method of measuring school quality based on schools’ 'impact on learning' and their results say that summer vacations and parent incomes have more to do with low test scores than the quality of the schools.

Using this 'impact' measure, about three-quarters of the schools now considered “failing” because of their low test scores would no longer would be failing.

A school system measuring tool where everyone is above average even if they have poor test scores and where good test scores can still mean failure? Is that legitimate or is it just Outcome Based Education for education?

The researchers say that in schools labeled as failing students may have low achievement scores but they are learning at a reasonable rate and they are learning substantially faster during the school year than they are during summer vacation.

In other words, the solution to under-performing schools would be to have school all year long. It is unlikely the National Education Association is going to endorse that without a 33% pay increase.

But they make a reasonable point: “Our impact measure more accurately gauges what is going on in the classroom, which is the way schools really should be evaluated if we’re trying to determine their effectiveness,” said Douglas Downey, co-author of the study and professor of sociology at Ohio State University. Downey conducted the study with Paul von Hippel, a research statistician, and Melanie Hughes, a doctoral student, both in sociology at Ohio State.

They say using achievement scores to measure school quality assumes that all schools have students with equivalent backgrounds and opportunities that will give them equal opportunities to succeed in school. And that’s obviously not true, von Hippel said.

Of course, if you throw out a standardized metric for schools, grades and scores are meaningless - but so is school. With 4 million students graduating each year how would we know which ones can read and do basic math - the very skills a school should be teaching them - if no tests matter?

Some schools are going to be at the top of this 'impact' success curve yet graduate few students with any kind of measurable academic skill. That hardly makes sense.

As you might guess, this is a criticism of measurable testing contained in No Child Left Behind and it comes on the heels of something of a No Child Left Behind win, namely scores showing that girls are no longer left behind boys in mathematics.

Do we want schools to be accountable for the performance of their jobs or not? This study says schools can't be accountable for poor scores, parents' economic situations are the problem.

The study used data from the Early Childhood Longitudinal Study, a national survey conducted by the U.S. Department of Education. The analysis focused on 4,217 children in 287 schools.

The survey measured children’s math and reading scores on four occasions: the beginning and end of their kindergarten year, and the beginning and end of first grade. By comparing test scores at the end of kindergarten and the beginning of first grade, the researchers say they were able to measure learning rates during summer vacation.

Comparing test scores from the beginning and end of first grade allowed the researchers to see how much children learn during the school year.

They then were able to calculate how much faster students learned during the first-grade school year compared to when they were on summer vacation. This was the “impact” score that showed how much schools were actually helping students learn.

By simply changing the definition of failure, 75 percent of the failing schools - those in the bottom 20 percent of achievement scores - were no longer failing.

They basically want to have two sets of scores for schools. Schools for children that can be considered disadvantaged and the schools everywhere else. They also take some schools that are doing just fine down a notch by saying that under their revised scoring 17 percent of schools that are not failing when rated by achievement test scores turn out to be failing when ranked on 'impact.'

“These schools may be serving children from advantaged backgrounds who do well on achievement tests, but the learning rate for their students isn’t dramatically faster when they are in school versus when they are not. In other words, these schools are not having much positive impact,” according to Downey.

The bottom line is that, under the current system, “we are not pressuring the schools that need to be pressured,” Downey said.

This is overall a brilliant declaration of faith in the current educational system:

(1) Failing schools aren't really failing.

(2) Summer vacations when kids are not in school are the problem, not schools.

Based on achievement scores, failing schools tend to be in urban areas, serve a higher percentage of children who qualify for a free lunch, and have a high minority population. Using the logic they use here, the solution would seem to be to get rid of urban schools because the situation is, in their minds, hopeless. Not so, says Von Hippel, because the results of this study also suggest new ways to elevate achievement in students.

“If there’s a school that rates high in educational impact but low on achievement, maybe the school should have a summer program with those same teachers who are having such a positive impact,” von Hippel said.

The testing system could also be changed, they say. Right now, schools test students six times – once each year between 3rd grade and 8th grade. Tests could be given at the end of 3rd grade, the beginning of 4th grade and the end of 4th grade. That way student learning rates could be compared in the summer after third grade with the 4th grade school year. Another set of three tests could be given at the end of 7th grade, and the beginning and end of 8th grade.

“We would have the same number of tests, but information that is substantially more useful,” he said.

The study was funded by grants from the National Institute of Child Health and Human Development, the Spencer Foundation, The John Glenn Institute at Ohio State, and the Ohio State P-12 Project.

Their findings appear in the current issue of the journal Sociology of Education.