Dueling Brochures: Everyday Math vs. Saxon Math

December 24, 2003

We had an interesting experience when researching math programs for a proposed charter school. We did not want the educational program to be based on popularity or "wonderfulness," rather we wanted the program to have been proven in real classrooms.

We'd heard good things about both "Chicago Math" (aka Everyday Math and UCSMP) and "Saxon Math" so we wrote to both, asking for more information.

Now of course both would put their best marketing face on, and we knew to take their findings with a grain of salt.

Everyday Math

The Everyday Math materials arrived first. We were very impressed by all the research that they said went into the design of the program. They illustrated their results using four double-line graphs—one line in the graph represented Everyday Math and the other was a control group—each graph showing pre- and posttest results.

But in three of the four graphs the pretest was different from the posttest. This means that before the study, they tested the students using one form of assessment, then after the study they picked a different test entirely! These were the only graphs that appeared that the Everyday Math group did better than the control group, but once you factor in the switcheroo mid-experiment the data becomes meaningless.

In the fourth graph, we were pleased to note that both the pre- and post-test used were identical. However in this graph, the two lines were parallel, meaning the Everyday Math group posted an identical gain to the control group!

In none of the four graphs was the identity of the school given.

We were disillusioned, to say the least. Either the entire group of University of Chicago experts from the Everyday Math program had forgotten how to conduct proper research, or... (at that moment the doorbell rang.)

Saxon Math

It was the postman with the package from Saxon.

We looked at the Saxon materials with low expectations. Their research booklet was terse, just a simple seventy-eight pages of bar graphs, called ?The Saxon Report Card.?

Many of the bar graphs were longitudinal, illustrating a before-Saxon base line and a yearly progression afterwards. Some of the results were fair, some good, and others stellar. All of the results were positive.

They also compared apples-to-apples, in each case showing before and after results using the same assessment, such as the SAT-9.

But what really got our attention was the following: Not only were all seventy schools or districts identified by name and town (from twenty-one states plus the District of Columbia), each included the principal's name!

The results spoke for themselves, but we wanted to double-check for ourselves. Using a separate list of Saxon-using schools, we called two Philadelphia principals. Both were very pleased with the program, and both pointed out that their PSSA and SAT-9 scores had gone up every year since they adopted the Saxon Math program.

We weren't quite sure what to make of all this. Either this was an elaborate hoax—including the tapping of our phones—engineered by the Saxon folks, or... (at that moment, the kitchen timer rang.)

Dinner was ready.

---

An amazingly similar incident to our Chicago Math experience is recounted in Elaine McEwan's Angry Parents, Failing Schools, in which a pair of former college professors examined the research presented in an Everyday Mathematics (Chicago Math) brochure and concluded "the research was seriously flawed. The snazzy packet was 'smoke and mirrors' and unfortunately a lot of parents and educators were fooled."