ATESLA: Test every kid, every year.

December 23, 2005

Standardized testing in K-12 classrooms, especially as mandated by states' compliance with No Child Left Behind, is roundly criticized. While many of these criticisms are way off the mark (testing is racist, unfair, or just plain evil, testing doesn't measure the worth of a child), we feel that the way comparative testing is done currently (as we wrote yesterday) is seriously flawed.

Our solution? Have more frequent testing, measuring every child each year.

While it may seem strange to deal with the problems of testing by having more of it, we're completely serious. Of course if every child were tested each year using the same expensive tests as currently in use today, the cost would skyrocket. Thus we recommend ditching the longer tests (with their open-ended responses which are so expensive--and subjective--to grade) for the time-proven shorter tests such as the Iowa Test of Basic Skills, which can be administered in a few hours.

So we propose the ATESLA model for the use of standardized testing in schools: Annual Testing of Every Student with Longitudinal Analysis.

Annual Testing: No explanation should be needed here, since this is an aspect of current standardized testing. But what we propose isn't just annual testing, but annual testing of every child. So tests need to be streamlined to reduce costs, and made homogenous through all the grades a child is in a particular school, to permit valid comparisons.

Every Student: Current testing strategies take a cross-section of students, for example 5th, 8th, and 11th, and measure them each year. This makes the data nearly meaningless to analyze year-over-year, for last year's 8th graders and this year's 8th graders are apples and oranges. A far better method would be to test every child each year, for that permits the far more meaningful longitudinal analysis.

Longitudinal Analysis: Simply put, longitudinal analysis is the measurement of change over time, which is a perfect fit for schools, which are supposed to be developing students' knowledge and skills over time. It is amazing that this form of analysis is rarely used by states and schools.

The simple reality is that states and schools don't often consistently measure every child every year, using the same brand of test each year. In other words, children might be tested every year, but using different assessments, which cannot be compared. For example, in Philadelphia over the span of just a few years students were tested using the SAT-9, the TerraNova, and the PSSA, none of which is correlated with one another. What a wasted opportunity.

Rather, if the same assessment were used each year, multiple avenues of analysis become available. (By "same assessment" we mean the same brand of test, by one publishing company, most of which are available in multiple forms for all the grades in elementary through high school.) Three of these avenues are: Analysis by Cohort, Analysis by Teacher, and Analysis by Student, discussed below. In each case the basic comparison will be year-over-year, relating last year's scores to this year's. The big question will be "how much of a gain did the students make in one year?" More ambitious analyses can be done over spans of multiple years, but the ideal window is one school year.

Analysis by Cohort: By cohort we mean the same class of students (as in "Class of 2008") as it moves through the school from year to year. One year they're freshmen, next they're sophomores, et cetera. Simply average all the 9th grade scores from last year, and compare them with the 10th grade scores from this year, which is basically the same students. Sure, there will be some turnover and attrition, but for the most part the majority of the class will be the same sample of students. This is far better than comparing all 11th graders each year, which are most certainly not the same students.

Analysis by cohort will answer "big picture" questions such as, "How much growth have our 5th graders experienced in the past year?" Since each school will only have a handful of cohorts (depending on the grade range that the school serves), this makes an analysis the relative success of the school much easier to quantify with just a few numbers.

Analysis by Teacher: This is a very controversial idea, for it means taking all the students taught by one teacher, and averaging the change in scores year-over-year. In the elementary grades there probably would be one teacher per child (for reading and math). In later grades, where subjects are taught in periods by different teachers, then the math teachers can be differentiated from the reading teachers.

Any principal worth her salt, upon seeing the results from different teachers in her school, could plan for mentoring, professional development, or other forms of intervention to help the teachers with weaker scores. But the principal doesn't have to be "the heavy." In larger schools there will be a wider range of results, which can be an excellent opportunity for the math or English department to get together and share best practices, paying special attention to what's going on in the top teachers' classrooms. Without bothering the principal, the department head could take charge, banding the department together to work to raise achievement.

In our experience the failure of many schools is not due to rank incompetence (although we've seen plenty of that), rather it's due to either unfocussed efforts, or focussed efforts in the wrong direction (for example by the use of ineffective progressive education techniques). Longitudinal achievement data, disaggregated by teacher, can help to sharpen a school's focus, towards effective practices which really show results. That shouldn't be controversial at all.

Analysis by Student: Is Johnny learning? How much is he learning? Are his scores rising steadily each year? Does Johnny need help in reading or math? All of these questions are easily answered by looking at the longitudinal data from Johnny's scores. "Ah, but Johnny already takes standardized tests each year," you say. True, but currently many of these score reports are only affixed to Johnny's school record in some file folder in some cabinet in some office. A yearly chore is to take the results from standardized tests (which literally come on sheets of stickers) and stick 'em to the student records.

Longitudinal analysis would be far more valuable than these stickering sessions, for no one usually looks at those stickers unless the student is referred for Special Education placement, or Johnny is transferred to another school. If longitudinal analysis is done right, it will span more than two years, giving a truer picture of Johnny's educational growth. That's what teaching and schools are all about.

Grade Equivalence (GE) Scores: Most of the common standardized tests such as the Iowa Test of Basic Skills issue scores in multiple forms such as standard scores, stanines, and percentile ranks (compared locally and nationally). But a very user-friendly score is the "Grade Equivalent" score (GE), in years and months. A score typical of a 9th grade student in her 3rd month of school would be a GE of 9.3, which enables the reader to instantly see if a student (or a group of students) is on grade level or not. (For example, an 11th grader with a GE of 9.3 should raise eyebrows.)

Returning to our question of "how much of a gain did the students make in one year?" the Grade Equivalent scores make for excellent comparisons. If the tests are given each year at the same time (we'd prefer as late in the school year as possible, not March or April as is often done), then the two GE scores can simply be subtracted to find the change in score. For example, say last year the score was 4.7 (equal to a 4th grade student in the 7th month of school) and this year the score is 5.5. This gives us a difference of +0.8 which shows growth, to be sure, but growth of only 8 months after ten months of school! If this keeps up, the student will be a full year behind in five years.

If the GE scores were computed for every student, every year, and analyzed by cohort, by teacher, and by student, then there would begin to be true accountability in today's schools. Of course at the district or state level, the analysis could be by school as well. There may well be schools which give the students an increase in Grade Equivalence of +0.8 each and every year, which is like a six-cylinder engine running on only five. Any student who spends much time at that school will simply fall further and further behind.

Accountability: We've been teaching for years in the big city, and we've seen many, many schools which are not serving students well at all. They may care very deeply about their kids, and these schools usually are filled with a bunch of good-hearted people, yet somehow their students fail to meet reasonable standards of achievement. Amazingly, no one is held accountable for this failure. There are K-8 schools which fail to teach children to read fluently or do basic math, yet when faced with failing scores at either the 4th grade or 8th grade, they throw up their hands: "We have no idea how that happened!"

Longitudinal analysis would be able to show exactly where the problem is, which would help principals solve it. This analysis is only possible if every student were tested each year.

Annual testing. Every student. Longitudinal analysis. It's not rocket science!