ReformK12.com: Sampling, and the point of standardized tests

Sampling, and the point of standardized tests

March 08, 2004

A while back, we made some fun of anti-testing screedmeister Alfie Kohn. Recently gentle reader Pam posted the following comment:

I'm sorry but I think you're all missing the point of what Alfie has to say. It's not that grades and assessments aren't necessary--it's just that there needs to be more than one way to assess growth, achievement and success. Standardized testing provides one approach-- a pencil and paper approach to measure what a student knows, but this one-size-fits-all method to measuring individual learning is NOT a true assessment of mastery--or knowledge.

Pam, you are exactly correct in that paper-and-pencil assessments aren't "true" measurements of mastery or knowledge.

For a true measurement, you'd need either a Vulcan mind-meld, or you'd need to observe this person over a period of months or years in an authentic setting, to truly gauge learning and skills.

Neither of those is terribly practical (and Vulcans are notoriously hard to find).

Enter Standardized Tests.

There aren't many folks who'd claim that these bubble-in tests are the best way to measure learning. They are, however, the least worst solution. Out of all the ways to measure learning (direct observation, "authentic tasks," portfolio-based assessments, open-ended response, juried competitions, and standardized tests) there's only one that has any hope of providing some sort of standard at a reasonable cost.

We don't want to be cynical and claim it's all about money, but one shouldn't dismiss practicality. Any method of measuring learning must survive real-world criteria, and if your goal is to measure all the students in the state (any state), standardized tests are the way to go.

One reason they're so effective is because of the concept of sampling.

Statisticians know there are two ways to measure a population: measure every element in the population, or sample. For example, say you want to measure how well a candidate is doing in the polls in the weeks before an election. You could contact all registered voters and ask them how they're going to vote, which is extremely expensive, although you'd get a very accurate number. Or you could pick a "sample" of voters and ask them. The result you get isn't the best result, but it's a pretty good one.

Standardized tests work in much the same way. Although they can be designed such that you're not "sampling students" (for example, if you want to test every child in your population, say every 4th grader in the state), the point of the tests is you're sampling knowledge.

For example, a math standardized test cannot include every problem of every type that the student should know how to solve, the student would need a whole box of number 2 pencils! Rather, the test includes a variety of problems, sampling the skills for a given educational level.

To help put standardized tests in perspective, here's this comment from The Educated Child, by William Bennett:

Standardized tests can be useful instruments, but take their results in context. They are not necessarily evaluating whether your child knows what a well-educated pupil at her grade level ought to know, but rather how she compares to other pupils who've taken the same test.

This of course doesn't prevent the abuse of standardized tests, holding them up as the be-all, end-all result of education.

Getting back to Pam's original point, she writes, "there needs to be more than one way to assess growth, achievement and success."

Your wish is our command. The name of the means of assessing growth, achievement, and success? They're called "teachers."

Posted by ceb into Misconceptions , Testing & Grading
| ↑ top ↑ | « previous entry | next entry » | ReformK12 home

Comments

"...but this one-size-fits-all method to measuring individual learning is NOT a true assessment of mastery--or knowledge."

Who ever said it was? But, it is not meaningless. Standardized testing is a lowest-common denominator testing of minimal competence. It is after the fact and of little direct use to the student. These tests set a minimal level of accountability for schools that just as soon have no accountability. Look at the NAEP test questions and terrible scores. (They have a web site.) Please don't talk about testing true knowledge and mastery unless these scores increase. Is it possible that students have true knowledge and mastery if they do very poorly on these tests? Parents should look at the questions and decide for themselves.

As a parent, I look at schoolwork, homework, projects, quizzes and tests to gage true knowledge and mastery. That is the way it should be. Some, however, would have us believe that it is OK to do poorly on these tests because the kids have some other kind of true knowledge and skills. I don't buy it. Look at the questions.

Steve March 9, 2004 09:50 AM

Those who talk about 'other' kinds of knowledge and skills are very long on negativity about standardizing testing but very silent on exactly WHAT kinds of alternate knowledge and skills need to be tested/evaluated.

Until they can articulate their alternatives, I'm going to give them all the attention that their amorphous objections deserve: none.

Claire March 30, 2004 02:50 PM

Standardized tests, although cheap and fast, fail to provide data of any value to anyone except those more interested in ranking and rating in order to apply a data point on a bell shaped curve. (There are a lot of testing firms with healthy bottom lines, though. They make hefty political contributions too)

Data, the output of standardized tests, without context is meaningless! The same applies to all statistical applications. No context, no meaning.

To assume that other forms of assessment are more expensive is a gross error, the outcome of very short term thinking. What is the total lifetime cost of failure to assess real learning? What is the total life time cost of shutting down the desire to learn in children? Not to mention the long term health issues that result from childhood stress.

This cost is far more than anyone can measure since, what does it cost to lose a culture, the ability to think and progress civilization?

'Education' that operates within the small world of schools and professional educators is doomed to failure. If we do not let go of our paradigms, one of which is standardized testing, we will never see that there are third, fourth, fifth, etc., alternatives to what we have been doing for so long.

To subscribe to a single method of assessment is to admit to a lack of imagination and initiative.

What about apprenticeships - they are still used and work incredibly well. Assessment through performances of knowledge gained that makes an honest and real contribution to the community at large. Peer review! How about the student doing some teaching and sharing knowledge. These are all value added performances. Too see third graders figure out how to measure without a ruler, or an alogorithm, and then figure out how to communicate that measure with understanding to a total stranger - a test can never measure that. Memorizing rules provided by the 'big people' will never contribute to learning - only to short term memory skills.

And if your imagination is really stuck visit Formula SAE on the Web and investigate a process of performance assessment that provides life time learning and real job opportunities.

Numbers will provide an historical snapshot of a point in time and not much more. These numbers have nothing to do with learning for the future. And that's the point isn't it, learning.

Chuck June 15, 2004 07:30 PM