search BFS site

@BFS! archives

Testing

by Dr. Michael Nill
Head of School

The value of standardized tests is one of the more controversial issues in education today, often even superseding crumbling buildings and curricular reform. Poor record keeping and scoring errors have added to the intensity of the debate.

Having extensively analyzed standardized test results for BFS and two other independent schools, I am convinced that some of these tests do have value, particularly in helping a school locate relative strengths and weaknesses in its curriculum. On the other hand, I am increasingly concerned about the growing emphasis standardized tests are receiving and the wide-ranging ways they are being used and publicized. Newer high-profile and high-stakes tests such as the one introduced last year in New York for 4th graders had tremendous implications for individual students and even individual schools. Should independent schools like BFS administer such tests?

Testing and accountability

One major impetus for this renewed emphasis on standardized tests is the effort to hold schools, teachers and students more accountable. School officials and politicians believe it is the cheapest, most efficient “external” tool to achieve this objective. If a substantial number of students fail to achieve an established cut-off point on tests, teachers and/or schools can be labeled sub-par. Thus, it’s argued; educators and parents can determine which are the “best” schools and how public schools stack up against private or independent schools.

Can standardized tests reliably measure the effectiveness of different schools’ instruction and are they positive forces for education reform? I think not, and my opinion is supported by an unlikely source: Paul Barton, Director of the Policy Information Center of the Educational Testing Service and author of “Too Much Testing.”

Problems in interpreting tests

Students often come into classrooms with different levels of skill and content mastery. This can make it difficult to accurately measure learning in a particular year or set of years. One way to reduce this problem would be to administer a test at least twice over a set period and then compare mean scores for the same group of students for each of the times they took the test to determine whether skill levels have improved.

But, this approach to test interpretation is complex and seldom used. Barton pointed out the imbalance in labeling Arkansas as having the worst and Maine having the best overall scores of 4th graders in 1992 and 8th graders in 1996 on math tests administered under the National Assessment of Educational Progress. When the difference in scores between 1992 and 1996 was calculated, overall scores of each state’s students advanced by 52 points. Barton asked: Are Maine and Arkansas at the two ends of school quality, or are they actually equal? Further, the interpretations of the scores didn’t eliminate the scores of students who moved into or out of the two states during the perhaps too-long four-year interval between tests.

Different curricular goals pose another difficulty in using test scores to compare schools. When schools have their own site-based management and missions, they are likely to have distinct educational programs not easily compared under regional testing norms.

Even more problematic is the designation of a standardized score cut-off point that a student must meet in order to be promoted. Such a move takes assessment out of the hands of those in the best position to evaluate students: teachers, administrators and guidance staff.

Independent schools are put in a bind by the rush of the media and the public to draw conclusions from standardized tests. It was suggested that scores of independent schools that participated in recent New York tests did not measure favorably versus public schools. And the large number of independent schools that did not take the tests were suspected of trying to hide “the dirty little secret” that they’re not any better than public schools. Such conclusions are unwarranted but still result in poor public relations.

Standardized tests that can be useful in monitoring the curriculum of independent schools are the SAT II tests and Advanced Placement exams, as well as the annual battery of verbal and quantitative tests administered by the Educational Record Bureau (ERB). An advantage of these tests is that they are scored according to national, suburban and independent school norms and measure a number of skills. When patterns of weakness appear, schools can adjust their curriculum. Individual students’ ERB verbal and quantitative results often correlate well with scores on the SAT I test. But caution should be exercised: test taking abilities, the time of day, emotional factors, and the particular sample of a specific year’s test items can mitigate against accurate measurement of individual students’ skills. Results are best interpreted in the context of patterns over a number of years.

Because of the complexities of standardized test scoring, many independent schools do not automatically send ERB test results to families. The schools prefer to have a division head or learning specialist meet one-on-one to interpret results. Parents and students need to understand the limits of standardized tests. Results do not reflect levels of student effort and thus can foster the erroneous belief that intelligence levels are static and innate. Motivation and level of effort are the best predictors of future success in school and in life.

Other criteria in evaluating schools

Given the problematic features of standardized testing, it should never represent more than one factor in the evaluation of a particular school. For an N-12 college-preparatory school like BFS, a key evaluative factor must be its record of college admissions. Are the graduates accepted by their first-choice colleges, schools which represent diverse strengths and educational programs? Are selective colleges interested in the school’s graduates? The fact that 85% of last year’s BFS graduates were accepted by first-choice colleges in a year that was widely reported as being a very difficult one for acceptances would be one indication of the effectiveness of the school’s program, as would be the case for any school with a similar record of acceptances.

Other important criteria are the characteristics of good schools as defined by recent educational reform efforts: strong sense of community, high expectations for achievement, strong advisory system, close student faculty ties, involvement of parents, close communication between the school and parents, small class size, professional staff with solid knowledge of their subject area, on-site management, and a clear mission that drives the school’s practices and curriculum. Only the members of a school community, including parents, of course, are in a position to judge how well a school meets these criteria. Standardized tests would be poor instruments to determine whether a school meets these particular criteria and would be even less effective in measuring the degree to which students meet the affective goals of a school such as openness to criticism, cooperative attitude in groups, confidence, self-motivation and appreciation for subject matter.

Schools are likely to vary in the list of criteria by which they measure their success. But whatever the criteria, relying only on standardized test results to measure success is utter nonsense from an educational point of view. In no way can they show that a school is healthy and effective in any overall sense even if its students regularly score higher than students in any other school in the world. On the other hand, we have seen that standardized testing can help measure some aspects of a school’s effectiveness if the results are responsibly interpreted and used in strict accord with what the particular test in question can reliably and validly measure. Beyond that I would have to agree with Barton’s conclusion: we may need better-constructed tests, but we certainly don’t need more of them.

Back to @BFS! archives
Back to @BFS!