Do higher test scores really mean better N.C. schools?
Take a closer look
By Casey Hurley
Sept. 20, 2003 3:36 p.m.
From "Alice in Wonderland," by Lewis Carroll:
At last the Dodo said, "Everybody has won, and all must have prizes."
. . . and the whole party at once crowded round her, calling out in a confused
way, "Prizes! Prizes!"
Alice thought the whole thing very absurd, but they looked so grave that she
did not dare laugh.
I felt like Alice, last week, when I read in the AC-T (Sept. 10, and Sept. 14) that
11 of 39 Buncombe County Schools are designated Schools of Excellence and
21 are designated Schools of Distinction. My ears were ringing, "Prizes!
Prizes!"
Headlines were jubilant: "Statewide, students score higher on tests." "ABCs
results improve in WNC." But another headline captured the confusion and
misunderstanding rampant among citizens and educators, alike: "Complicated
formula means good schools get no recognition."
The purposes of this column are to ask if jubilation is warranted and to clarify
some of the confusion surrounding the ABC tests.
First, why the jubilation? Like Alice, I think it absurd to award "prizes" because
of students' test scores. To illustrate this absurdity, I will compare the North
Carolina driver's license testing program with our state's emphasis on ABCs
test scores.
The purpose of the written drivers' test is to assure that drivers know the
rules of the road. The purpose of the road test is to assure that drivers have
the ability to use their knowledge, skill and judgment to drive safely in traffic.
The first step toward obtaining a driver's license is to acquire a learner's permit
by correctly answering 80 percent of the questions on a written exam. Before
1999, the cutoff was 70 percent. Did DMV officials lose sight of the purposes of
the battery of driving tests when they raised the written exam standard?
I ask this because raising the standard on the written exam brought additional
costs to the citizens and taxpayers of North Carolina (more people had to
re-take the test) without bringing a corresponding benefit in safer driving.
Does anybody believe that raising the written exam standard decreased the
number of accidents, injuries and deaths on N.C. roads? I don't, because I
have never known of an accident caused by the inability of a driver to answer
multiple-choice questions.
The point is that the 70 percent standard was adequate; and, although 80
percent is higher, it carries no additional benefit at a higher cost.
This is similar to what recently happened in North Carolina public schools.
In 1995 the state legislature directed the State Board of Education (SBE) to
develop a restructuring plan for public education.
The SBE responded with The School-based Management and Accountability
Program, which "holds schools accountable for the educational growth of
students over time ..." (Accountability Brief, Public Schools of North Carolina,
March, 2003).
This publication goes on to specify that, "The growth of students is determined
by scores on the North Carolina End- of-Grade Tests of Reading
Comprehension and Mathematics."
Just as the DMV established a standard that cost the state more, with little or
no benefit to the safety of North Carolina drivers, so the State Board of
Education established standards, and testing on those standards, at great
cost to the state, with little or no benefit to the education of North Carolina
youth.
Emphasizing written tests has reduced student opportunities for "road tests."
It seems absurd to me.
Twenty years ago, research on the purposes of public education found that
Americans want public schools to pursue four purposes - the personal, social,
vocational and academic development of our youth.
By focusing on raising student test scores, North Carolina has diverted money
from these four purposes.
Recently, John Goodlad, educational researcher and philosopher, asked, "Have
we been beguiled into assuming that the academic, assessed by tests, begets
understanding and desired behaviors in these other domains (personal, social,
and vocational)? If so, tomorrow's adults and the nation will pay dearly for
today's egregious mistake." (Education Week, April 23)
I sadly admit that North Carolina students have improved their End-of Grade
and End-of-Course exam scores - sadly because this accomplishment has been
achieved at the expense of more important purposes.
The shallowness of higher test scores needs to be contrasted with the
richness of authentic personal, social, vocational, and academic performance.
Higher test scores are only marginally related to higher academic achievement,
and they relate not at all to other public school purposes.
So, let's control our jubilation. Like Alice, I see it as absurd and I am trying not
to laugh; unlike Alice, I am also trying not to cry.
Next, I will offer my explanations for some common questions citizens may
have about testing vocabulary and data.
First, many readers may wonder, "Why are our schools suddenly so much
better?" Politicians and educators say North Carolina schools are now some of
the best in the nation.
And buoyed by recent national acclaim and supposed success, the 2003
session of the North Carolina General Assembly strongly endorsed "the
governor's goal of making North Carolina's system of education first in America
by 2010."
This kind of rhetoric causes confusion if teachers', students', and citizens'
experiences with their public school have not improved appreciably since 1995.
Since 1995, educators and politicians have invested heavily in improving
students' test scores. Consequently, North Carolina students' scores have
improved.
When organizations invest time, money, personnel and other resources into
improving one area of operation, that area is likely to improve.
To understand the meaning of this improvement, however, students, teachers
and citizens should compare their experiences in public schools with claims of
improvement. If their experiences in the school have not improved appreciably
since 1995, it may mean that improving test scores is a shallow
accomplishment.
Citizens should decide for themselves if investing in higher test scores has
paid off in better schools.
Are today's students having a better school experience than those who
attended school when North Carolina was not acclaimed?
Another confusion has arisen concerning what the tests actually measure. Do
they measure effectiveness of school programs? Teacher effectiveness?
Student achievement?
The ABCs accountability program was designed to measure a school's progress
toward helping all students reach an acceptable level of growth.
The key terms are "school" and "growth."
The ABC testing program is designed to measure the growth achieved by the
student population of each school.
Confusion occurs when scores are used for other purposes. Principals,
teachers, and parents, for example, might assume that scores can be used to
measure teacher effectiveness. That would be a misuse of the data.
Similarly, parents may think that the purpose of the exam is to assess their
child's growth. It is not.
A secondary purpose of the tests is to provide parents with percentile scores,
which simply tell parents where their child scored in relation to others who
took the test.
Percentiles are easy to understand, but they do not provide data about the
main purpose of the testing program.
Percentile scores are reported because they cost nothing to compute, and it
tells parents how their child compares with others. In this case, however,
reporting percentile scores may cloud parental understanding of the real
purpose of the exams.
The scores that are important are the scale scores.
The only way to avoid confusion about what the ABCs tests actually measure
is to talk with your principals and teachers about the school's scale scores.
These carry the meaning.
Finally, there is much confusion about the relationship between the state ABCs
accountability program and the federal No Child Left Behind (NCLB) legislation.
First, citizens should understand that the NCLB program does not involve a
separate set of tests. The federal program is based on a high school's
graduation rate and test data from the administration of the high school exit
exam, and the ABCs end-of grade tests.
Second, citizens should realize that the two accountability programs use
different standards to assess student growth. The ABCs program uses a
standard which measures the annual growth of the whole school population,
whereas NCLB assesses progress of student subgroups.
Specifically, NCLB assesses the annual yearly progress (AYP) of all students,
and of students in the following subgroups: American Indian, Asian, black,
multi-racial, white, economically disadvantaged, limited English proficiency, and
disabled. Students in all subgroups must meet AYP, or the school fails to
satisfy NCLB requirements.
Third, citizens should know that the procedure for setting NCLB standards (as
with the ABCs) has as much to do with politics as it does with education.
For example, when it became clear to Michigan legislators that their AYP
standard was so high that few of their schools would reach it, the standard
was lowered.
The politics of standard setting is a column in itself. If readers have questions
about this or education topics in our state and region, please e-mail the AC-T
editorial department. I may be able to address them in a future column.