Do higher test scores really mean better N.C. schools?

                  Take a closer look

                  By Casey Hurley

                  Sept. 20, 2003 3:36 p.m.

 

                  From "Alice in Wonderland," by Lewis Carroll:

 

                  At last the Dodo said, "Everybody has won, and all must have prizes."

 

                  . . . and the whole party at once crowded round her, calling out in a confused

                  way, "Prizes! Prizes!"

 

                  Alice thought the whole thing very absurd, but they looked so grave that she

                  did not dare laugh.

 

                  I felt like Alice, last week, when I read in the AC-T (Sept. 10, and Sept. 14) that

                  11 of 39 Buncombe County Schools are designated Schools of Excellence and

                  21 are designated Schools of Distinction. My ears were ringing, "Prizes!

                  Prizes!"

 

                  Headlines were jubilant: "Statewide, students score higher on tests." "ABCs

                  results improve in WNC." But another headline captured the confusion and

                  misunderstanding rampant among citizens and educators, alike: "Complicated

                  formula means good schools get no recognition."

 

                  The purposes of this column are to ask if jubilation is warranted and to clarify

                  some of the confusion surrounding the ABC tests.

 

                  First, why the jubilation? Like Alice, I think it absurd to award "prizes" because

                  of students' test scores. To illustrate this absurdity, I will compare the North

                  Carolina driver's license testing program with our state's emphasis on ABCs

                  test scores.

 

                  The purpose of the written drivers' test is to assure that drivers know the

                  rules of the road. The purpose of the road test is to assure that drivers have

                  the ability to use their knowledge, skill and judgment to drive safely in traffic.

 

                  The first step toward obtaining a driver's license is to acquire a learner's permit

                  by correctly answering 80 percent of the questions on a written exam. Before

                  1999, the cutoff was 70 percent. Did DMV officials lose sight of the purposes of

                  the battery of driving tests when they raised the written exam standard?

 

                  I ask this because raising the standard on the written exam brought additional

                  costs to the citizens and taxpayers of North Carolina (more people had to

                  re-take the test) without bringing a corresponding benefit in safer driving.

                  Does anybody believe that raising the written exam standard decreased the

                  number of accidents, injuries and deaths on N.C. roads? I don't, because I

                  have never known of an accident caused by the inability of a driver to answer

                  multiple-choice questions.

 

                  The point is that the 70 percent standard was adequate; and, although 80

                  percent is higher, it carries no additional benefit at a higher cost.

 

                  This is similar to what recently happened in North Carolina public schools.

 

                  In 1995 the state legislature directed the State Board of Education (SBE) to

                  develop a restructuring plan for public education.

 

                  The SBE responded with The School-based Management and Accountability

                  Program, which "holds schools accountable for the educational growth of

                  students over time ..." (Accountability Brief, Public Schools of North Carolina,

                  March, 2003).

 

                  This publication goes on to specify that, "The growth of students is determined

                  by scores on the North Carolina End- of-Grade Tests of Reading

                  Comprehension and Mathematics."

 

                  Just as the DMV established a standard that cost the state more, with little or

                  no benefit to the safety of North Carolina drivers, so the State Board of

                  Education established standards, and testing on those standards, at great

                  cost to the state, with little or no benefit to the education of North Carolina

                  youth.

 

                  Emphasizing written tests has reduced student opportunities for "road tests."

                  It seems absurd to me.

 

                  Twenty years ago, research on the purposes of public education found that

                  Americans want public schools to pursue four purposes - the personal, social,

                  vocational and academic development of our youth.

 

                  By focusing on raising student test scores, North Carolina has diverted money

                  from these four purposes.

 

                  Recently, John Goodlad, educational researcher and philosopher, asked, "Have

                  we been beguiled into assuming that the academic, assessed by tests, begets

                  understanding and desired behaviors in these other domains (personal, social,

                  and vocational)? If so, tomorrow's adults and the nation will pay dearly for

                  today's egregious mistake." (Education Week, April 23)

 

                  I sadly admit that North Carolina students have improved their End-of Grade

                  and End-of-Course exam scores - sadly because this accomplishment has been

                  achieved at the expense of more important purposes.

 

                  The shallowness of higher test scores needs to be contrasted with the

                  richness of authentic personal, social, vocational, and academic performance.

 

                  Higher test scores are only marginally related to higher academic achievement,

                  and they relate not at all to other public school purposes.

 

                  So, let's control our jubilation. Like Alice, I see it as absurd and I am trying not

                  to laugh; unlike Alice, I am also trying not to cry.

 

                  Next, I will offer my explanations for some common questions citizens may

                  have about testing vocabulary and data.

 

                  First, many readers may wonder, "Why are our schools suddenly so much

                  better?" Politicians and educators say North Carolina schools are now some of

                  the best in the nation.

 

                  And buoyed by recent national acclaim and supposed success, the 2003

                  session of the North Carolina General Assembly strongly endorsed "the

                  governor's goal of making North Carolina's system of education first in America

                  by 2010."

 

                  This kind of rhetoric causes confusion if teachers', students', and citizens'

                  experiences with their public school have not improved appreciably since 1995.

 

                  Since 1995, educators and politicians have invested heavily in improving

                  students' test scores. Consequently, North Carolina students' scores have

                  improved.

 

                  When organizations invest time, money, personnel and other resources into

                  improving one area of operation, that area is likely to improve.

 

                  To understand the meaning of this improvement, however, students, teachers

                  and citizens should compare their experiences in public schools with claims of

                  improvement. If their experiences in the school have not improved appreciably

                  since 1995, it may mean that improving test scores is a shallow

                  accomplishment.

 

                  Citizens should decide for themselves if investing in higher test scores has

                  paid off in better schools.

 

                  Are today's students having a better school experience than those who

                  attended school when North Carolina was not acclaimed?

 

                  Another confusion has arisen concerning what the tests actually measure. Do

                  they measure effectiveness of school programs? Teacher effectiveness?

                  Student achievement?

 

                  The ABCs accountability program was designed to measure a school's progress

                  toward helping all students reach an acceptable level of growth.

 

                  The key terms are "school" and "growth."

 

                  The ABC testing program is designed to measure the growth achieved by the

                  student population of each school.

 

                  Confusion occurs when scores are used for other purposes. Principals,

                  teachers, and parents, for example, might assume that scores can be used to

                  measure teacher effectiveness. That would be a misuse of the data.

 

                  Similarly, parents may think that the purpose of the exam is to assess their

                  child's growth. It is not.

 

                  A secondary purpose of the tests is to provide parents with percentile scores,

                  which simply tell parents where their child scored in relation to others who

                  took the test.

 

                  Percentiles are easy to understand, but they do not provide data about the

                  main purpose of the testing program.

 

                  Percentile scores are reported because they cost nothing to compute, and it

                  tells parents how their child compares with others. In this case, however,

                  reporting percentile scores may cloud parental understanding of the real

                  purpose of the exams.

 

                  The scores that are important are the scale scores.

 

                  The only way to avoid confusion about what the ABCs tests actually measure

                  is to talk with your principals and teachers about the school's scale scores.

                  These carry the meaning.

 

                  Finally, there is much confusion about the relationship between the state ABCs

                  accountability program and the federal No Child Left Behind (NCLB) legislation.

 

                  First, citizens should understand that the NCLB program does not involve a

                  separate set of tests. The federal program is based on a high school's

                  graduation rate and test data from the administration of the high school exit

                  exam, and the ABCs end-of grade tests.

 

                  Second, citizens should realize that the two accountability programs use

                  different standards to assess student growth. The ABCs program uses a

                  standard which measures the annual growth of the whole school population,

                  whereas NCLB assesses progress of student subgroups.

 

                  Specifically, NCLB assesses the annual yearly progress (AYP) of all students,

                  and of students in the following subgroups: American Indian, Asian, black,

                  multi-racial, white, economically disadvantaged, limited English proficiency, and

                  disabled. Students in all subgroups must meet AYP, or the school fails to

                  satisfy NCLB requirements.

 

                  Third, citizens should know that the procedure for setting NCLB standards (as

                  with the ABCs) has as much to do with politics as it does with education.

 

                  For example, when it became clear to Michigan legislators that their AYP

                  standard was so high that few of their schools would reach it, the standard

                  was lowered.

 

                  The politics of standard setting is a column in itself. If readers have questions

                  about this or education topics in our state and region, please e-mail the AC-T

                  editorial department. I may be able to address them in a future column.