A steeper, better road to graduation. (Feature).

Bishop, John H.

Students learn more and their high-school diplomas become more valuable when they must pass a curriculum-based exit exam like France's Baccalaureate in order to graduate. So why isn't the United States following Europe's and East Asia's lead?

PROMOTERS OF SCHOOL ACCOUNTABILITY SYSTEMS based on rigorous testing often point to the high achievement of secondary-school students in those European and East Asian countries that use curriculum-based exit exams such as France's Baccalaureate and England's GCSE and A-level exams. Such exams can carry extremely high stakes. In England, for instance, they effectively determine whether students are eligible to enroll at a university and to which university and field of study they are admitted. In the United States, the only worthy comparisons are New York's famed Regents exams and a more recently developed system in North Carolina. Traditionally, students in New York who passed the Regents exams in various subjects, all tied to the state's curricula, received a more prestigious diploma that signaled their mastery.

While a number of states, including Texas, Tennessee, Virginia, Maryland, and possibly California, are phasing in or plan to develop a system of rigorous curriculum-based external exams, most states have chosen instead to administer minimum-competency exams. Known generally as exit exams, students must pass minimum-competency exams like New Jersey's High School Proficiency Test (HSPT) to graduate from high school. These exams usually test basic skills in English and mathematics, rarely testing students' knowledge of science, history, or other subjects. Eighteen states required the graduating class of 2000 to pass minimum-competency exams; another 11 states are developing or phasing in such exams, Five states--Connecticut, Illinois, Michigan, Oregon, and Pennsylvania--put on transcripts and honors diplomas students' scores on state tests taken in the 10th and 11th grades, but they do not use them as a prerequisite for graduation.

In reality, the United States may never move to a system of truly high-stakes exams along the lines of France's. The ideal of equal opportunity, and education's unique role in advancing that ideal, is deeply grooved into America's national ideology. Sorting and sifting is certainly a part of the U.S. education system (witness the SAT), but the system is also endlessly forgiving; any student of any age, so long as the money or loans are available, can find a university to attend, unlike in some European and East Asian countries. So reformers who routinely invoke the academic excellence of Europe and East Asia should understand the differences between European and East Asian-style curriculum-based exams and the minimum-competency exams being used by many states. Do they both raise student achievement? By how much? What kinds of positive incentives do they create? And what are the negative repercussions, if any?

Curriculum-Based Exams

My analysis of data collected by the 1995 Third International Mathematics and Science Study (TIMSS) of students in 40 countries shows that curriculum-based exit exams do raise achievement. The study found that students from countries with medium- and high-stakes exit examination systems outperform students from other countries at a comparable level of economic development by 1.3 U.S. grade-level equivalents in science and by 1.0 U.S. grade-level equivalents in mathematics. A similar analysis of 1991 International Assessment of Educational Progress data on 13-year-olds in 15 nations found that students from countries with curriculum-based exit exams outperformed their peers in other countries by about 2.0 U.S. grade-level equivalents in math and about two-thirds of a U.S. grade-level equivalent in science and geography. Analysis of data from the International Association for the Evaluation of Educational Achievement's study of the reading literacy of 14-year-olds in 24 countries found that students in countrie s with rigorous, curriculum-based exams were about 1.0 U.S. grade-level equivalents ahead of students in nations at comparable levels of development but lacking such exams. The final study of the effects of curriculum-based exams compared students living in different Canadian provinces. Students attending school in provinces with rigorous exam systems were a statistically significant one-half of a U.S. grade-level equivalent ahead of comparable students living in provinces without such exams in math and science. Other estimates show similarly positive impacts of curriculum-based exams (see Figure 1).

Why do students in nations and provinces with rigorous exams learn more? How do curriculum-based exams influence school policies and instructional practices? The data show that curriculum-based exams are associated with neither higher teacher-pupil ratios nor greater spending on K-12 education. They are, however, associated with higher standards for entry into the teaching profession, higher teacher salaries (30 to 34 percent higher for secondary-school teachers), and teachers who are more likely to specialize in one subject in middle school and to have majored in the subjects they teach. Teachers appear to be less satisfied with their jobs, possibly due to the increased pressure for accountability under an exam system. Schools, countries, and provinces with rigorous exams devote more hours to math and science instruction, and they build and equip better science labs. The number of computers and library books per student is unaffected by the existence of curriculum-based exams.

Fears that curriculum-based exams have caused the quality of instruction to deteriorate appear to be unfounded. Students in nations with rigorous exam systems were less likely to report that memorization is the best way to learn and more likely to report that they conducted experiments in science class. Apparently, teachers subject to the subtle pressure of an external exam four years into the future adopted strategies that are conventionally viewed as best practices, not strategies designed to maximize scores on multiple-choice tests. Quizzes and tests were more common; otherwise, a variety of pedagogical indicators showed no differences in regions with rigorous exams. Students were also more likely to get tutoring assistance from teachers after school. They were no less likely to like the subject and they were more likely to agree with the statement that science is useful in everyday life. Students also talked more with their patents about schoolwork and reported that their parents had more positive attitud es about the subject.

Similarities

Should we expect similar gains in achievement from the use of minimum-competency exams? Not necessarily. There are important differences between curriculum-based exit exams and tests of minimum competency that may lead to different results, There are important similarities as well. For instance, they both:

* Elicit signals of accomplishment that have real consequences for students. In many education systems, exam results are averaged with teacher assessments to generate final grades for certain courses. In some cases, passing the exam is necessary to graduate from high school. In other cases, passing the exam confers eligibility for a more prestigious diploma or the right to enroll in university. In Europe and East Asia, exam grades also influence the hiring decisions of employers and limit access to oversubscribed lines of study at universities.

* Define achievement relative to an external standard, not relative to other students in the classroom or the school. Exams, whether curriculum-based or minimum-competency, make possible comparisons across schools and among students taught by different teachers, In a theoretical analysis published in the American Economic Review, Robert Costrell of the University of Massachusetts concluded that more centralized standard setting (state or national achievement exams) results in higher standards, higher achievement, and higher social welfare than decentralized standard setting (in other words, teachers' grades or local schools' graduation requirements).

* Are controlled by the education authority that establishes the curriculum for and funds K-12 education. When a state department of education sponsors an external exam, it is more likely to be aligned with the state's curriculum. It is, consequently, more likely to be used for school accountability, not just as an instrument of student accountability. It makes coordinated changes in curricula and exams feasible. Tests established and mandated by other organizations serve the interests of other masters. America's most influential high-stakes exams--the SAT-I and the ACT--serve higher education's need to sort students by aptitude, not the needs of high schools that are trying to reward students who have learned what the school is trying to teach.

* Cover the vast majority of secondary-school students. Exams intended for a set of elite schools or advanced courses influence standards at the top, but they have little effect on the rest of the students. A single exam taken by all is not essential. Many nations allow students to select the subjects they will be examined on; some, such as Ireland, the Netherlands, Scotland, and England, offer both high-level and intermediate-level exams for some subjects.

* Assess a major portion of what students are expected to know and be able to do. Studying to prepare for an exam should mean learning important material and developing valued skills.

Differences

Three critical characteristics distinguish curriculum-based exams from minimum-competency exams. Curriculum-based exit exams:

* Are collections of end-of-course exams. Since curriculum-based exams assess student performance in specific courses, the teachers of those courses (or course sequences) will inevitably feel responsible for how well their students do on the exams. Teachers will not only want to set higher standards, they will also find their students more attentive in class and more likely to complete demanding homework assignments. They become coaches helping their team to do battle with the state exam. Grades on the external exam are typically part of the overall course grade, further integrating the external exam into the classroom culture.

* Report multiple levels of achievement in the subject. If students can only pass or fail an exam, as is the case with almost all minimum-competency exams, the standard will, for political reasons, have to be set low enough so that almost everyone will pass. This will not stimulate most students to put in more effort. End-of-course exams measure a student's achievement level in the subject, not just whether the student exceeds or falls below a specific cutoff point. Consequently, all students, not just those at the bottom, have an incentive to study hard.

* Assess more difficult material. Since curriculum-based end-of-course exams are supposed to measure the full range of achievement in the subject, they contain more difficult questions and problems. This induces teachers to spend more time on cognitively demanding skills and topics. Minimum-competency exams, by contrast, are designed to identify students who have failed to pass a rather low minimum standard. As a result, they tend not to ask questions or pose problems that students near the borderline are unlikely to be able to answer or solve. The likely result is that too much class time will be devoted to practicing low-level skills.

Culture of the Classroom

Curriculum-based exit exams often have profound effects on the relationships between teachers and students and among the students themselves. Consider what happened when a proposal was put forward in Ireland to drop the nation's system of external assessments in favor of having teachers assess their students. The union representing Ireland's secondary-school teachers reacted with a statement saying that a major strength of the Irish education system has been students' perception of their teachers as,, an advocate in terms of nationally certified examinations rather than as a judge." Asking teachers to assess their students, the union wrote, would "automatically result in a distancing between the teacher, the pupil, and the parent. It also opens the door to possible distortion of the results in response to either parental pressure or to pressure emanating from competition among local schools for pupils."

Note how Irish teachers feared that doing away with external assessments would result in their being under pressure to lower standards. For American teachers such pressure is a daily reality. According to an American Federation of Teachers survey, 30 percent of American teachers say they" feel pressure to give higher grades than students' work deserves." Thirty percent also feel pressure to reduce the difficulty and amount of work they assign. Curriculum-based end-of-course exams are likely to alleviate such pressures.

End-of-course examinations may also ameliorate another scourge of contemporary classroom culture: nerd harassment, In Beyond the Classroom, Laurence Steinberg, Bradford Brown, and Sanford Dornbusch's recent study of nine high schools in California and Wisconsin, the authors concluded that "less than 5 percent of all students are members of a high-achieving crowd that defines itself mainly on the basis of academic excellence....Of all the crowds the 'brains' were the least happy with who they are--nearly half wished they were in a different crowd."

Why are the studious called suck ups, dorks, and nerds or accused of "acting white"? In part, it is because many teachers grade on a curve. This means that doing well in a class makes it more difficult for others to get top grades. When exams are graded on a curve or college admissions are based on class rank, students can maximize their joint welfare if no one puts in extra effort. In the game that results, rewards, such as friendship and respect, and punishments, such as ridicule, harassment, and ostracism, enforce the cooperative solution: "don't study much." If, by contrast, students are gauged by an outside standard, they no longer have a personal interest in getting teachers off track or persuading one another to refrain from studying. Peers should become less supportive of students who joke around in class and more supportive of those who cooperate with the teacher.

Evidence from the United States

In the United States, states with minimum-competency exams tend also to have adopted school accountability systems that reward high-achieving schools or sanction failing schools. Therefore, in trying to isolate the effects of minimum-competency exams and end-of-course exams on achievement in the United States, it is necessary to account for the presence or absence of other standards-based reforms, In a study of states' 8th-grade reading, math, and science scores on the 1996 and 1998 National Assessments of Educational Progress, my colleagues and I studied the effect of five different standards-based reform strategies:

* School-by-school reporting of the results of statewide testing

* Rewards for schools that improve on statewide tests or exceed targets set for them

* Sanctions for failing schools, such as closure, reconstitution, or loss of accreditation

* Minimum-competency exams

* Voluntary end-of-course exams combined with minimum-competency exams, la New York and North Carolina's policy mix during the 1990s.

We also controlled for the following demographic characteristics of the students in each state: the share of children living in poverty, parental education, and the percentages of public school students who are African-American, Hispanic, or Asian-American.

The hybrid end-of-course/minimum-competency exam systems that have been in place in New York State since the early 1980s and in North Carolina since about 1990 clearly had the largest effects on test scores, In science and math, 8th graders in New York and North Carolina were approximately 45 percent of a grade-level equivalent ahead of comparable students in states without such exams. They were also 65 percent of a grade-level equivalent ahead in reading (see Figure 1). This confirms my earlier findings that New York State did significantly better on SAT tests and on the 1992 8th-grade NAEP math tests than other states with demographically similar populations.

High stakes for teachers and schools had significant effects on all three measures of 8th-grade achievement, Students living in states that, during the 1996-97 school year, both rewarded successful schools and threatened to sanction failing schools scored about 28 percent of a grade-level equivalent higher in all three subjects than students in states that did neither. Public reporting is necessary for the execution of these other policies, but on its own it had no discernable effect on student achievement.

The effects of minimum-competency exams on average 8th grade NAEP test scores were positive but small and mainly insignificant. For students who were approaching graduation, however, the effects grew. Analysis of longitudinal data found that students with C- grade-point averages in 8th grade learned about 16 percent of a grade-level equivalent more when they lived in states requiring minimum-competency exams before graduation, Students with higher GPAs were unaffected by minimum-competency exams.

After High School

Studies of 1990 census data at the state level show that increasing the number of courses required to graduate raised dropout rates and reduced graduation rates, Minimum-competency exams had no such effect. When, however, my colleagues and I analyzed longitudinal data that adjusted for the grades and test scores of students in 8th grade, we found that students at schools with minimum-competency exams with C- grades in 8th grade, while not more likely to drop out, were about 7 percentage points less likely to get a high-school diploma or a General Education Diploma (GED) within six years. Minimum-competency exams had no significant effect on the graduation rates of students with A or B/B- averages.

The study of longitudinal data also found that college attendance rates were reduced by higher course graduation requirements, but increased by minimum-competency exams. Eighth graders living in states with a minimum-competency exam were 2 to 4 percentage points more likely to be attending college six years later than were comparable students from states without such exams. 5 Curriculum-based exit exams substantially increased the college-attendance rates of students with low GPAs in 8th grade, but had no effect on students with high GPAs.

Students who grew up in states with minimum-competency exams earned significantly more in the years immediately after graduating than students growing up in other states. Growing up in a state with minimum-competency exams raises by about 11 percent the earnings of those who had low GPAs in 8th grade. Students with high grades in 8th grade earn about 7.5 percent extra when they grow up in a state with minimum-competency exams.

Policy Implications

Our analysis showed that states that reward schools for success and sanction schools that are failing had significantly higher achievement levels than states without these incentives. We also found that they had lower dropout rates, State requirements that students pass minimum-competency exams in order to graduate had both positive and negative effects on students. While students with average or above-average grades were unaffected, students with low grades in 8th grade were less likely to graduate during the next six years. The effects of minimum-competency exams on achievement in 8th grade and test score gains during high school were small and often not statistically significant. But students at both ends of the spectrum--that is, with either high or low grades--were significantly (about 2 to 4 percentage points) more likely to attend college in 1993-94 when they lived in a state with minimum-competency exams, In addition, employers responded to the enhanced reputation of recent high-school graduates by pa ying them about 9 percent more immediately after high school.

Curriculum-based external exit exam systems had by far the greatest effects on test scores. On the negative side, New York students of the early 1990s were mote likely to get GEDs and tended to take longer to get their diplomas. They were not, however, less likely to graduate, and students with low grade-point averages were significantly more likely to go to college. Achievement levels at the end of high school were roughly one grade-level equivalent ahead of comparable states. These are the effects of a voluntary Regents examination system with moderate stakes, not the compulsory high-stakes exam system that New York is now phasing in. States that are reluctant to implement a high-stakes high school graduation test might want to look at the old Regents end-of-course exam system as a possible model for a moderate-stakes student accountability system.

Figure 1

Curriculum-Based External Exit Exams Improve Performance

Studies show that students in countries and states that require students
to pass curriculum-based external exit exams in order to graduate learn
more than their peers who do not take such exams.

Gains for students who take exit exams, expressed in grade-level
equivalents

National Assessment of Educational Progress, Math 0.4
(New York and North Carolina as compared with
other states, 1998)

National Assessment of Educational Progress, Science 0.5
(New York and North Carolina as compared with
other states, 1998)

International Assessment of Educational Progress, 0.5
Math and Science (Canada, 1991)

National Assessment of Educational Progress, Reading 0.7
(New York and North Carolina as compared with
other states, 1998)

International Assessment of Educational Progress, Science 0.7
(15 nations, 1991)

International Assessment of Educational Progress, Math 2.0
(15 nations, 1991)

International Assessment for the Evaluation of Educational 1.0
Achievement, Reading (24 nations, 1990)

Third International Math and Science Study, Math 1.0
(40 nations, 1995)

Third International Math and Science Study, Science 1.3
(40 nations, 1995)

All results are significant at the p < 0.05 level, with the exception of
the International Assessment of Educational Progress, Science. Numbers
are rounded to the nearest 0.1

SOURCE: Author's estimates from these datasets

John H. Bishop is a professor of human resource studies at Cornell University.