Using Performance Assessment in Outcomes-Based Accountability Systems. ERIC Digest E533

Outcomes-based accountability ensures that the educational system is responsible for student attainment of specific learner outcomes. Increasingly, outcomes-based accountability systems are using results of both traditional assessments, such as norm-referenced tests, and authentic or performance assessments in an effort to measure broad domains of student knowledge.PERFORMANCE ASSESSMENT AND STUDENTS WITH DISABILITIESPerformance assessments can offer a number of benefits over the use of traditional standardized assessments for students with disabilities. The most important benefit is the potential for linking instruction and assessment: As the student completes the assessment task, the teacher uses the data to improve instruction.

To ensure success of performance assessments in an outcomes-based system, the following issues must be addressed for students with disabilities:

1. Defining the Outcomes. Assessment programs are constructed to measure progress toward valued educational goals. When outcome frameworks are defined too narrowly (e.g., academic content domains) and neglect other valued areas (e.g., vocational skills, personal management, social skills, and communication), the outcomes may not reflect all of the skills that are valued for students with disabilities. Assessment tasks need to be relevant to the students' program goals.

2. Developing Performance Standards. Standards are benchmarks against which student performance may be compared. A critical decision in designing an assessment system is whether students will be compared to themselves to determine change in their performance over time, or whether they will be compared to fixed standards of performance. Many students with disabilities cannot meet absolute standards, particularly in academic areas. When participation in the assessment program is linked to high school diplomas, students with disabilities may be at a particular disadvantage.

3. Assessment Accommodations. Students with disabilities may benefit from accommodations made during assessment:

*Additional time to complete the task.

*Alternative testing locations.

*Alternative means of administration (e.g., reading, interpretation, Braille).

*Alternative supplies or equipment (e.g., computers).

*Alternative forms of assessment.

4. Scoring. When assessment results must be reported in the aggregate and when results matter, such as for diplomas, scoring reliability becomes critical. Rigorous scoring is as necessary in alternative assessments as in traditional assessments.PERFORMANCE ASSESSMENTS IN ACTION: DESCRIPTIONS OF SELECTED

STATES AND DISTRICTSA number of states and local school districts have adopted performance assessments in their outcomes-based systems. Many of these sites have included students with disabilities in the assessments.

Kentucky: The Kentucky Educational Reform Act outlines six performance goals that all students are expected to attain upon graduation from Kentucky schools:

*Communication and math.

*Core concepts from the sciences, arts, humanities, social studies, and practical living studies.

*Self-sufficiency.

*Membership in family, work group, or community.

*Thinking and problem-solving.

*Connecting and integrating knowledge.

Students are also expected to have mastered 75 outcomes in specific academic content areas.

Performance assessments include:

*Portfolios in writing and mathematics for all students in grades 4, 8, and 12.

*Performance events for all students in grades 4, 8, and 12, with focus on mathematics, science, social studies, arts and humanities, and vocational education/practical living.

*Transitional assessments (open-ended and multiple-choice questions) for all students in grades 4, 8, and 12, with focus on mathematics, science, social studies, arts and humanities, and vocational education/practical living. Alternative portfolios are developed by students with severe disabilities in grades 4, 8, and 12.

All students are required to participate in the transitional assessments or alternative portfolios unless a physician provides a statement documenting significant negative impact on the student's health as a result of participating.

Maryland: The Maryland School Performance Program (MSPP) was developed as a comprehensive student outcomes accountability system. Reflecting state-level goals and strategies, student learning outcomes have been developed in the areas of reading and writing, mathematics, social studies, and science.

Assessment of student outcomes within the MSPP includes:

*Norm-referenced tests (Comprehensive Tests of Basic Skills).

*Criterion-referenced performance assessments (Maryland School Performance Assessment Program).

*Maryland Functional Tests, criterion-referenced minimal competency tests.

An alternative performance assessment is currently being proposed for students with significant cognitive disabilities.

Students with disabilities may be exempted by their IEP team from participation in any of the three assessments.

Vermont: The Vermont Assessment Program was implemented statewide during the 1991-1992 school year. The program employs both standardized assessments and portfolios to collect information about the performance of 4th and 8th graders in mathematics and writing:

*The standardized assessment used is the Uniform Assessment, which includes two 40-item multiple choice tests and a single on-demand writing task that the student must complete independently.

*Each student's portfolio is expected to include 10-20 items. From these, students select 5 to 7 "best pieces" to be scored.

School-based staffing teams determine whether or not students with disabilities are eligible for exemption from the portfolio development and assessments. Exempted students may build portfolios that do not conform to the state requirements for use in instruction.WHAT ARE THE IMPLICATIONS OF USING PERFORMANCE ASSESSMENTS FOR

ACCOUNTABILITY?There are a number of issues that need to be addressed when using performance assessment as part of large scale assessment programs:

*Dealing with increased program costs.

*Ensuring scoring reliability and establishing fair, yet flexible, scoring rubrics.

*Setting performance standards.

*Specifying the outcomes and indicators.

*Making certain that the assessments provide many opportunities for students to demonstrate proficiency in an outcome area.

*Using the assessment results to influence instruction either individually or at the school level.

To address these issues, it is important to include both general and special education teachers in designing and implementing the assessment program.WHAT ARE THE CONSIDERATIONS FOR INCLUDING STUDENTS WITH

DISABILITIES IN OUTCOMES-BASED ACCOUNTABILITY SYSTEMS?Despite the strong impetus to include students with disabilities in assessments, there are still major considerations that must be addressed:

1. Outcomes-based systems present special educators with a difficult conceptual switch from believing that each student with a disability should have individualized outcomes to accepting the notion of a common set of outcomes across students.

2. There is still ambiguity among assessment experts regarding how much accommodation should be provided within an assessment program.

3. When one set of scoring standards is defined for all students, with no modifications made for students with disabilities, students with disabilities may be denied diplomas or otherwise penalized.

4. When results are used for high stakes accountability, there may be greater pressure to exempt students with disabilities. Once the decision to exempt students with disabilities is made, there may also be pressure to identify more students as having disabilities in order to exempt more students from the assessments.STRATEGIES TO SUPPORT USING PERFORMANCE ASSESSMENT IN

OUTCOMES-BASED SYSTEMSWhen using performance assessments in outcomes-based systems, educators can increase the potential for success of students with disabilities by considering the following:

*Identify meaningful outcomes.

*Define performance standards in sufficiently broad terms or in ways that emphasize growth.

*Create enough flexibility in the assessment system to accommodate individual student needs.

*Employ multiple data-gathering strategies including on-demand assessments, examples of student work, and teacher judgments.REFERENCESDerived from McLaughlin, M. J. & Warren, S. H. (1994). Performance Assessment and Students with Disabilities: Usage in Outcomes-Based Accountability Systems. Reston, VA: The Council for Exceptional Children. Product #P5061.

ERIC Digests are in the public domain and may be freely reproduced and disseminated.

This publication was prepared with funding from the National Library of Education (NLE), Office of Educational Research and Improvement, U.S. Department of Education, under contract no. RR93002005. The opinions expressed in this report do not necessarily reflect the positions or policies of NLE, OERI, or the Department of Education. Title: Using Performance Assessment in Outcomes-Based Accountability Systems. ERIC Digest E533. Note: Derived from "Performance Assessment and Students with Disabilities: Usage in Outcomes-Based Accountability Systems," by M. J. McLaughlin and S. H. Warren; see ED 375 568. Document Type: Information Analyses---ERIC Information Analysis Products (IAPs) (071); Information Analyses---ERIC Digests (Selected) in Full Text (073); Available From: Clearinghouse on Disabilities and Gifted Education, Council for Exceptional Children, 1920 Association Dr., Reston, VA 22091-1589 ($1 each; minimum order $5 prepaid). Descriptors: Academic Standards, Accountability, Disabilities, Educational Assessment, Educational Objectives, Elementary Secondary Education, Evaluation Methods, Performance, State Programs, Student Evaluation Identifiers: ERIC Digests, Performance Based Evaluation

###

[Return to ERIC Digest Search Page]