Program building through evaluation - camping

Camp directors sometimes view research and evaluation as technical and far removed from their realm of experience. However, by employing basic evaluation tools, camp directors can incorporate their camp expertise into the evaluation process and reap great benefits for their camp programs.

Over the span of four years, camp director Gary DeVore of Camp White Eagle, Inc. in Illinois evaluated his wilderness adventure camp program for adjudicated adolescents. The primary focus of the annual evaluations was to assess the program's effectiveness in reaching goals and to share the results at the end of the summer with the funder and referral sources. The reports proved invaluable in maintaining funding for several years, in educating referral sources about the potential gains of participants, and in further development of the program.

Program evaluation

Review program elements

Program evaluation begins with a review of the objectives, history, and development of the program. Those who know the program well, including staff, should be involved.

The 15-day camp program at Camp White Eagle is based on Outward Bound adventure camp models. Camp White Eagle's program objectives include youth abiding by behavioral contracts and learning camp skills. Other objectives relate to pre-test and posttest changes in self-esteem, amount and quality of peer and group interactions, and recidivism, which is defined as new delinquency petitions.

At the beginning of each camp session, behavioral contracts are discussed with each camper, who then signs an agreement to abide by basic rules such as self care, self control, completion of chores, and no gang symbols at camp. Camp activities typically involve three- to four-day canoe and bike trips as well as climbing and other activities that allow youth to master new outdoor and camp skills.

Group sessions are held with six to eight campers and two to three counselors. Intensive peer group participation and interactions with counselors provide guided and rich opportunities for the youth to improve communication and interpersonal problem-solving skills. The campers increase their sense of responsibility by completing chores necessary for group functioning.

At the end of the camp session, each camper receives individual feedback from staff. Success is completion of the camp program. A pretest and posttest assessment of self-esteem is taken at the beginning and end of each camp session. About six months after camp, a follow-up with the referring probation officers indicates whether a camper has picked up additional court petitions.

Review camp literature

Compare the elements of your program to other programs through a review of camp literature. By reading theory, practice, and research articles about similar programs, you can draw on others' ideas and research to support your program. Camp White Eagle located literature in academic journals, camp journals, and from a youth-at-risk clearinghouse. We also considered presentations at camp conferences and discussions with other camp directors.

Formulate questions

What are the questions you want to answer about the program? They should be related to your program goals, but more narrowly stated. Typically, questions examine the relationship between the program and its outcomes. A focused and clearly stated research question is important as it determines the design and measurements for your study. Factors you routinely consider in evaluating the success of a camper's experience provide good ideas for research questions.

At Camp White Eagle, we initially identified two research questions for the first evaluation (1992), both defining a relationship between the camp program and specific outcomes.

1. Does the adventure camp experience increase the self esteem of the participants?

2. Does the adventure camp experience reduce recidivism for the campers?

In the process of conducting the study, we identified a third question.

3. Did campers exhibit positive change as a result of completing the adventure camp experience?

Select evaluation design

Your research questions provide guidance for the design of the study. They imply a comparison of before and after measures: in Camp White Eagle's case, self-esteem before and after camp and delinquent behavior before and after camp. The type of research design suitable for this project was a descriptive, pre-experimental model, one group pretest-posttest design. We coded the counselors' narrative summaries of the campers for use in the evaluation of the campers' behavior. (See Figure 1.)

Develop measurements

Standard measurements, or instruments, help you to track and quantify behavior and results.

In the area of self-esteem, Camp White Eagle used the Tennessee Self Concept Scale the first year. This standardized scale, designed for adults and adolescents, had nine subscales and provided an overall score reflecting the camper's level of self-esteem. This scale proved problematic as the campers struggled to comprehend and answer some of the questions.

The next year, we used the Piers Harris Children's Self-Concept Scale. This test had 80 yes/no questions for campers to answer in describing how they felt about themselves. The six subscales included behavior, intellectual and school status, physical appearance and attributes, anxiety, popularity, and happiness and satisfaction. This scale, designed for children, worked better for the campers.

Collect data

Who will collect data is as important as the method of collecting it. Consider those at camp as well as outside camp.

Camp White Eagle asked counselors to prepare a narrative report on each camper. We reviewed the reports to identify factors about the camp experience which were salient for most campers. These included interpersonal communication skills, mastery of camp skills, responsibility, and group interaction. Response to authority (counselors) and overall improvement during camp were added later.

Certain patterns emerged in counselors' reports. Extensive detail was available for campers having a positive or negative camp experience, but less detail was documented for the average camper. The following year, counselors were trained for data collection and asked to always include some comments on selected topics. Even with training, there were ongoing omissions.

In the area of delinquent behavior, each camper was known to be receiving probation services for a current delinquent petition. We asked referring probation officers to describe each camper's court history prior to this probation period and, six months after the camp program, to describe additional court petitions.

Discuss results

Carefully review, analyze, and discuss the implications of your findings.

After Camp White Eagle's evaluation, we examined how the camp program might be improved. We placed increased emphasis on helping the campers learn new coping methods such as interpersonal problem-solving, leadership skills, and attitude modification. The data collection methods were expanded and refined so that patterns in the data would be more obvious.

By the fourth evaluation (1995) we were able to expand the research questions and identify differences between several camp sessions. As a total group the campers showed significant improvement in the behavior, physical appearance, and total self-esteem scores. Recidivism remained constant; 28 percent of the campers returned to court on new offenses. Specific camp behaviors correlated with the campers' response to authority; the groups with experienced counselors improved the most.

Share results with others

You can use your evaluation results to inform funders, educate staff, attract new clientele, and improve your overall camp offerings.

Through its evaluation efforts, Camp White Eagle secured continued funding for the program for several years. The camp director and staff used the information to make changes in the program as well as improve the camp's marketing and public relations efforts.

The greatest benefit of the evaluation was the spillover effect to other summer programs which were enhanced by the fresh perspective of the camp director and staff. Once the process of examining interactions between staff and campers in the adventure camp program began, a similar interactive process was initiated in the counselors-in-training program. Also, improvements in specific program elements, such as canoeing and biking, translated easily into other programs with day trips.

Campers were given more autonomy in planning use of tents, food menus, and other areas of tripping, adding to their skills and increased self-esteem. Counselors initiated daily logs in other programs, providing a more vivid picture of how other groups of campers changed.

The reassessment and changes in one camp program drove changes and growth in other programs. The director and staff began to think in new ways. The evaluation process helped them build better camp programs.

References

Condon, S. (1992). Research report on the incarceration diversion camp project for adjudicated delinquents. [Summer 1991, Camp White Eagle, Inc.] Report to the Illinois Juvenile Justice Commission.

Condon, S. (1993). Research report on the adventure challenge program for juvenile court wards. [Summer 1992. Camp White Eagle, Inc.] Report to the Illinois Juvenile Justice Commission.

Condon, S. (1995). Research report on the situational leadership/attitude modification program for adjudicated delinquents. [Summer 1994. Camp White Eagle, Inc.] Report to the Illinois Juvenile Justice Commission.

Henderson, K.A. & Bialeschki, M.D. (May 1993). Camp was great, but the water was too cold. Camping Magazine 66 (5), 31-36.

Jordan, D.J., Edginton, C.R., & Edginton, S.R. (Nov. 1994). Evaluation: What to measure and why. Camping Magazine 67 (2), 20-24.

Piers, E.V. (1984). Piers-Harris Children's Self Concept Scale. [Revised Manual.] California: Western Psychological Services.

Roid, G.H. & Fitts, W.H. (1991). Tennessee Self Concept Scale (TSCS). [Revised Manual.] California: Western Psychological Services.

Sandra Condon is an assistant professor in the School of Social Work at Loyola University in Chicago. She worked as a consultant to the Camp White Eagle evaluation project. Gary DeVore is the camp director at Camp White Eagle in Leaf River, Ill.

"This project was supported by a grant to the American Camping Association/Illinois Section from the Illinois Juvenile Justice Commission. Points of view or opinions stated in this publication are those of Dr. Sandra Condon and/or Camp White Eagle and do not necessarily represent he official position of American Camping Association/Illinois Section or the Illinois Juvenile Justice Commission, the Illinois Department of Children and Family Services, or the Office of Juvenile Justice and Delinquency Prevention."

COPYRIGHT 1996 American Camping Association
COPYRIGHT 2004 Gale Group