Development of the National Standards Coaching Efficacy Scale

Past research exploring the effectiveness of coaches has involved a variety of research methodologies and measures. Traditionally, the most common means of evaluating a coach is through his or her win-loss record (Leland, 1988). However, contemporary scholars suggest that win-loss records may not truly reflect the ability of an individual to be an effective coach. Other factors such as leadership (Chelladurai & Saleh, 1980), athlete-coach relationships (Jowett & Ntoumanis, 2004) and coaching efficacy (Feltz, Chase, Moritz, & Sullivan, 1999; Malete & Feltz, 2000; Myers, Feltz, Chase, Reckase & Hancock, 2008) can also play a role in coaching effectiveness. In particular, coaching efficacy has gained much recent attention and has been linked to several salient outcomes including athlete satisfaction (Myers, Vargas-Tonsing, & Feltz, 2005), team efficacy (Vargas-Tonsing, Warners, and Feltz, 2003), commitment to coaching (Feltz, Short & Sullivan, 2008), leadership behaviors (Sullivan, Paquette, Holt & Bloom, 2012), and win-loss records (Feltz et al., 1999; Myers et al, 2005).

Coaching efficacy is a form of self-efficacy. Self-efficacy is defined as "beliefs in one's ability to organize and execute the course of action required to produce a given attainment" (Bandura, 1997, p.3). Self-efficacy is concerned with people's beliefs in their ability to influence events that affect their lives and it is considered the foundation of human motivation and performance accomplishments (Bandura, 1997, 2006). Coaching efficacy is defined "as the extent to which coaches believe they have the capacity to affect the learning and performance of their athletes" (Feltz et al., 1999, p. 765). Feltz and colleagues identified four components of coaching efficacy: game strategy, motivation, technique, and character-building efficacy. These components were developed partially from the National Standards for Athletic Coaches (NASPE, 1995) as well as previous literature on coaching confidence (Park, 1992). Based on this framework, Feltz et al. (1999) developed the Coaching Efficacy Scale (CES), a 24-item questionnaire designed and tested to measure the four dimensions of coaching efficacy. A revised version of the CES for high school team sport coaches (CES II-HST) added a fifth dimension, physical conditioning (Myers et al., 2008).

In 2006, the National Standards for Sport Coaches (NASPE, 2006) was revised to include eight domains (i.e., philosophy and ethics, safety and injury prevention, physical conditioning, growth and development, teaching and communication, sport skills and tactics, organization and administration, and evaluation). These eight domains represent the essential elements for effective coaching of young athletes and serve as the foundation for several coaching education programs (NASPE, 2008). While the previously designed scales (Feltz et al., 1999; Myers et al., 2008) were based in part on the previous national standards, they did not directly measure coaching efficacy associated with each of the eight domains of the latest National Standards for Sport Coaches (NASPE, 2006). A better understanding of coaches' beliefs in their capacity to effectively implement the standards in each of these eight domains would allow coaches and administrators of coaching education programs to recognize specific areas of strength as well as identify areas in need of improvement. Thus, the purpose of this study was to develop and validate a tool to measure coaching efficacy associated with the eight domains of the National Standards for Sport Coaches.

Methods and Results

The development of the National Standards Coaching Efficacy scale (NSCES) was conducted in three phases. Phase I involved the development of the scale items and the measurement of fidelity or the degree to which the scale items measured the specific domains of the National Standards for Sport Coaches (Wright, 2008). Fidelity and appropriateness were verified using a test blueprint to relate each scale item to the eight coaching domains, as well as having items evaluated by a panel of experts in the field of coaching. Phase II tested for commonality or the shared features of another validated instrument (Wright, 2008). This was done by correlating the NSCES with the CES (Feltz et al., 1999). Finally, Phase III was conducted to determine the scale's reliability by using Cronbach's alpha coefficient to assess the internal consistency of each of the eight domain subscales. All procedures were reviewed and approved by the authors' university's institutional review board prior to participant involvement.

Phase I: Item Development

Phase I involved item development for the NSCES. Items for the NSCES were initially developed by the three members of the research team. The research team consisted of a 58 year old white male with over 35 years of coaching experience, a 24 year old white female who was an assistant field hockey coach at a Division I university, and a 50 year old white female with over 20 years of experience as a coach and athletic administrator. Each of the three researchers independently generated five to eight efficacy statements related to each domain of the National Standards for Sport Coaches (NASPE, 2006). They then met to discuss the statements and reached consensus on 50 items addressing coaching efficacy based on the national standards.

After initial item development, items were evaluated by a panel of experts to determine clarity and relevance. The panel included two males and two females ranging in age from 38 to 71 (M = 52.3, SD = 14.6), with coaching experience ranging from 15 to 30 years (M = 20.8, SD = 6.7). Members of the panel were all former or current high school coaches. In addition, one member of the panel was a current athletic director, two members were members of the Virginia High School League (VHSL) coaching education committee, and one member was a university professor who taught courses and conducted research on coaching education. The panel was asked to evaluate the clarity of each item based on a three-point scale and then submit comments regarding clarity. They were also asked to evaluate appropriateness of each item by categorizing it into one of the eight domains of the National Standards for Sport Coaches (NASPE, 2006).

Based on the responses of the panel of experts, each item was categorized as either acceptable (i.e., mean score of 2.5 and above) or unacceptable (i.e., mean score of below 2.5) (Myers et al., 2008). All 50 items were rated as acceptable so no revisions were needed based on this assessment. Next, inter-rater reliability was used to evaluate the appropriateness or fit of each item into its respective domain. Acceptability of each item in the NSCES was based on agreement among at least three out of four panel members, resulting in an inter-rater reliability score of .75 or higher (Miles & Huberman, 1994). At this stage, ten items were eliminated due to low (<.75) inter-rater reliability scores. In the final step of phase I, the primary researcher constructed a survey blueprint which is a matrix to ensure appropriate and equitable coverage of all domains. After phase I, the NSCES included 40 total scale items with four to six items measuring each of the eight domains.

Phase II: Determining Commonality

Once fidelity and appropriateness were established in phase I, the next step was to determine commonality. Commonality was demonstrated by examining the correlation coefficients between the efficacy scores of the NSCES and the previously validated CES (Feltz et al., 1999). To determine commonality, 21 university students (15 male, 6 female; [M.sub.age] = 20.8, SD = 3.3; [M.sub.yearscoaching] = 1.3, SD = 1.7) attending a coaching education course at a mid-Atlantic University were asked to complete both the NSCES and the CES. Two participants were Hispanic, seven were Caucasian, and 12 were African-American. Participants completed both the NSCES and the CES online approximately two weeks apart.

A Pearson product-moment correlation coefficient was computed to assess the relationship between the mean scores of the NSCES and the CES. To demonstrate fidelity and appropriateness within each of the subcategories, or domains, an inter-item correlation was examined. The Pearson product-moment correlation coefficient revealed a positive correlation (r = .824, n = 21, p = .000) between the two scales. The overall correlation between the NSCES and the CES was considered acceptable (Nunnally, 1978). In addition, the inter-item correlation scores were all above .70 which is considered acceptable (see Table 1). The Pearson product-moment correlation coefficients and the inter-item correlations supported the commonality and appropriateness of the NSCES as related to coaching efficacy.

Phase III: Determining Reliability

The final phase of the study was to determine the reliability or internal consistency of the NSCES and its eight subscales. For this phase, individuals who enrolled in the VHSL online coaching education program during the three month period of this investigation were invited to participate in this study. They were provided with information about the study and assured that participation was entirely voluntary. Those who agreed to participate simply clicked on a link to access an online survey with the NSCES questions. A total of 315 coaches (201 male, 80 female, 34 gender not disclosed; ages 19 to 66 [M = 21.5, SD = 13.5) agreed to participate in the study. Participants' coaching experience ranged from 0 to 6 (M = 3.4 SD = 1.6) years.

To determine the internal consistency of the survey instrument as well as each of the subscales representing the eight domains of the National Standards for Sport Coaches, Cronbach's alpha coefficients were calculated. The NSCES produced an overall Cronbach's alpha coefficient of .985, demonstrating a high level of reliability. In addition, Cronbach's alpha scores ranged from .868 to .931 (see Table 2) across subscales. All subscales maintained above acceptable alpha levels (Nunnally, 1978).

Discussion and Conclusion

The purpose of this study was to develop and validate a tool to measure coaching efficacy associated with the eight domains of the National Standards for Sport Coaches (NASPE, 2006). Coaching efficacy has been shown to correlate with a variety of athlete, team and coaching behavior outcomes (Feltz et al., 1999; Feltz et al, 2008; Myers, et al., 2005; Sullivan et al., 2012; Vargas-Tonsing et al, 2003). Most previous research measured coaching efficacy with the CES (Feltz et al., 1999). Although the CES is a valid and reliable instrument, it assesses only four components of coaching efficacy. The current study sought to develop a scale that includes all eight domains of the National Standards for Sport Coaches. By including all eight domains in the NSCES, this measurement tool can assess a broader spectrum of abilities and skills essential to effective coaching. The NSCES can help coaches understand their own strengths and weaknesses related to coaching. Also, it can assist administrators of coaching education programs in identifying possible areas where coaches may not be as confident in their abilities and thus require additional training. Through a thorough three-phase development process, the NSCES was developed, tested, and supported to be a valid and reliable instrument.

As with all research, there are limitations to address. First, the sample sizes for both phase II and III were relatively small. Larger samples could allow researchers to evaluate the factorial validity and composite reliability of the NSCES. In addition, in phase II there were fewer female participants than male participants; further efforts to examine commonality should seek out more female participants to prevent any gender bias. Also, the participants in phase II were college-aged individuals with limited coaching experience, and therefore, future analyses to confirm commonality should be done with individuals with more coaching experience.

In the United States, there has been an increase in coaching education programs over the last decade and many of these programs are based on the 2006 NASPE standards (NASPE, 2008). This current research involved the development of the NSCES as an instrument to measure coaching efficacy related to the National Coaching Standards for Sport Coaches. Although this study demonstrated that the NSCES is a valid and reliable tool, further research conducted with larger samples, different populations, various sports, and equitable representation from both male and female coaches is recommended.

References

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.

Bandura, A. (2006). Toward a psychology of human agency. Perspectives on Psychological Science, 1, 164-180.

Chelladurai, P., & Saleh, S. D. (1980). Dimensions of leader behavior in sports: Development of a leadership scale. Journal of Sport Psychology, 2, 34-45.

Feltz, D. L., Chase, M. A., Moritz, S. E., & Sullivan, P. J. (1999). A conceptual model of coaching efficacy: Preliminary investigation and instrument development. Journal of Educational Psychology, 91, 765-776. doi:10.1037/0022-0663.91.4.765

Feltz, D. L., Short, S. E., & Sullivan, P. J. (2008). Self-efficay in sport: Research and strategies for working with athletes, teams, and coaches. Champaign, IL; Human Kinetics,

Jowett, S., & Ntoumanis, N. (2004). The coach-athlete relationship questionnaire (CART-Q): Development and initial validation. Scandinavian Journal of Medicine & Science in Sports, 14(4), 245-257. doi:10.1111/j.1600-0838.2003.00338.x

Leland, T. (1988). Evaluating coaches--Formalizing the process. Journal of Physical Education, Recreation, & Dance, 59(9), 21-23.

Malete, L., & Feltz, D. L. (2000). The effect of a coaching education program on coaching efficacy. Sport Psychologist, 14(4), 410.

Miles, M. B., & Huberman, A. M. (1994). Qualitative analysis: Am expanded sourcebook (2nd edition). Thousand Oaks, CA: Sage.

Myers, N. D., Feltz, D. L., Chase, M. A., Reckase, M. D., & Hancock, G. R. (2008). The coaching efficacy scale II--high school teams. Educational and Psychological Measurement, 68, 1059-1076. doi:10.1177/0013164408318773

Myers, N. D., Vargas--Tonsing, T. M., & Feltz, D. L. (2005). Coaching efficacy in intercollegiate coaches: Sources, coaching behavior, and team variables. Psychology of Sport and Exercise, 6, 129-143. doi: 10.1016/j.psychsport.2003.10.007

National Association for Sport and Physical Education (1995). National standards for athletic coaches: Quality coaches quality sports. Reston: Author.

National Association for Sport and Physical Education (2006). National standards for sport coaches: Quality coaches quality sports. Reston: Author.

National Association for Sport and Physical Education (2008). National coaching report: The state of coaching in the U.S., Reston: Author.

Nunnally, J. (1978). Psychometric methods. McGraw-Hill, New York, NY.

Park, J. K. (1992). Construction of the coaching confidence Scale. Unpublished doctoral dissertation. Michigan State University, East Lansing.

Sullivan, P., Paquette, K. J., Holt, N. L., & Bloom, G. A. (2012). The relations of coaching context and coach education to coaching efficacy and perceived leadership behaviors in youth sport. The Sport Psychologist, 26, 122-134.

Vargas-Tonsing, T. M., Warners, A. L., & Feltz, D. L. (2003). The predictability of coaching efficacy on team efficacy and player efficacy in volleyball. Journal of Sport Behavior, 26(4), 396-407.

Wright, R.J. (2008). Educational assessments: Tests and measurements in the age of accountability. Thousand Oaks, CA: Sage.

Stephen E. Knott, Senior Lecturer, Old Dominion University

Lynn L. Ridinger, Associate Professor, Old Dominion University

Katelyn S. Makovec, Adjunct Instructor, Old Dominion University

Table 1
Inter-Item Correlations

Domain                             Inter-item Average

Philosophy and Ethics                     .759
Safety and Injury Prevention              .776
Physical Conditioning                     .740
Growth and Development                    .819
Teaching and Communication                .768
Sport Skills and Tactics                  .788
Organization and Administration           .741
Evaluation                                .799

Table 2
Internal Consistency Scores

Domain                              Cronbach's Alpha

Philosophy and Ethics                     .870
Safety and Injury Prevention              .924
Physical Conditioning                     .889
Growth and Development                    .910
Teaching and Communication                .931
Sport Skills and Tactics                  .880
Organization and Administration           .887
Evaluation                                .923