文章基本信息

标题：Competency assessment of an undergraduate program using a third-party, objective pre-post examination
本地全文：下载
作者：Lawrence Fulton ; Cristian Lieneck ; Zo Ramamonjiarivelo 等
期刊名称：BMC Medical Education
印刷版ISSN：1472-6920
出版年度：2021
卷号：21
期号：1
页码：1-13
DOI：10.1186/s12909-020-02448-z
出版社：BioMed Central
摘要：Assessing competencies or program learning outcomes in educational programs is often a leadership challenge. This case study reports medical education program’s efforts to document undergraduate competency attainment using a pre-post, third-party, objective testing service that allows for inter-university comparison, a testing service that is being adopted by some certification and accrediting bodies. Students completed a pre-test after program acceptance and a post-test at the end of the last didactic semester (1.5 years later) just prior to their required internships. Scores and subscores were evaluated using t-tests (Holm-adjusted p-values). MANOVA models of sub-competency difference scores were also evaluated. Results indicate competency improvement for each of the 12 areas based on the n = 55 student sample, (p < .001 for all scores). These improvements were independent of ethnicity, age, gender, and grades. The average student improved by 12.85 points (95% CI of 10.52 to 15.18) with the largest improvements in strategic planning and leadership competency areas (21.30 and 18.33 percentage points, respectively). The third-party pre-post has some face validity given that student performance improved after completing a related curriculum as would be expected. Congruent with earlier studies, we find that repeated testing helps document competency attainment and that a single method for assessment is insufficient. We further document limitations of this 3d-party exam.
其他摘要：Abstract Background Assessing competencies or program learning outcomes in educational programs is often a leadership challenge. This case study reports medical education program’s efforts to document undergraduate competency attainment using a pre-post, third-party, objective testing service that allows for inter-university comparison, a testing service that is being adopted by some certification and accrediting bodies. Methods Students completed a pre-test after program acceptance and a post-test at the end of the last didactic semester (1.5 years later) just prior to their required internships. Scores and subscores were evaluated using t-tests (Holm-adjusted p -values). MANOVA models of sub-competency difference scores were also evaluated. Results Results indicate competency improvement for each of the 12 areas based on the n = 55 student sample, ( p < .001 for all scores). These improvements were independent of ethnicity, age, gender, and grades. The average student improved by 12.85 points (95% CI of 10.52 to 15.18) with the largest improvements in strategic planning and leadership competency areas (21.30 and 18.33 percentage points, respectively). Conclusions The third-party pre-post has some face validity given that student performance improved after completing a related curriculum as would be expected. Congruent with earlier studies, we find that repeated testing helps document competency attainment and that a single method for assessment is insufficient. We further document limitations of this 3d-party exam.
关键词：Competency-based medical education (CBME);Competency assessment;Pre-post testing;Quasi-experimental;Peregrine testing