文章基本信息

标题：The Effects of Failing to Include Hard-to-Reach Respondents in Longitudinal Surveys
本地全文：下载
作者：Donna H. Odierna ; Laura A. Schmidt
期刊名称：American journal of public health
印刷版ISSN：0090-0036
出版年度：2009
卷号：99
期号：8
页码：1515-1521
DOI：10.2105/AJPH.2007.111138
语种：English
出版社：American Public Health Association
摘要：Objectives. We sought to determine whether failure to locate hard-to-reach respondents in longitudinal studies causes biased and inaccurate study results. Methods. We performed a nonresponse simulation in a survey of 498 low-income women who received cash aid in a California county. Our simulation was based on a previously published analysis that found that women without children who applied for General Assistance experienced more violence than did women with children who applied for Temporary Assistance to Needy Families. We compared hard-to-reach respondents whom we reinterviewed only after extended follow-up effort 12 months after baseline with other respondents. We then removed these hard-to-reach respondents from our analysis. Results. Other than having a greater prevalence of substance dependence (14% vs 6%), there were no significant differences between hard- and easy-to-reach respondents. However, excluding the hard to reach would have decreased response rates from 89% to 71% and nullified the findings, a result that did not stem primarily from reduced statistical power. Conclusions. The effects of failure to retain hard-to-reach respondents are not predicable based on respondent characteristics. Retention of these respondents should be a priority in public health research. Respondents who participate in all phases of longitudinal studies are likely to differ from those who are lost to follow-up.^{1 – 5} Differential attrition, or nonrandom loss of respondents, can lead to bias in a study's findings by changing the composition of the sample so that it no longer represents the study population, especially when response rates are low and there are large differences between responders and nonresponders.⁶ Attrition also reduces sample sizes, contributing to the risk of type 2 error by decreasing statistical power to detect effects.⁷ Although much is written about respondents who are hard to reach and about ways to increase response rates in hard-to-reach populations,^{2 , 8 – 19} an examination of the literature uncovered few precise descriptions of follow-up protocols in longitudinal studies.²⁰ Descriptions were often imprecise, and there was a good deal of variation in the procedures and amount of effort used to track populations considered difficult to reach.^{10 , 21 – 23} Nonetheless, the literature provides recommendations for improving the retention of respondents, including limiting sample sizes to allow for adequate follow-up, offering incentives to study participants, implementing tracking procedures with multiple contact methods and flexible protocols, permitting file sharing among interviewers, engaging the target population at the outset of the project, and allowing an open-ended number of contact attempts.^{3 , 5 , 6 , 16 , 19 , 24} However, many of these methods are costly and time consuming; researchers in a range of US survey centers consistently report that budgetary limitations, more than anything else, determine how much effort is put into tracking respondents.²⁰ Moskowitz suggested that attrition bias is endemic in public health research and may be related to competition for funding and publication.²⁵ Funding agencies are often reluctant to pay for costly extended follow-up efforts, and investigators may contribute to the problem by minimizing study weaknesses that result from attrition.²⁵ Groves²⁶ wrote that although nonresponse bias is a problem in survey research, low response rates do not necessarily introduce bias. He found that different estimates within surveys may be subject to greater bias-related variation than are estimates in surveys with different response rates. Moreover, he warned that “blind pursuit” of high response rates may introduce measurement error and bias, in part because reluctance to participate might affect the validity of respondents' answers. An analysis that examined this idea showed that answers from reluctant respondents were generally consistent with data found in records. Although some measurement error was apparent, overall bias was lower when hard-to-reach and reluctant respondents were included in the sample.²⁷ Methods of increasing response rates without increasing error rates are still being developed,²⁸ and it is unclear whether findings on nonresponse bias in probability sampling can be applied with confidence to attrition bias in longitudinal research. We empirically examined the effects on attrition bias of retaining hard-to-reach respondents in the Welfare Client Longitudinal Study (WCLS),^{29 , 30} a longitudinal survey that used particularly intensive tracking procedures. We compared the characteristics of hard-to-reach respondents with those who were more easily found and examined the effects on response rates and on study results of failure to retain the hard to reach. To this end, we developed systematic procedures for empirically identifying hard-to-reach respondents within the WCLS sample. To determine if the failure to locate hard-to-reach respondents would have affected study conclusions, we performed a nonresponse simulation based on the 2006 study by Lown et al.³¹ on violent victimization among women in the WCLS. That study found that female General Assistance applicants, mostly single and without children, were more likely to be victims of violence than were female Temporary Assistance to Needy Families (TANF) applicants, who were mostly young single mothers. These findings led Lown et al. to conclude that it is misguided to provide domestic violence prevention programs nearly exclusively for women receiving TANF and thus ignore the even more disenfranchised women in poverty who are without children and therefore not eligible for TANF.